research-article

Privacy and Security Evaluation of Mobile Payment Applications Through User-Generated Reviews

Authors:

Urvashi Kishnani,

Rinku DewriAuthors Info & Claims

WPES'22: Proceedings of the 21st Workshop on Privacy in the Electronic Society

Pages 159 - 173

https://doi.org/10.1145/3559613.3563196

Published: 07 November 2022 Publication History

Abstract

Mobile payment applications are crucial to ensure seamless day-to-day digital transactions. However, users' perceived privacy- and security-related concerns are continually rising. Users express such thoughts, complaints, and suggestions through app reviews. To this aim, we collected 1,886,352 reviews from the top 50 mobile payment applications. Furthermore, we conducted a mixed-methods in-depth evaluation of the privacy- and security-related reviews resulting in a total of 163,210 reviews. Finally, we implemented sentiment analysis and did a mixed-methods analysis of the resulting 52,749 negative reviews. Such large-scale evaluation through user reviews informs developers about the user perception of digital threats and app behaviors. Our analysis highlights that users share concerns about sharing sensitive information with the application, confidentiality of their data, and permissions requested by the apps. Users have shown significant concerns regarding the usability of these applications (48.47%), getting locked out of their accounts (38.73%), and being unable to perform successful digital transactions (31.52%). We conclude by providing actionable recommendations to address such user concerns to aid the development of secure and privacy-preserving mobile payment applications.

References

[1]

Aijn Abraham, D Schlecht, G Ma, M Dobrushin, and V Nadal. 2016. Mobile security framework (MobSF).

[2]

Yasemin Acar, Sascha Fahl, and Michelle L Mazurek. 2016. You are not your developer, either: A research agenda for usable security and privacy research beyond end users. 2016 IEEE Cybersecurity Development (SecDev) (2016), 3--8.

[3]

Shivani Agarwal, Mitesh Khapra, Bernard Menezes, and Nirav Uchat. 2007. Security issues in mobile payment systems. Computer Society of India (2007), 142--152.

[4]

Hazim Almuhimedi, Florian Schaub, Norman Sadeh, Idris Adjerid, Alessandro Acquisti, Joshua Gluck, Lorrie Faith Cranor, and Yuvraj Agarwal. 2015. Your location has been shared 5,398 times! A field study on mobile app privacy nudging. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 787--796.

Digital Library

[5]

Yoris A Au and Robert J Kauffman. 2008. The economics of mobile payments: Understanding stakeholder issues for an emerging financial technology application. Electronic commerce research and applications 7, 2 (2008), 141--164.

[6]

Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason I Hong, and Lorrie Faith Cranor. 2014. The privacy and security behaviors of smartphone app developers. (2014).

[7]

Zlatko Bezovski. 2016. The future of the mobile payment as electronic payment system. European Journal of Business and Management 8, 8 (2016), 127--132.

[8]

Monica Caraway, Daniel A Epstein, and Sean A Munson. 2017. Friends don't need receipts: The curious case of social awareness streams in the mobile payment app Venmo. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1--17.

Digital Library

[9]

Lei Cen, Deguang Kong, Hongxia Jin, and Luo Si. 2015. Mobile app security risk assessment: A crowdsourcing ranking approach from user comments. In Proceedings of the 2015 SIAM International Conference on Data Mining. SIAM, 658--666.

[10]

Ramesh Chandran, S Rakesh Kumar, and N Gayathri. 2021. Designing a locating scams for mobile transaction with the aid of operational activity analysis in cloud. Wireless Personal Communications 117, 4 (2021), 3015--3028.

Digital Library

[11]

Cybercrimemag. 2021. Cybercrime to cost the world 10.5 trillion annually by 2025. https://cybersecurityventures.com/hackerpocalypse-cybercrime-report-2016/

[12]

Tomi Dahlberg, Niina Mallat, Jan Ondrus, and Agnieszka Zmijewska. 2008. Past, present and future of mobile payments research: A literature review. Electronic commerce research and applications 7, 2 (2008), 165--181.

[13]

Sanchari Das, Andrew Dingman, and L Jean Camp. 2018. Why Johnny doesn't use two factor a two-phase usability study of the FIDO U2F security key. In International Conference on Financial Cryptography and Data Security. Springer, 160--179.

Digital Library

[14]

Sanchari Das, Robert S Gutzwiller, Rod D Roscoe, Prashanth Rajivan, YangWang, L Jean Camp, and Roberto Hoyle. 2020. Humans and technology for inclusive privacy and security. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 64. SAGE Publications Sage CA: Los Angeles, CA, 461--464.

[15]

Sanchari Das, Bingxing Wang, Zachary Tingle, and L Jean Camp. 2019. Evaluating User Perception of Multi-Factor Authentication: A Systematic Review. In Proceedings of the Thirteenth International Symposium on Human Aspects of Information Security & Assurance (HAISA 2019).

[16]

Jose M Del Alamo, Danny Guaman, Belen Balmori, and Ana Diez. 2021. Privacy Assessment in Android Apps: A Systematic Mapping Study. Electronics 10, 16 (2021), 1999.

[17]

Jayati Dev, Sanchari Das, and Linda Jean Camp. 2018. Privacy Practices, Preferences, and Compunctions: WhatsApp Users in India. In HAISA. 135--146.

[18]

Yvonne D Eaves. 2001. A synthesis technique for grounded theory data analysis. Journal of advanced nursing 35, 5 (2001), 654--663.

[19]

Adrienne Porter Felt, Elizabeth Ha, Serge Egelman, Ariel Haney, Erika Chin, and David Wagner. 2012. Android permissions: User attention, comprehension, and behavior. In Proceedings of the eighth symposium on usable privacy and security. 1--14.

Digital Library

[20]

Denzil Ferreira, Vassilis Kostakos, Alastair R Beresford, Janne Lindqvist, and Anind K Dey. 2015. Securacy: an empirical investigation of Android applications' network usage, privacy and security. In Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks. 1--11.

Digital Library

[21]

Elizabeth Fife and Juan Orjuela. 2012. The privacy calculus: Mobile apps and user perceptions of privacy and security. International Journal of Engineering Business Management 4, Godi?te 2012 (2012), 4--11.

[22]

Barbara L Filkins, Ju Young Kim, Bruce Roberts, Winston Armstrong, Mark A Miller, Michael L Hultner, Anthony P Castillo, Jean-Christophe Ducom, Eric J Topol, and Steven R Steinhubl. 2016. Privacy and security in the era of digital health: what should translational researchers know and do about it? American journal of translational research 8, 3 (2016), 1560.

[23]

Peter Flach and Meelis Kull. 2015. Precision-recall-gain curves: PR analysis done right. Advances in neural information processing systems 28 (2015).

[24]

Xiang Gong, Kem ZK Zhang, Chongyang Chen, Christy MK Cheung, and Matthew KO Lee. 2019. What drives self-disclosure in mobile payment applications? The effect of privacy assurance approaches, network externality, and technology complementarity. Information Technology & People (2019).

[25]

Android Developers Guide. [n.d.]. App Security Best Practices: Android Developers. https://developer.android.com/topic/security/best-practices

[26]

Hongmu Han, Ruixuan Li, and Xiwu Gu. 2016. Identifying malicious Android apps using permissions and system events. International Journal of Embedded Systems 8, 1 (2016), 46--58.

[27]

Yijun Huang and Wenwen Liu. 2012. The impact of privacy concern on users' usage intention of mobile payment. In 2012 International Conference on Information Management, Innovation Management and Industrial Engineering, Vol. 3. IEEE, 90--93.

[28]

Johannes Huebner, Remo Manuel Frey, Christian Ammendola, Elgar Fleisch, and Alexander Ilic. 2018. What people like in mobile finance apps: An analysis of user reviews. In Proceedings of the 17th International Conference on Mobile and Ubiquitous Multimedia. 293--304.

Digital Library

[29]

Jesús Téllez Isaac and Zeadally Sherali. 2014. Secure mobile payment systems. It professional 16, 3 (2014), 36--43.

[30]

Vess L Johnson, Angelina Kiser, Ronald Washington, and Russell Torres. 2018. Limitations to the rapid adoption of M-payment services: Understanding the impact of privacy risk on M-Payment services. Computers in Human Behavior 79 (2018), 111--122.

Digital Library

[31]

Monisha Kanakaraj and Ram Mohana Reddy Guddeti. 2015. Performance analysis of Ensemble methods on Twitter sentiment analysis using NLP techniques. In Proceedings of the 2015 IEEE 9th international conference on semantic computing (IEEE ICSC 2015). IEEE, 169--170.

[32]

Mubasher Khalid, Muhammad Asif, and Usman Shehzaib. 2015. Towards improving the quality of mobile app reviews. International Journal of Information Technology and Computer Science (IJITCS) 7, 10 (2015), 35.

[33]

Mubasher Khalid, Usman Shehzaib, and Muhammad Asif. 2015. A Case of Mobile App Reviews as a Crowdsource. International Journal of Information Engineering & Electronic Business 7, 5 (2015).

[34]

Burhan Ul Islam Khan, Rashidah F Olanrewaju, Asifa Mehraj Baba, Adil Ahmad Langoo, and Shahul Assad. 2017. A compendious study of online payment systems: Past developments, present impact, and future considerations. International journal of advanced computer science and applications 8, 5 (2017).

[35]

Kat Krol, Eleni Philippou, Emiliano De Cristofaro, and M Angela Sasse. 2015. "They brought in the horrible key ring thing!" Analysing the Usability of Two- Factor Authentication in UK Online Banking. arXiv preprint arXiv:1501.04434 (2015).

[36]

Francisco Liébana-Cabanillas, Francisco Muñoz-Leiva, and Juan Sánchez- Fernández. 2018. A global approach to the analysis of user behavior in mobile payment systems in the new electronic environment. Service Business 12, 1 (2018), 25--64.

[37]

Yu-Cheng Lin. 2015. Androbugs framework: An android application security vulnerability scanner. Blackhat Europe 2015 (2015).

[38]

Sanam Ghorbani Lyastani, Michael Schilling, Michaela Neumayr, Michael Backes, and Sven Bugiel. 2020. Is FIDO2 the Kingslayer of User Authentication? A Comparative Usability Study of FIDO2 Passwordless Authentication. In IEEE Symposium on Security and Privacy. 268--285.

[39]

Hafiz Abid Mahmood Malik, Abdulhafeez Muhammad, and Usama Sajid. 2021. Analyzing Usability of Mobile Banking Applications in Pakistan. Sukkur IBA Journal of Computing and Mathematical Sciences 5, 2 (2021), 25--35.

[40]

Niina Mallat. 2007. Exploring consumer adoption of mobile payments--A qualitative study. The Journal of Strategic Information Systems 16, 4 (2007), 413--432.

Digital Library

[41]

Stuart McIlroy, Nasir Ali, Hammad Khalid, and Ahmed E Hassan. 2016. Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empirical Software Engineering 21, 3 (2016), 1067--1106.

Digital Library

[42]

Alessio Merlo and Gabriel Claudiu Georgiu. 2017. Riskindroid: Machine learningbased risk analysis on android. In Ifip international conference on ict systems security and privacy protection. Springer, 538--552.

[43]

Nurul Momen, Majid Hatamian, and Lothar Fritsch. 2019. Did app privacy improve after the GDPR? IEEE Security & Privacy 17, 6 (2019), 10--20.

Digital Library

[44]

Seema Nambiar, C-T Lu, and Lily R Liang. 2004. Analysis of payment transaction security in mobile commerce. In Proceedings of the 2004 IEEE International Conference on Information Reuse and Integration, 2004. IRI 2004. IEEE, 475--480.

[45]

Fatih Nayebi, Jean-Marc Desharnais, and Alain Abran. 2012. The state of the art of mobile application usability evaluation. In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE, 1--4.

[46]

Naheem Noah, Sommer Shearer, and Sanchari Das. 2022. Security and Privacy Evaluation of Popular Augmented and Virtual Reality Technologies. In Proceedings of the 2022 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence, and Neural Engineering (IEEE MetroXRAINE 2022).

[47]

Abu Saleh Md Noman, Sanchari Das, and Sameer Patil. 2019. Techies against Facebook: understanding negative sentiment toward Facebook via user generated content. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--15.

Digital Library

[48]

Sai Teja Peddinti, Igor Bilogrevic, Nina Taft, Martin Pelikan, Úlfar Erlingsson, Pauline Anthonysamy, and Giles Hogben. 2019. Reducing permission requests in mobile apps. In Proceedings of the internet measurement conference. 259--266.

Digital Library

[49]

Minh Vu Phong, Tam The Nguyen, Hung Viet Pham, and Tung Thanh Nguyen. 2015. Mining user opinions in mobile app reviews: A keyword-based approach . In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 749--759.

Digital Library

[50]

Nikolaos Polatidis and Christos K Georgiadis. 2013. Mobile recommender systems: An overview of technologies and challenges. In 2013 Second International Conference on Informatics & Applications (ICIA). IEEE, 282--287.

[51]

Sören Preibusch, Thomas Peetz, Gunes Acar, and Bettina Berendt. 2016. Shopping for privacy: Purchase details leaked to PayPal. Electronic Commerce Research and Applications 15 (2016), 52--64.

Digital Library

[52]

Rahime Belen Saglam, Jason RC Nurse, and Duncan Hodges. 2022. Personal information: Perceptions, types and evolution. Journal of Information Security and Applications 66 (2022), 103163.

Digital Library

[53]

Zakaria Sahnoune, Esma Aïmeur, Ghada El Haddad, and Rodrigue Sokoudjou. 2015. Watch your mobile payment: an empirical study of privacy disclosure. In 2015 IEEE Trustcom/BigDataSE/ISPA, Vol. 1. IEEE, 934--941.

[54]

Maria Shitkova, Justus Holler, Tobias Heide, Nico Clever, and Jörg Becker. 2015. Towards usability guidelines for mobile websites and applications. (2015).

[55]

Nadiah Mohamad Sofian, Ahmad Sobri Hashim, and Wan Fatimah Wan Ahmad. 2018. A review on usability guidelines for designing mobile apps user interface for children with autism. In AIP conference proceedings, Vol. 2016. AIP Publishing LLC, 020094.

[56]

Statista. [n.d.]. Digital payments - worldwide: Statista market forecast. https: //www.statista.com/outlook/dmo/fintech/digital-payments/worldwide

[57]

Chuanqi Tao, Hongjing Guo, and Zhiqiu Huang. 2020. Identifying security issues for mobile applications based on user review summarization. Information and Software Technology 122 (2020), 106290.

[58]

Welderufael B Tesfay, Peter Hofmann, Toru Nakamura, Shinsaku Kiyomoto, and Jetzabel Serna. 2018. PrivacyGuide: towards an implementation of the EU GDPR on internet privacy policy evaluation. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics. 15--21.

Digital Library

[59]

Rajesh Vasa, Leonard Hoon, Kon Mouzakis, and Akihiro Noguchi. 2012. A preliminary analysis of mobile app user reviews. In Proceedings of the 24th Australian computer-human interaction conference. 241--244.

Digital Library

[60]

Silas Formunyuy Verkijika and Brownhilder Ngek Neneh. 2021. Standing up for or against: A text-mining study on the recommendation of mobile payment apps. Journal of Retailing and Consumer Services 63 (2021), 102743.

[61]

Timothy Vidas, Nicolas Christin, and Lorrie Cranor. 2011. Curbing android permission creep. In Proceedings of the Web, Vol. 2.

[62]

YongWang, Christen Hahn, and Kruttika Sutrave. 2016. Mobile payment security, threats, and challenges. In 2016 second international conference on mobile and secure services (MobiSecServ). IEEE, 1--5.

[63]

Xuetao Wei, Lorenzo Gomez, Iulian Neamtiu, and Michalis Faloutsos. 2012. Permission evolution in the android ecosystem. In Proceedings of the 28th Annual Computer Security Applications Conference. 31--40.

Digital Library

[64]

Xuetao Wei and Michael Wolf. 2017. A survey on HTTPS implementation by Android apps: issues and countermeasures. Applied Computing and Informatics 13, 2 (2017), 101--117.

[65]

Charles Weir, Ben Hermann, and Sascha Fahl. 2020. From needs to actions to secure apps? the effect of requirements and developer practices on app security. In 29th USENIX Security Symposium (USENIX Security 20). 289--305.

[66]

Qianwen Yang, Xiang Gong, Kem ZK Zhang, Hefu Liu, and Matthew KO Lee. 2020. Self-disclosure in mobile payment applications: Common and differential effects of personal and proxy control enhancing mechanisms. International Journal of Information Management 52 (2020), 102065.

Digital Library

[67]

Razieh Nokhbeh Zaeem, Ahmad Ahbab, Josh Bestor, Hussam H Djadi, Sunny Kharel, Victor Lai, Nick Wang, and K Suzanne Barber. 2022. PrivacyCheck v3: Empowering Users with Higher-Level Understanding of Privacy Policies. In WSDM. 1593--1596.

[68]

Hengshu Zhu, Hui Xiong, Yong Ge, and Enhong Chen. 2014. Mobile app recommendations with security and privacy awareness. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 951--960.

Digital Library

Cited By

Surani ABawaked AWheeler Mkelsey BRoberts NVincent DDas S(2023)Security and Privacy of Digital Mental Health: An Analysis of Web Services and Mobile AppsSSRN Electronic Journal10.2139/ssrn.4469981Online publication date: 2023
https://doi.org/10.2139/ssrn.4469981
Kim JPark JOakley I(2023)SonarAuth: Using Around Device Sensing to Improve Smartwatch Behavioral BiometricsAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610696(83-87)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3610696

Index Terms

Privacy and Security Evaluation of Mobile Payment Applications Through User-Generated Reviews
1. Security and privacy

Recommendations

User Perceptions of the Security of Mobile Applications

With the exponential increase in the use of mobile devices across the globe, there is a concomitant need to understand how mobile users perceive the security of mobile application, and the potential risks involved in accessing and downloading them. ...
Privacy concerns for mobile app download

In the mobile age, protecting users' information from privacy-invasive apps becomes increasingly critical. To precaution users against possible privacy risks, a few Android app stores prominently disclose app permission requests on app download pages. ...
Privacy Nudges for Mobile Applications: Effects on the Creepiness Emotion and Privacy Attitudes
CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing

To assist users' privacy decision-making with mobile applications, prior research has investigated ways of enhancing information transparency, via improving privacy permission interfaces. This study takes a soft paternalism approach by proposing two ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WPES'22: Proceedings of the 21st Workshop on Privacy in the Electronic Society

November 2022

227 pages

ISBN:9781450398732

DOI:10.1145/3559613

Program Chairs:
Yuan Hong
University of Connecticut, USA
,
Lingyu Wang
Concordia University, Canada

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '22

Sponsor:

SIGSAC

CCS '22: 2022 ACM SIGSAC Conference on Computer and Communications Security

November 7, 2022

CA, Los Angeles, USA

Acceptance Rates

Overall Acceptance Rate 106 of 355 submissions, 30%

Upcoming Conference

CCS '24

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 14 - 18, 2024

Salt Lake City , UT , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
296
Total Downloads

Downloads (Last 12 months)130
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Surani ABawaked AWheeler Mkelsey BRoberts NVincent DDas S(2023)Security and Privacy of Digital Mental Health: An Analysis of Web Services and Mobile AppsSSRN Electronic Journal10.2139/ssrn.4469981Online publication date: 2023
https://doi.org/10.2139/ssrn.4469981
Kim JPark JOakley I(2023)SonarAuth: Using Around Device Sensing to Improve Smartwatch Behavioral BiometricsAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610696(83-87)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3594739.3610696

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents