-
Fairness Concerns in App Reviews: A Study on AI-based Mobile Apps
Authors:
Ali Rezaei Nasab,
Maedeh Dashti,
Mojtaba Shahin,
Mansooreh Zahedi,
Hourieh Khalajzadeh,
Chetan Arora,
Peng Liang
Abstract:
Fairness is one of the socio-technical concerns that must be addressed in AI-based systems. Unfair AI-based systems, particularly unfair AI-based mobile apps, can pose difficulties for a significant proportion of the global population. This paper aims to analyze fairness concerns in AI-based app reviews. We first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-f…
▽ More
Fairness is one of the socio-technical concerns that must be addressed in AI-based systems. Unfair AI-based systems, particularly unfair AI-based mobile apps, can pose difficulties for a significant proportion of the global population. This paper aims to analyze fairness concerns in AI-based app reviews. We first manually constructed a ground-truth dataset, including 1,132 fairness and 1,473 non-fairness reviews. Leveraging the ground-truth dataset, we developed and evaluated a set of machine learning and deep learning models that distinguish fairness reviews from non-fairness reviews. Our experiments show that our best-performing model can detect fairness reviews with a precision of 94%. We then applied the best-performing model on approximately 9.5M reviews collected from 108 AI-based apps and identified around 92K fairness reviews. Next, applying the K-means clustering technique to the 92K fairness reviews, followed by manual analysis, led to the identification of six distinct types of fairness concerns (e.g., 'receiving different quality of features and services in different platforms and devices' and 'lack of transparency and fairness in dealing with user-generated content'). Finally, the manual analysis of 2,248 app owners' responses to the fairness reviews identified six root causes (e.g., 'copyright issues') that app owners report to justify fairness concerns.
△ Less
Submitted 20 June, 2024; v1 submitted 15 January, 2024;
originally announced January 2024.
-
A Study of Gender Discussions in Mobile Apps
Authors:
Mojtaba Shahin,
Mansooreh Zahedi,
Hourieh Khalajzadeh,
Ali Rezaei Nasab
Abstract:
Mobile software apps ("apps") are one of the prevailing digital technologies that our modern life heavily depends on. A key issue in the development of apps is how to design gender-inclusive apps. Apps that do not consider gender inclusion, diversity, and equality in their design can create barriers (e.g., excluding some of the users because of their gender) for their diverse users. While there ha…
▽ More
Mobile software apps ("apps") are one of the prevailing digital technologies that our modern life heavily depends on. A key issue in the development of apps is how to design gender-inclusive apps. Apps that do not consider gender inclusion, diversity, and equality in their design can create barriers (e.g., excluding some of the users because of their gender) for their diverse users. While there have been some efforts to develop gender-inclusive apps, a lack of deep understanding regarding user perspectives on gender may prevent app developers and owners from identifying issues related to gender and proposing solutions for improvement. Users express many different opinions about apps in their reviews, from sharing their experiences, and reporting bugs, to requesting new features. In this study, we aim at unpacking gender discussions about apps from the user perspective by analysing app reviews. We first develop and evaluate several Machine Learning (ML) and Deep Learning (DL) classifiers that automatically detect gender reviews (i.e., reviews that contain discussions about gender). We apply our ML and DL classifiers on a manually constructed dataset of 1,440 app reviews from the Google App Store, composing 620 gender reviews and 820 non-gender reviews. Our best classifier achieves an F1-score of 90.77%. Second, our qualitative analysis of a randomly selected 388 out of 620 gender reviews shows that gender discussions in app reviews revolve around six topics: App Features, Appearance, Content, Company Policy and Censorship, Advertisement, and Community. Finally, we provide some practical implications and recommendations for developing gender-inclusive apps.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
Understanding the Issues, Their Causes and Solutions in Microservices Systems: An Empirical Study
Authors:
Muhammad Waseem,
Peng Liang,
Aakash Ahmad,
Arif Ali Khan,
Mojtaba Shahin,
Pekka Abrahamsson,
Ali Rezaei Nasab,
Tommi Mikkonen
Abstract:
Many small to large organizations have adopted the Microservices Architecture (MSA) style to develop and deliver their core businesses. Despite the popularity of MSA in the software industry, there is a limited evidence-based and thorough understanding of the types of issues (e.g., errors, faults, failures, and bugs) that microservices system developers experience, the causes of the issues, and th…
▽ More
Many small to large organizations have adopted the Microservices Architecture (MSA) style to develop and deliver their core businesses. Despite the popularity of MSA in the software industry, there is a limited evidence-based and thorough understanding of the types of issues (e.g., errors, faults, failures, and bugs) that microservices system developers experience, the causes of the issues, and the solutions as potential fixing strategies to address the issues. To ameliorate this gap, we conducted a mixed-methods empirical study that collected data from 2,641 issues from the issue tracking systems of 15 open-source microservices systems on GitHub, 15 interviews, and an online survey completed by 150 practitioners from 42 countries across 6 continents. Our analysis led to comprehensive taxonomies for the issues, causes, and solutions. The findings of this study inform that Technical Debt, Continuous Integration and Delivery, Exception Handling, Service Execution and Communication, and Security are the most dominant issues in microservices systems. Furthermore, General Programming Errors, Missing Features and Artifacts, and Invalid Configuration and Communication are the main causes behind the issues. Finally, we found 177 types of solutions that can be applied to fix the identified issues. Based on our study results, we formulated future research directions that could help researchers and practitioners to engineer emergent and next-generation microservices systems.
△ Less
Submitted 11 July, 2023; v1 submitted 3 February, 2023;
originally announced February 2023.
-
Which bugs are missed in code reviews: An empirical study on SmartSHARK dataset
Authors:
F. Khoshnoud,
A. Rezaei Nasab,
Z. Toudeji,
A. Sami
Abstract:
In pull-based development systems, code reviews and pull request comments play important roles in improving code quality. In such systems, reviewers attempt to carefully check a piece of code by different unit tests. Unfortunately, sometimes they miss bugs in their review of pull requests, which lead to quality degradations of the systems. In other words, disastrous consequences occur when bugs ar…
▽ More
In pull-based development systems, code reviews and pull request comments play important roles in improving code quality. In such systems, reviewers attempt to carefully check a piece of code by different unit tests. Unfortunately, sometimes they miss bugs in their review of pull requests, which lead to quality degradations of the systems. In other words, disastrous consequences occur when bugs are observed after merging the pull requests. The lack of a concrete understanding of these bugs led us to investigate and categorize them. In this research, we try to identify missed bugs in pull requests of SmartSHARK dataset projects. Our contribution is twofold. First, we hypothesized merged pull requests that have code reviews, code review comments, or pull request comments after merging, may have missed bugs after the code review. We considered these merged pull requests as candidate pull requests having missed bugs. Based on our assumption, we obtained 3,261 candidate pull requests from 77 open-source GitHub projects. After two rounds of restrictive manual analysis, we found 187 bugs missed in 173 pull requests. In the first step, we found 224 buggy pull requests containing missed bugs after merging the pull requests. Secondly, we defined and finalized a taxonomy that is appropriate for the bugs that we found and then found the distribution of bug categories after analysing those pull requests all over again. The categories of missed bugs in pull requests and their distributions are: semantic (51.34%), build (15.5%), analysis checks (9.09%), compatibility (7.49%), concurrency (4.28%), configuration (4.28%), GUI (2.14%), API (2.14%), security (2.14%), and memory (1.6%).
△ Less
Submitted 19 May, 2022;
originally announced May 2022.
-
Human Values Violations in Stack Overflow: An Exploratory Study
Authors:
Sara Krishtul,
Mojtaba Shahin,
Humphrey O. Obie,
Hourieh Khalajzadeh,
Fan Gai,
Ali Rezaei Nasab,
John Grundy
Abstract:
A growing number of software-intensive systems are being accused of violating or ignoring human values (e.g., privacy, inclusion, and social responsibility), and this poses great difficulties to individuals and society. Such violations often occur due to the solutions employed and decisions made by developers of such systems that are misaligned with user values. Stack Overflow is the most popular…
▽ More
A growing number of software-intensive systems are being accused of violating or ignoring human values (e.g., privacy, inclusion, and social responsibility), and this poses great difficulties to individuals and society. Such violations often occur due to the solutions employed and decisions made by developers of such systems that are misaligned with user values. Stack Overflow is the most popular QA website among developers to share their issues, solutions (e.g., code snippets), and decisions during software development. We conducted an exploratory study to investigate the occurrence of human values violations in Stack Overflow posts. As comments under posts are often used to point out the possible issues and weaknesses of the posts, we analyzed 2000 Stack Overflow comments and their corresponding posts (1980 unique questions or answers) to identify the types of human values violations and the reactions of Stack Overflow users to such violations. Our study finds that 315 out of 2000 comments contain concerns indicating their associated posts (313 unique posts) violate human values. Leveraging Schwartz's theory of basic human values as the most widely used values model, we show that hedonism and benevolence are the most violated value categories. We also find the reaction of Stack Overflow commenters to perceived human values violations is very quick, yet the majority of posts (76.35%) accused of human values violation do not get downvoted at all. Finally, we find that the original posters rarely react to the concerns of potential human values violations by editing their posts. At the same time, they usually are receptive when responding to these comments in follow-up comments of their own.
△ Less
Submitted 20 March, 2022;
originally announced March 2022.
-
An Empirical Study of Security Practices for Microservices Systems
Authors:
Ali Rezaei Nasab,
Mojtaba Shahin,
Seyed Ali Hoseyni Raviz,
Peng Liang,
Amir Mashmool,
Valentina Lenarduzzi
Abstract:
Despite the numerous benefits of microservices systems, security has been a critical issue in such systems. Several factors explain this difficulty, including a knowledge gap among microservices practitioners on properly securing a microservices system. To (partially) bridge this gap, we conducted an empirical study. We first manually analyzed 861 microservices security points, including 567 issue…
▽ More
Despite the numerous benefits of microservices systems, security has been a critical issue in such systems. Several factors explain this difficulty, including a knowledge gap among microservices practitioners on properly securing a microservices system. To (partially) bridge this gap, we conducted an empirical study. We first manually analyzed 861 microservices security points, including 567 issues, 9 documents, and 3 wiki pages from 10 GitHub open-source microservices systems and 306 Stack Overflow posts concerning security in microservices systems. In this study, a microservices security point is referred to as "a GitHub issue, a Stack Overflow post, a document, or a wiki page that entails 5 or more microservices security paragraphs". Our analysis led to a catalog of 28 microservices security practices. We then ran a survey with 74 microservices practitioners to evaluate the usefulness of these 28 practices. Our findings demonstrate that the survey respondents affirmed the usefulness of the 28 practices. We believe that the catalog of microservices security practices can serve as a valuable resource for microservices practitioners to more effectively address security issues in microservices systems. It can also inform the research community of the required or less explored areas to develop microservices-specific security practices and tools.
△ Less
Submitted 18 November, 2022; v1 submitted 30 December, 2021;
originally announced December 2021.
-
Quotients of span categories that are allegories and the representation of regular categories
Authors:
S. Naser Hosseini,
Amir R. Shir Ali Nasab,
Walter Tholen,
Leila Yeganeh
Abstract:
We consider the ordinary category Span(C) of (isomorphism classes of) spans of morphisms in a category C with finite limits as needed, composed horizontally via pullback, and give a general criterion for a quotient of Span(C) to be an allegory. In particular, when C carries a pullback-stable, but not necessarily proper, (E, M)-factorization system, we establish a quotient category Span_E(C) that i…
▽ More
We consider the ordinary category Span(C) of (isomorphism classes of) spans of morphisms in a category C with finite limits as needed, composed horizontally via pullback, and give a general criterion for a quotient of Span(C) to be an allegory. In particular, when C carries a pullback-stable, but not necessarily proper, (E, M)-factorization system, we establish a quotient category Span_E(C) that is isomorphic to the category Rel_M(C) of M-relations in C, and show that it is a (unitary and tabular) allegory precisely when M is a class of monomorphisms in C. Without this restriction, one can still find a least pullback-stable and composition-closed class E. containing E such that Span_E.(C) is a unitary and tabular allegory. In this way one obtains a left adjoint to the 2-functor that assigns to every unitary and tabular allegory the regular category of its Lawverian maps. With the Freyd-Scedrov Representation Theorem for regular categories, we conclude that every finitely complete category with a stable factorization system has a reflection into the huge 2-category of all regular categories.
△ Less
Submitted 8 December, 2021;
originally announced December 2021.
-
A Qualitative Study of Architectural Design Issues in DevOps
Authors:
Mojtaba Shahin,
Ali Rezaei Nasab,
Muhammad Ali Babar
Abstract:
Software architecture is critical in succeeding with DevOps. However, designing software architectures that enable and support DevOps (DevOps-driven software architectures) is a challenge for organizations. We assert that one of the essential steps towards characterizing DevOps-driven architectures is to understand architectural design issues raised in DevOps. At the same time, some of the archite…
▽ More
Software architecture is critical in succeeding with DevOps. However, designing software architectures that enable and support DevOps (DevOps-driven software architectures) is a challenge for organizations. We assert that one of the essential steps towards characterizing DevOps-driven architectures is to understand architectural design issues raised in DevOps. At the same time, some of the architectural issues that emerge in the DevOps context (and their corresponding architectural practices or tactics) may stem from the context (i.e., domain) and characteristics of software organizations. To this end, we conducted a mixed-methods study that consists of a qualitative case study of two teams in a company during their DevOps transformation and a content analysis of Stack Overflow and DevOps Stack Exchange posts to understand architectural design issues in DevOps. Our study found eight specific and contextual architectural design issues faced by the two teams and classified architectural design issues discussed in Stack Overflow and DevOps Stack Exchange into 11 groups. Our aggregated results reveal that the main characteristics of DevOps-driven architectures are: being loosely coupled and prioritizing deployability, testability, supportability, and modifiability over other quality attributes. Finally, we discuss some concrete implications for research and practice.
△ Less
Submitted 12 November, 2021; v1 submitted 15 August, 2021;
originally announced August 2021.
-
Automated Identification of Security Discussions in Microservices Systems: Industrial Surveys and Experiments
Authors:
Ali Rezaei Nasab,
Mojtaba Shahin,
Peng Liang,
Mohammad Ehsan Basiri,
Seyed Ali Hoseyni Raviz,
Hourieh Khalajzadeh,
Muhammad Waseem,
Amine Naseri
Abstract:
Lack of awareness and knowledge of microservices-specific security challenges and solutions often leads to ill-informed security decisions in microservices system development. We claim that identifying and leveraging security discussions scattered in existing microservices systems can partially close this gap. We define security discussion as "a paragraph from developer discussions that includes d…
▽ More
Lack of awareness and knowledge of microservices-specific security challenges and solutions often leads to ill-informed security decisions in microservices system development. We claim that identifying and leveraging security discussions scattered in existing microservices systems can partially close this gap. We define security discussion as "a paragraph from developer discussions that includes design decisions, challenges, or solutions relating to security". We first surveyed 67 practitioners and found that securing microservices systems is a unique challenge and that having access to security discussions is useful for making security decisions. The survey also confirms the usefulness of potential tools that can automatically identify such security discussions. We developed fifteen machine/deep learning models to automatically identify security discussions. We applied these models on a manually constructed dataset consisting of 4,813 security discussions and 12,464 non-security discussions. We found that all the models can effectively identify security discussions: an average precision of 84.86%, recall of 72.80%, F1-score of 77.89%, AUC of 83.75% and G-mean 82.77%. DeepM1, a deep learning model, performs the best, achieving above 84% in all metrics and significantly outperforms three baselines. Finally, the practitioners' feedback collected from a validation survey reveals that security discussions identified by DeepM1 have promising applications in practice.
△ Less
Submitted 21 July, 2021;
originally announced July 2021.
-
On the Nature of Issues in Five Open Source Microservices Systems: An Empirical Study
Authors:
Muhammad Waseem,
Peng Liang,
Mojtaba Shahin,
Aakash Ahmad,
Ali Rezaei Nasab
Abstract:
Due to its enormous benefits, the research and industry communities have shown an increasing interest in the Microservices Architecture (MSA) style over the last few years. Despite this, there is a limited evidence-based and thorough understanding of the types of issues (e.g., faults, errors, failures, mistakes) faced by microservices system developers and causes that trigger the issues. Such evid…
▽ More
Due to its enormous benefits, the research and industry communities have shown an increasing interest in the Microservices Architecture (MSA) style over the last few years. Despite this, there is a limited evidence-based and thorough understanding of the types of issues (e.g., faults, errors, failures, mistakes) faced by microservices system developers and causes that trigger the issues. Such evidence-based understanding of issues and causes is vital for long-term, impactful, and quality research and practice in the MSA style. To that end, we conducted an empirical study on 1,345 issue discussions extracted from five open source microservices systems hosted on GitHub. Our analysis led to the first of its kind taxonomy of the types of issues in open source microservices systems, informing that the problems originating from Technical debt (321, 23.86%), Build (145, 10.78%), Security (137, 10.18%), and Service execution and communication (119, 8.84%) are prominent. We identified that "General programming errors", "Poor security management", "Invalid configuration and communication", and "Legacy versions, compatibility and dependency" are the predominant causes for the leading four issue categories. Study results streamline a taxonomy of issues, their mapping with underlying causes, and present empirical findings that could facilitate research and development on emerging and next-generation microservices systems.
△ Less
Submitted 4 May, 2021; v1 submitted 25 April, 2021;
originally announced April 2021.
