Showing 1–14 of 14 results for author: Dalal, S

Rideshare Transparency: Translating Gig Worker Insights on AI Platform Design to Policy

Authors: Varun Nagaraj Rao, Samantha Dalal, Eesha Agarwal, Dana Calacci, Andrés Monroy-Hernández

Abstract: Rideshare platforms exert significant control over workers through algorithmic systems that can result in financial, emotional, and physical harm. What steps can platforms, designers, and practitioners take to mitigate these negative impacts and meet worker needs? In this paper, through a novel mixed methods study combining a LLM-based analysis of over 1 million comments posted to online platform… ▽ More Rideshare platforms exert significant control over workers through algorithmic systems that can result in financial, emotional, and physical harm. What steps can platforms, designers, and practitioners take to mitigate these negative impacts and meet worker needs? In this paper, through a novel mixed methods study combining a LLM-based analysis of over 1 million comments posted to online platform worker communities with semi-structured interviews of workers, we thickly characterize transparency-related harms, mitigation strategies, and worker needs while validating and contextualizing our findings within the broader worker community. Our findings expose a transparency gap between existing platform designs and the information drivers need, particularly concerning promotions, fares, routes, and task allocation. Our analysis suggests that rideshare workers need key pieces of information, which we refer to as indicators, to make informed work decisions. These indicators include details about rides, driver statistics, algorithmic implementation details, and platform policy information. We argue that instead of relying on platforms to include such information in their designs, new regulations that require platforms to publish public transparency reports may be a more effective solution to improve worker well-being. We offer recommendations for implementing such a policy. △ Less

Submitted 19 June, 2024; v1 submitted 15 June, 2024; originally announced June 2024.

QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forums

Authors: Varun Nagaraj Rao, Eesha Agarwal, Samantha Dalal, Dan Calacci, Andrés Monroy-Hernández

Abstract: Online discussion forums provide crucial data to understand the concerns of a wide range of real-world communities. However, the typical qualitative and quantitative methods used to analyze those data, such as thematic analysis and topic modeling, are infeasible to scale or require significant human effort to translate outputs to human readable forms. This study introduces QuaLLM, a novel LLM-base… ▽ More Online discussion forums provide crucial data to understand the concerns of a wide range of real-world communities. However, the typical qualitative and quantitative methods used to analyze those data, such as thematic analysis and topic modeling, are infeasible to scale or require significant human effort to translate outputs to human readable forms. This study introduces QuaLLM, a novel LLM-based framework to analyze and extract quantitative insights from text data on online forums. The framework consists of a novel prompting methodology and evaluation strategy. We applied this framework to analyze over one million comments from two Reddit's rideshare worker communities, marking the largest study of its type. We uncover significant worker concerns regarding AI and algorithmic platform decisions, responding to regulatory calls about worker insights. In short, our work sets a new precedent for AI-assisted quantitative data analysis to surface concerns from online forums. △ Less

Submitted 8 May, 2024; originally announced May 2024.

Comments: Accepted to CHI LLM as Research Tools Workshop (2024)

Sweeping Arrangements of Non-Piercing Curves in Plane

Authors: Suryendu Dalal, Rahul Gangopadhyay, Rajiv Raman, Saurabh Ray

Abstract: Let $Γ$ be a finite set of Jordan curves in the plane. For any curve $γ\in Γ$, we denote the bounded region enclosed by $γ$ as $\tildeγ$. We say that $Γ$ is a non-piercing family if for any two curves $α, β\in Γ$, $\tildeα \setminus \tildeβ$ is a connected region. A non-piercing family of curves generalizes a family of $2$-intersecting curves in which each pair of curves intersect in at most two p… ▽ More Let $Γ$ be a finite set of Jordan curves in the plane. For any curve $γ\in Γ$, we denote the bounded region enclosed by $γ$ as $\tildeγ$. We say that $Γ$ is a non-piercing family if for any two curves $α, β\in Γ$, $\tildeα \setminus \tildeβ$ is a connected region. A non-piercing family of curves generalizes a family of $2$-intersecting curves in which each pair of curves intersect in at most two points. Snoeyink and Hershberger (``Sweeping Arrangements of Curves'', SoCG '89) proved that if we are given a family $\mathcal{C}$ of $2$-intersecting curves and a fixed curve $C\in\mathcal{C}$, then the arrangement can be \emph{swept} by $C$, i.e., $C$ can be continuously shrunk to any point $p \in \tilde{C}$ in such a way that the we have a family of $2$-intersecting curves throughout the process. In this paper, we generalize the result of Snoeyink and Hershberger to the setting of non-piercing curves. We show that given an arrangement of non-piercing curves $Γ$, and a fixed curve $γ\in Γ$, the arrangement can be swept by $γ$ so that the arrangement remains non-piercing throughout the process. We also give a shorter and simpler proof of the result of Snoeyink and Hershberger and cite applications of their result, where our result leads to a generalization. △ Less

Submitted 2 April, 2024; v1 submitted 25 March, 2024; originally announced March 2024.

The Matrix: A Bayesian learning model for LLMs

Authors: Siddhartha Dalal, Vishal Misra

Abstract: In this paper, we introduce a Bayesian learning model to understand the behavior of Large Language Models (LLMs). We explore the optimization metric of LLMs, which is based on predicting the next token, and develop a novel model grounded in this principle. Our approach involves constructing an ideal generative text model represented by a multinomial transition probability matrix with a prior, and… ▽ More In this paper, we introduce a Bayesian learning model to understand the behavior of Large Language Models (LLMs). We explore the optimization metric of LLMs, which is based on predicting the next token, and develop a novel model grounded in this principle. Our approach involves constructing an ideal generative text model represented by a multinomial transition probability matrix with a prior, and we examine how LLMs approximate this matrix. We discuss the continuity of the mapping between embeddings and multinomial distributions, and present the Dirichlet approximation theorem to approximate any prior. Additionally, we demonstrate how text generation by LLMs aligns with Bayesian learning principles and delve into the implications for in-context learning, specifically explaining why in-context learning emerges in larger models where prompts are considered as samples to be updated. Our findings indicate that the behavior of LLMs is consistent with Bayesian Learning, offering new insights into their functioning and potential applications. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 12 pages, 6 figures

ACM Class: I.2.7

A Cross Attention Approach to Diagnostic Explainability using Clinical Practice Guidelines for Depression

Authors: Sumit Dalal, Deepa Tilwani, Kaushik Roy, Manas Gaur, Sarika Jain, Valerie Shalin, Amit Sheth

Abstract: The lack of explainability using relevant clinical knowledge hinders the adoption of Artificial Intelligence-powered analysis of unstructured clinical dialogue. A wealth of relevant, untapped Mental Health (MH) data is available in online communities, providing the opportunity to address the explainability problem with substantial potential impact as a screening tool for both online and offline ap… ▽ More The lack of explainability using relevant clinical knowledge hinders the adoption of Artificial Intelligence-powered analysis of unstructured clinical dialogue. A wealth of relevant, untapped Mental Health (MH) data is available in online communities, providing the opportunity to address the explainability problem with substantial potential impact as a screening tool for both online and offline applications. We develop a method to enhance attention in popular transformer models and generate clinician-understandable explanations for classification by incorporating external clinical knowledge. Inspired by how clinicians rely on their expertise when interacting with patients, we leverage relevant clinical knowledge to model patient inputs, providing meaningful explanations for classification. This will save manual review time and engender trust. We develop such a system in the context of MH using clinical practice guidelines (CPG) for diagnosing depression, a mental health disorder of global concern. We propose an application-specific language model called ProcesS knowledge-infused cross ATtention (PSAT), which incorporates CPGs when computing attention. Through rigorous evaluation on three expert-curated datasets related to depression, we demonstrate application-relevant explainability of PSAT. PSAT also surpasses the performance of nine baseline models and can provide explanations where other baselines fall short. We transform a CPG resource focused on depression, such as the Patient Health Questionnaire (e.g. PHQ-9) and related questions, into a machine-readable ontology using SNOMED-CT. With this resource, PSAT enhances the ability of models like GPT-3.5 to generate application-relevant explanations. △ Less

Submitted 28 April, 2024; v1 submitted 23 November, 2023; originally announced November 2023.

Understanding Human Intervention in the Platform Economy: A case study of an indie food delivery service

Authors: Samantha Dalal, Ngan Chiem, Nikoo Karbassi, Yuhan Liu, Andrés Monroy-Hernández

Abstract: This paper examines the sociotechnical infrastructure of an "indie" food delivery platform. The platform, Nosh, provides an alternative to mainstream services, such as Doordash and Uber Eats, in several communities in the Western United States. We interviewed 28 stakeholders including restauranteurs, couriers, consumers, and platform administrators. Drawing on infrastructure literature, we learned… ▽ More This paper examines the sociotechnical infrastructure of an "indie" food delivery platform. The platform, Nosh, provides an alternative to mainstream services, such as Doordash and Uber Eats, in several communities in the Western United States. We interviewed 28 stakeholders including restauranteurs, couriers, consumers, and platform administrators. Drawing on infrastructure literature, we learned that the platform is a patchwork of disparate technical systems held together by human intervention. Participants join this platform because they receive greater agency, financial security, and local support. We identify human intervention's key role in making food delivery platform users feel respected. This study provides insights into the affordances, limitations, and possibilities of food delivery platforms designed to prioritize local contexts over transnational scales. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: Accepted at CHI 2023

"Hey, Can You Add Captions?": The Critical Infrastructuring Practices of Neurodiverse People on TikTok

Authors: Ellen Simpson, Samantha Dalal, Bryan Semaan

Abstract: Accessibility efforts, how we can make the world usable and useful to as many people as possible, have explicitly focused on how we can support and allow for the autonomy and independence of people with disabilities, neurotypes, chronic conditions, and older adults. Despite these efforts, not all technology is designed or implemented to support everyone's needs. Recently, a community-organized pus… ▽ More Accessibility efforts, how we can make the world usable and useful to as many people as possible, have explicitly focused on how we can support and allow for the autonomy and independence of people with disabilities, neurotypes, chronic conditions, and older adults. Despite these efforts, not all technology is designed or implemented to support everyone's needs. Recently, a community-organized push by creators and general users of TikTok urged the platform to add accessibility features, such as closed captioning to user-generated content, allowing more people to use the platform with greater ease. Our work focuses on an understudied population -- people with ADHD and those who experience similar challenges -- exploring the creative practices people from this community engage in, focusing on the kinds of accessibility they create through their creative work. Through an interview study exploring the experiences of creatives on TikTok, we find that creatives engage in critical infrastructuring -- a process of bottom-up (re)design -- to make the platform more accessible despite the challenges the platform presents to them as creators. We present these critical infrastructuring practices through the themes of: creating and augmenting video editing infrastructures and creating and augmenting video captioning infrastructures. We reflect on the introduction of a top-down infrastructure - the implementation of an auto-captioning feature - shifts the critical infrastructure practices of content creators. Through their infrastructuring, creatives revised sociotechnical capabilities of TikTok to support their own needs as well as the broader needs of the TikTok community. We discuss how the routine of infrastructuring accessibility is actually best conceptualized as incidental care work. We further highlight how accessibility is an evolving sociotechnical construct, and forward the concept of contextual accessibility. △ Less

Submitted 12 December, 2022; originally announced December 2022.

Comments: To be published in: Proc. ACM Hum.-Comput. Interact. CSCW '23

Finding Emotions in Faces: A Meta-Classifier

Authors: Siddartha Dalal, Sierra Vo, Michael Lesk, Wesley Yuan

Abstract: Machine learning has been used to recognize emotions in faces, typically by looking for 8 different emotional states (neutral, happy, sad, surprise, fear, disgust, anger and contempt). We consider two approaches: feature recognition based on facial landmarks and deep learning on all pixels; each produced 58% overall accuracy. However, they produced different results on different images and thus we… ▽ More Machine learning has been used to recognize emotions in faces, typically by looking for 8 different emotional states (neutral, happy, sad, surprise, fear, disgust, anger and contempt). We consider two approaches: feature recognition based on facial landmarks and deep learning on all pixels; each produced 58% overall accuracy. However, they produced different results on different images and thus we propose a new meta-classifier combining these approaches. It produces far better results with 77% accuracy △ Less

Submitted 20 August, 2022; originally announced August 2022.

ACM Class: I.4.8

Identifying Ransomware Actors in the Bitcoin Network

Authors: Siddhartha Dalal, Zihe Wang, Siddhanth Sabharwal

Abstract: Due to the pseudo-anonymity of the Bitcoin network, users can hide behind their bitcoin addresses that can be generated in unlimited quantity, on the fly, without any formal links between them. Thus, it is being used for payment transfer by the actors involved in ransomware and other illegal activities. The other activity we consider is related to gambling since gambling is often used for transfer… ▽ More Due to the pseudo-anonymity of the Bitcoin network, users can hide behind their bitcoin addresses that can be generated in unlimited quantity, on the fly, without any formal links between them. Thus, it is being used for payment transfer by the actors involved in ransomware and other illegal activities. The other activity we consider is related to gambling since gambling is often used for transferring illegal funds. The question addressed here is that given temporally limited graphs of Bitcoin transactions, to what extent can one identify common patterns associated with these fraudulent activities and apply them to find other ransomware actors. The problem is rather complex, given that thousands of addresses can belong to the same actor without any obvious links between them and any common pattern of behavior. The main contribution of this paper is to introduce and apply new algorithms for local clustering and supervised graph machine learning for identifying malicious actors. We show that very local subgraphs of the known such actors are sufficient to differentiate between ransomware, random and gambling actors with 85% prediction accuracy on the test data set. △ Less

Submitted 28 August, 2021; originally announced August 2021.

Journal ref: MLIOB-2 Conference, 2021

Measuring Data Collection Diligence for Community Healthcare

Authors: Ramesha Karunasena, Mohammad Sarparajul Ambiya, Arunesh Sinha, Ruchit Nagar, Saachi Dalal, Divy Thakkar, Dhyanesh Narayanan, Milind Tambe

Abstract: Data analytics has tremendous potential to provide targeted benefit in low-resource communities, however the availability of high-quality public health data is a significant challenge in developing countries primarily due to non-diligent data collection by community health workers (CHWs). In this work, we define and test a data collection diligence score. This challenging unlabeled data problem is… ▽ More Data analytics has tremendous potential to provide targeted benefit in low-resource communities, however the availability of high-quality public health data is a significant challenge in developing countries primarily due to non-diligent data collection by community health workers (CHWs). In this work, we define and test a data collection diligence score. This challenging unlabeled data problem is handled by building upon domain expert's guidance to design a useful data representation of the raw data, using which we design a simple and natural score. An important aspect of the score is relative scoring of the CHWs, which implicitly takes into account the context of the local area. The data is also clustered and interpreting these clusters provides a natural explanation of the past behavior of each data collector. We further predict the diligence score for future time steps. Our framework has been validated on the ground using observations by the field monitors of our partner NGO in India. Beyond the successful field test, our work is in the final stages of deployment in the state of Rajasthan, India. △ Less

Submitted 7 April, 2021; v1 submitted 5 November, 2020; originally announced November 2020.

ACM Class: I.2.1

Deep Learning to Quantify Pulmonary Edema in Chest Radiographs

Authors: Steven Horng, Ruizhi Liao, Xin Wang, Sandeep Dalal, Polina Golland, Seth J Berkowitz

Abstract: Purpose: To develop a machine learning model to classify the severity grades of pulmonary edema on chest radiographs. Materials and Methods: In this retrospective study, 369,071 chest radiographs and associated radiology reports from 64,581 (mean age, 51.71; 54.51% women) patients from the MIMIC-CXR chest radiograph dataset were included. This dataset was split into patients with and without con… ▽ More Purpose: To develop a machine learning model to classify the severity grades of pulmonary edema on chest radiographs. Materials and Methods: In this retrospective study, 369,071 chest radiographs and associated radiology reports from 64,581 (mean age, 51.71; 54.51% women) patients from the MIMIC-CXR chest radiograph dataset were included. This dataset was split into patients with and without congestive heart failure (CHF). Pulmonary edema severity labels from the associated radiology reports were extracted from patients with CHF as four different ordinal levels: 0, no edema; 1, vascular congestion; 2, interstitial edema; and 3, alveolar edema. Deep learning models were developed using two approaches: a semi-supervised model using a variational autoencoder and a pre-trained supervised learning model using a dense neural network. Receiver operating characteristic curve analysis was performed on both models. Results: The area under the receiver operating characteristic curve (AUC) for differentiating alveolar edema from no edema was 0.99 for the semi-supervised model and 0.87 for the pre-trained models. Performance of the algorithm was inversely related to the difficulty in categorizing milder states of pulmonary edema (shown as AUCs for semi-supervised model and pre-trained model, respectively): 2 versus 0, 0.88 and 0.81; 1 versus 0, 0.79 and 0.66; 3 versus 1, 0.93 and 0.82; 2 versus 1, 0.69 and 0.73; and, 3 versus 2, 0.88 and 0.63. Conclusion: Deep learning models were trained on a large chest radiograph dataset and could grade the severity of pulmonary edema on chest radiographs with high performance. △ Less

Submitted 7 January, 2021; v1 submitted 13 August, 2020; originally announced August 2020.

Comments: The two first authors contributed equally

Semi-supervised Learning for Quantification of Pulmonary Edema in Chest X-Ray Images

Authors: Ruizhi Liao, Jonathan Rubin, Grace Lam, Seth Berkowitz, Sandeep Dalal, William Wells, Steven Horng, Polina Golland

Abstract: We propose and demonstrate machine learning algorithms to assess the severity of pulmonary edema in chest x-ray images of congestive heart failure patients. Accurate assessment of pulmonary edema in heart failure is critical when making treatment and disposition decisions. Our work is grounded in a large-scale clinical dataset of over 300,000 x-ray images with associated radiology reports. While e… ▽ More We propose and demonstrate machine learning algorithms to assess the severity of pulmonary edema in chest x-ray images of congestive heart failure patients. Accurate assessment of pulmonary edema in heart failure is critical when making treatment and disposition decisions. Our work is grounded in a large-scale clinical dataset of over 300,000 x-ray images with associated radiology reports. While edema severity labels can be extracted unambiguously from a small fraction of the radiology reports, accurate annotation is challenging in most cases. To take advantage of the unlabeled images, we develop a Bayesian model that includes a variational auto-encoder for learning a latent representation from the entire image set trained jointly with a regressor that employs this representation for predicting pulmonary edema severity. Our experimental results suggest that modeling the distribution of images jointly with the limited labels improves the accuracy of pulmonary edema scoring compared to a strictly supervised approach. To the best of our knowledge, this is the first attempt to employ machine learning algorithms to automatically and quantitatively assess the severity of pulmonary edema in chest x-ray images. △ Less

Submitted 9 April, 2019; v1 submitted 27 February, 2019; originally announced February 2019.

DigiLock: User-controlled and Server-aware Digital Locker System

Authors: Atrayee Deb, Saloni Dalal, Manik Lal Das

Abstract: The growing popularity of digital systems have paved the way for digital locker that ensures security and safety of the digital documents in store. While facilitating this system to user and availing its services offered by service provider, non-repudiation of service offered and service consumed is an important security requirement in the digital locker system. In this paper, we present a digital… ▽ More The growing popularity of digital systems have paved the way for digital locker that ensures security and safety of the digital documents in store. While facilitating this system to user and availing its services offered by service provider, non-repudiation of service offered and service consumed is an important security requirement in the digital locker system. In this paper, we present a digital locker system that addresses the aspect of confidentiality, integrity, and non-repudiation along with other security properties. The proposed protocol ensures the confirmed participation of the user as well as the service provider while accessing the digital locker. The protocol is analyzed against potential threats in the context of safety and security of the digital locker system. △ Less

Submitted 19 October, 2018; originally announced October 2018.

Comments: 10 pages

A Testbed for Experimenting Internet of Things Applications

Authors: Parthkumar Patel, Jayraj Dave, Shreedhar Dalal, Pankesh Patel, Sanjay Chaudhary

Abstract: The idea of IoT world has grown to multiple dimensions enclosing different technologies and standards which can provide solutions and goal oriented intelligence to the widespread things via network or internet. In spite of different advancement in technology, challenges related to assessment of IoT solutions under real scenarios and empirical deployments still hinder their evolvement and significa… ▽ More The idea of IoT world has grown to multiple dimensions enclosing different technologies and standards which can provide solutions and goal oriented intelligence to the widespread things via network or internet. In spite of different advancement in technology, challenges related to assessment of IoT solutions under real scenarios and empirical deployments still hinder their evolvement and significant expansion. To design a system that can adequately bolster substantial range of applications and be compliant with superfluity of divergent requirements and also integrating heterogeneous technologies is a difficult task. Thus, simulations and testing to design robust applications becomes paramount elements of a development process. For this, there rises a need of a tool or a methodology to test and manage the applications. This paper presents a novel approach by proposing a testbed for experimenting Internet of Things (IoT) applications. An idea of an open source test bed helps in developing an exploited and sustainable smart system. In order to validate the idea of such testbed we have also implemented two use cases. △ Less

Submitted 22 May, 2017; originally announced May 2017.

Comments: 13 pages