-
Accuracy-Privacy Trade-off in the Mitigation of Membership Inference Attack in Federated Learning
Authors:
Sayyed Farid Ahamed,
Soumya Banerjee,
Sandip Roy,
Devin Quinn,
Marc Vucovich,
Kevin Choi,
Abdul Rahman,
Alison Hu,
Edward Bowen,
Sachin Shetty
Abstract:
Over the last few years, federated learning (FL) has emerged as a prominent method in machine learning, emphasizing privacy preservation by allowing multiple clients to collaboratively build a model while keeping their training data private. Despite this focus on privacy, FL models are susceptible to various attacks, including membership inference attacks (MIAs), posing a serious threat to data co…
▽ More
Over the last few years, federated learning (FL) has emerged as a prominent method in machine learning, emphasizing privacy preservation by allowing multiple clients to collaboratively build a model while keeping their training data private. Despite this focus on privacy, FL models are susceptible to various attacks, including membership inference attacks (MIAs), posing a serious threat to data confidentiality. In a recent study, Rezaei \textit{et al.} revealed the existence of an accuracy-privacy trade-off in deep ensembles and proposed a few fusion strategies to overcome it. In this paper, we aim to explore the relationship between deep ensembles and FL. Specifically, we investigate whether confidence-based metrics derived from deep ensembles apply to FL and whether there is a trade-off between accuracy and privacy in FL with respect to MIA. Empirical investigations illustrate a lack of a non-monotonic correlation between the number of clients and the accuracy-privacy trade-off. By experimenting with different numbers of federated clients, datasets, and confidence-metric-based fusion strategies, we identify and analytically justify the clear existence of the accuracy-privacy trade-off.
△ Less
Submitted 26 July, 2024;
originally announced July 2024.
-
PathAlign: A vision-language model for whole slide images in histopathology
Authors:
Faruk Ahmed,
Andrew Sellergren,
Lin Yang,
Shawn Xu,
Boris Babenko,
Abbi Ward,
Niels Olson,
Arash Mohtashamian,
Yossi Matias,
Greg S. Corrado,
Quang Duong,
Dale R. Webster,
Shravya Shetty,
Daniel Golden,
Yun Liu,
David F. Steiner,
Ellery Wulczyn
Abstract:
Microscopic interpretation of histopathology images underlies many important diagnostic and treatment decisions. While advances in vision-language modeling raise new opportunities for analysis of such images, the gigapixel-scale size of whole slide images (WSIs) introduces unique challenges. Additionally, pathology reports simultaneously highlight key findings from small regions while also aggrega…
▽ More
Microscopic interpretation of histopathology images underlies many important diagnostic and treatment decisions. While advances in vision-language modeling raise new opportunities for analysis of such images, the gigapixel-scale size of whole slide images (WSIs) introduces unique challenges. Additionally, pathology reports simultaneously highlight key findings from small regions while also aggregating interpretation across multiple slides, often making it difficult to create robust image-text pairs. As such, pathology reports remain a largely untapped source of supervision in computational pathology, with most efforts relying on region-of-interest annotations or self-supervision at the patch-level. In this work, we develop a vision-language model based on the BLIP-2 framework using WSIs paired with curated text from pathology reports. This enables applications utilizing a shared image-text embedding space, such as text or image retrieval for finding cases of interest, as well as integration of the WSI encoder with a frozen large language model (LLM) for WSI-based generative text capabilities such as report generation or AI-in-the-loop interactions. We utilize a de-identified dataset of over 350,000 WSIs and diagnostic text pairs, spanning a wide range of diagnoses, procedure types, and tissue types. We present pathologist evaluation of text generation and text retrieval using WSI embeddings, as well as results for WSI classification and workflow prioritization (slide-level triaging). Model-generated text for WSIs was rated by pathologists as accurate, without clinically significant error or omission, for 78% of WSIs on average. This work demonstrates exciting potential capabilities for language-aligned WSI embeddings.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
A Teacher Is Worth A Million Instructions
Authors:
Nikhil Kothari,
Ravindra Nayak,
Shreyas Shetty,
Amey Patil,
Nikesh Garera
Abstract:
Large Language Models(LLMs) have shown exceptional abilities, yet training these models can be quite challenging. There is a strong dependence on the quality of data and finding the best instruction tuning set. Further, the inherent limitations in training methods create substantial difficulties to train relatively smaller models with 7B and 13B parameters. In our research, we suggest an improved…
▽ More
Large Language Models(LLMs) have shown exceptional abilities, yet training these models can be quite challenging. There is a strong dependence on the quality of data and finding the best instruction tuning set. Further, the inherent limitations in training methods create substantial difficulties to train relatively smaller models with 7B and 13B parameters. In our research, we suggest an improved training method for these models by utilising knowledge from larger models, such as a mixture of experts (8x7B) architectures. The scale of these larger models allows them to capture a wide range of variations from data alone, making them effective teachers for smaller models. Moreover, we implement a novel post-training domain alignment phase that employs domain-specific expert models to boost domain-specific knowledge during training while preserving the model's ability to generalise. Fine-tuning Mistral 7B and 2x7B with our method surpasses the performance of state-of-the-art language models with more than 7B and 13B parameters: achieving up to $7.9$ in MT-Bench and $93.04\%$ on AlpacaEval.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Towards a Personal Health Large Language Model
Authors:
Justin Cosentino,
Anastasiya Belyaeva,
Xin Liu,
Nicholas A. Furlotte,
Zhun Yang,
Chace Lee,
Erik Schenck,
Yojan Patel,
Jian Cui,
Logan Douglas Schneider,
Robby Bryant,
Ryan G. Gomes,
Allen Jiang,
Roy Lee,
Yun Liu,
Javier Perez,
Jameson K. Rogers,
Cathy Speed,
Shyam Tailor,
Megan Walker,
Jeffrey Yu,
Tim Althoff,
Conor Heneghan,
John Hernandez,
Mark Malhotra
, et al. (9 additional authors not shown)
Abstract:
In health, most large language model (LLM) research has focused on clinical tasks. However, mobile and wearable devices, which are rarely integrated into such tasks, provide rich, longitudinal data for personal health monitoring. Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from Gemini for understanding and reasoning over numerical time-series personal health data. We…
▽ More
In health, most large language model (LLM) research has focused on clinical tasks. However, mobile and wearable devices, which are rarely integrated into such tasks, provide rich, longitudinal data for personal health monitoring. Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from Gemini for understanding and reasoning over numerical time-series personal health data. We created and curated three datasets that test 1) production of personalized insights and recommendations from sleep patterns, physical activity, and physiological responses, 2) expert domain knowledge, and 3) prediction of self-reported sleep outcomes. For the first task we designed 857 case studies in collaboration with domain experts to assess real-world scenarios in sleep and fitness. Through comprehensive evaluation of domain-specific rubrics, we observed that Gemini Ultra 1.0 and PH-LLM are not statistically different from expert performance in fitness and, while experts remain superior for sleep, fine-tuning PH-LLM provided significant improvements in using relevant domain knowledge and personalizing information for sleep insights. We evaluated PH-LLM domain knowledge using multiple choice sleep medicine and fitness examinations. PH-LLM achieved 79% on sleep and 88% on fitness, exceeding average scores from a sample of human experts. Finally, we trained PH-LLM to predict self-reported sleep quality outcomes from textual and multimodal encoding representations of wearable data, and demonstrate that multimodal encoding is required to match performance of specialized discriminative models. Although further development and evaluation are necessary in the safety-critical personal health domain, these results demonstrate both the broad knowledge and capabilities of Gemini models and the benefit of contextualizing physiological data for personal health applications as done with PH-LLM.
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Logic-Skill Programming: An Optimization-based Approach to Sequential Skill Planning
Authors:
Teng Xue,
Amirreza Razmjoo,
Suhan Shetty,
Sylvain Calinon
Abstract:
Recent advances in robot skill learning have unlocked the potential to construct task-agnostic skill libraries, facilitating the seamless sequencing of multiple simple manipulation primitives (aka. skills) to tackle significantly more complex tasks. Nevertheless, determining the optimal sequence for independently learned skills remains an open problem, particularly when the objective is given sole…
▽ More
Recent advances in robot skill learning have unlocked the potential to construct task-agnostic skill libraries, facilitating the seamless sequencing of multiple simple manipulation primitives (aka. skills) to tackle significantly more complex tasks. Nevertheless, determining the optimal sequence for independently learned skills remains an open problem, particularly when the objective is given solely in terms of the final geometric configuration rather than a symbolic goal. To address this challenge, we propose Logic-Skill Programming (LSP), an optimization-based approach that sequences independently learned skills to solve long-horizon tasks. We formulate a first-order extension of a mathematical program to optimize the overall cumulative reward of all skills within a plan, abstracted by the sum of value functions. To solve such programs, we leverage the use of tensor train factorization to construct the value function space, and rely on alternations between symbolic search and skill value optimization to find the appropriate skill skeleton and optimal subgoal sequence. Experimental results indicate that the obtained value functions provide a superior approximation of cumulative rewards compared to state-of-the-art reinforcement learning methods. Furthermore, we validate LSP in three manipulation domains, encompassing both prehensile and non-prehensile primitives. The results demonstrate its capability to identify the optimal solution over the full logic and geometric path. The real-robot experiments showcase the effectiveness of our approach to cope with contact uncertainty and external disturbances in the real world.
△ Less
Submitted 16 July, 2024; v1 submitted 7 May, 2024;
originally announced May 2024.
-
Advancing Multimodal Medical Capabilities of Gemini
Authors:
Lin Yang,
Shawn Xu,
Andrew Sellergren,
Timo Kohlberger,
Yuchen Zhou,
Ira Ktena,
Atilla Kiraly,
Faruk Ahmed,
Farhad Hormozdiari,
Tiam Jaroensri,
Eric Wang,
Ellery Wulczyn,
Fayaz Jamil,
Theo Guidroz,
Chuck Lau,
Siyuan Qiao,
Yun Liu,
Akshay Goel,
Kendall Park,
Arnav Agharwal,
Nick George,
Yang Wang,
Ryutaro Tanno,
David G. T. Barrett,
Wei-Hung Weng
, et al. (22 additional authors not shown)
Abstract:
Many clinical tasks require an understanding of specialized data, such as medical images and genomics, which is not typically found in general-purpose large multimodal models. Building upon Gemini's multimodal models, we develop several models within the new Med-Gemini family that inherit core capabilities of Gemini and are optimized for medical use via fine-tuning with 2D and 3D radiology, histop…
▽ More
Many clinical tasks require an understanding of specialized data, such as medical images and genomics, which is not typically found in general-purpose large multimodal models. Building upon Gemini's multimodal models, we develop several models within the new Med-Gemini family that inherit core capabilities of Gemini and are optimized for medical use via fine-tuning with 2D and 3D radiology, histopathology, ophthalmology, dermatology and genomic data. Med-Gemini-2D sets a new standard for AI-based chest X-ray (CXR) report generation based on expert evaluation, exceeding previous best results across two separate datasets by an absolute margin of 1% and 12%, where 57% and 96% of AI reports on normal cases, and 43% and 65% on abnormal cases, are evaluated as "equivalent or better" than the original radiologists' reports. We demonstrate the first ever large multimodal model-based report generation for 3D computed tomography (CT) volumes using Med-Gemini-3D, with 53% of AI reports considered clinically acceptable, although additional research is needed to meet expert radiologist reporting quality. Beyond report generation, Med-Gemini-2D surpasses the previous best performance in CXR visual question answering (VQA) and performs well in CXR classification and radiology VQA, exceeding SoTA or baselines on 17 of 20 tasks. In histopathology, ophthalmology, and dermatology image classification, Med-Gemini-2D surpasses baselines across 18 out of 20 tasks and approaches task-specific model performance. Beyond imaging, Med-Gemini-Polygenic outperforms the standard linear polygenic risk score-based approach for disease risk prediction and generalizes to genetically correlated diseases for which it has never been trained. Although further development and evaluation are necessary in the safety-critical medical domain, our results highlight the potential of Med-Gemini across a wide range of medical tasks.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Asymmetric drift in MaNGA: Mass and radially-dependent stratification rates in galaxy disks
Authors:
Matthew A. Bershady,
Kyle B. Westfall,
Shravan Shetty,
David R. Law,
Michele Cappellari,
Niv Drory,
Kevin Bundy,
Renbin Yan
Abstract:
We measure the age-velocity relationship from the lag between ionized gas and stellar tangential speeds in ~500 nearby disk galaxies from MaNGA in SDSS-IV. Selected galaxies are kinematically axisymmetric. Velocity lags are asymmetric drift, seen in the Milky Way's (MW) solar neighborhood and other Local Group galaxies; their amplitude correlates with stellar population age. The trend is qualitati…
▽ More
We measure the age-velocity relationship from the lag between ionized gas and stellar tangential speeds in ~500 nearby disk galaxies from MaNGA in SDSS-IV. Selected galaxies are kinematically axisymmetric. Velocity lags are asymmetric drift, seen in the Milky Way's (MW) solar neighborhood and other Local Group galaxies; their amplitude correlates with stellar population age. The trend is qualitatively consistent in rate (d(sigma)/dt) with a simple power-law model where sigma is proportional to t^b that explains the dynamical phase-space stratification in the solar neighborhood. The model is generalized based on disk dynamical times to other radii and other galaxies. We find in-plane radial stratification parameters sigma_(0,r} (dispersion of the youngest populations) in the range of 10-40 km/s and 0.2<b_r<0.5 for MaNGA galaxies. Overall b_r increases with galaxy mass, decreases with radius for galaxies above 10.4 dex (M_solar) in stellar mass, but is ~constant with radius at lower mass. The measurement scatter indicates the stratification model is too simple to capture the complexity seen in the data, unsurprising given the many possible astrophysical processes that may lead to stellar population dynamical stratification. Nonetheless, the data show dynamical stratification is broadly present in the galaxy population, with systematic trends in mass and density. The amplitude of the asymmetric drift signal is larger for the MaNGA sample than the MW, and better represented in the mean by what is observed in the disks of M31 and M33. Either typical disks have higher surface-density or, more likely, are dynamically hotter (hence thicker) than the MW.
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
-
A Zero Trust Framework for Realization and Defense Against Generative AI Attacks in Power Grid
Authors:
Md. Shirajum Munir,
Sravanthi Proddatoori,
Manjushree Muralidhara,
Walid Saad,
Zhu Han,
Sachin Shetty
Abstract:
Understanding the potential of generative AI (GenAI)-based attacks on the power grid is a fundamental challenge that must be addressed in order to protect the power grid by realizing and validating risk in new attack vectors. In this paper, a novel zero trust framework for a power grid supply chain (PGSC) is proposed. This framework facilitates early detection of potential GenAI-driven attack vect…
▽ More
Understanding the potential of generative AI (GenAI)-based attacks on the power grid is a fundamental challenge that must be addressed in order to protect the power grid by realizing and validating risk in new attack vectors. In this paper, a novel zero trust framework for a power grid supply chain (PGSC) is proposed. This framework facilitates early detection of potential GenAI-driven attack vectors (e.g., replay and protocol-type attacks), assessment of tail risk-based stability measures, and mitigation of such threats. First, a new zero trust system model of PGSC is designed and formulated as a zero-trust problem that seeks to guarantee for a stable PGSC by realizing and defending against GenAI-driven cyber attacks. Second, in which a domain-specific generative adversarial networks (GAN)-based attack generation mechanism is developed to create a new vulnerability cyberspace for further understanding that threat. Third, tail-based risk realization metrics are developed and implemented for quantifying the extreme risk of a potential attack while leveraging a trust measurement approach for continuous validation. Fourth, an ensemble learning-based bootstrap aggregation scheme is devised to detect the attacks that are generating synthetic identities with convincing user and distributed energy resources device profiles. Experimental results show the efficacy of the proposed zero trust framework that achieves an accuracy of 95.7% on attack vector generation, a risk measure of 9.61% for a 95% stable PGSC, and a 99% confidence in defense against GenAI-driven attack.
△ Less
Submitted 10 March, 2024;
originally announced March 2024.
-
HeAR -- Health Acoustic Representations
Authors:
Sebastien Baur,
Zaid Nabulsi,
Wei-Hung Weng,
Jake Garrison,
Louis Blankemeier,
Sam Fishman,
Christina Chen,
Sujay Kakarmath,
Minyoi Maimbolwa,
Nsala Sanjase,
Brian Shuma,
Yossi Matias,
Greg S. Corrado,
Shwetak Patel,
Shravya Shetty,
Shruthi Prabhakara,
Monde Muyoyeta,
Diego Ardila
Abstract:
Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other t…
▽ More
Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other tasks. To mitigate these gaps, we develop HeAR, a scalable self-supervised learning-based deep learning system using masked autoencoders trained on a large dataset of 313 million two-second long audio clips. Through linear probes, we establish HeAR as a state-of-the-art health audio embedding model on a benchmark of 33 health acoustic tasks across 6 datasets. By introducing this work, we hope to enable and accelerate further health acoustics research.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Experimental investigation on the effect of temperature on the frequency limit of GaAs-AlGaAs and AlGaN-GaN 2DEG Hall-effect sensors
Authors:
Anand V Lalwani,
Abel John,
Satish Shetty,
Miriam Giparakis,
Kanika Arora,
Avidesh Maharaj,
Gottfried Strasser,
Aaron Maxwell Andrews,
Helmut Koeck,
Alan Mantooth,
Gregory Salamo,
Debbie G Senesky
Abstract:
This follow-on work investigates the effect of temperature on the frequency limit of 2-dimensional electron gas (2DEG) Hall-effect sensors.
This follow-on work investigates the effect of temperature on the frequency limit of 2-dimensional electron gas (2DEG) Hall-effect sensors.
△ Less
Submitted 17 February, 2024;
originally announced February 2024.
-
SDSS-IV MaNGA: Calibration of astrophysical line-widths in the Hα region using HexPak observations
Authors:
Sabyasachi Chattopadhyay,
Matthew A. Bershady,
David R. Law,
Kyle Westfall,
Shravan Shetty,
Camilo Machuca,
Michele Cappellari,
Kate H. R. Rubin,
Kevin Bundy,
Samantha Penny
Abstract:
We have re-observed $\rm\sim$40 low-inclination, star-forming galaxies from the MaNGA survey ($\upsigma\sim65$~\kms) at $\sim$6.5 times higher spectral resolution ($\upsigma\sim10$~\kms) using the HexPak integral field unit on the WIYN 3.5m telescope. The aim of these observations is to calibrate MaNGA's instrumental resolution and to characterize turbulence in the warm interstellar medium and ion…
▽ More
We have re-observed $\rm\sim$40 low-inclination, star-forming galaxies from the MaNGA survey ($\upsigma\sim65$~\kms) at $\sim$6.5 times higher spectral resolution ($\upsigma\sim10$~\kms) using the HexPak integral field unit on the WIYN 3.5m telescope. The aim of these observations is to calibrate MaNGA's instrumental resolution and to characterize turbulence in the warm interstellar medium and ionized galactic outflows. Here we report the results for the H$\rm\upalpha$ region observations as they pertain to the calibration of MaNGA's spectral resolution. Remarkably, we find that the previously-reported MaNGA line-spread-function (LSF) Gaussian width is systematically underestimated by only 1\%. The LSF increase modestly reduces the characteristic dispersion of HII regions-dominated spectra sampled at 1-2 kpc spatial scales from 23 to 20 km s$^{-1}$ in our sample, or a 25\% decrease in the random-motion kinetic energy. This commensurately lowers the dispersion zeropoint in the relation between line-width and star-formation rate surface-density in galaxies sampled on the same spatial scale. This modest zero-point shift does not appear to alter the power-law slope in the relation between line-width and star-formation rate surface-density. We also show that adopting a scheme whereby corrected line-widths are computed as the square root of the median of the difference in the squared measured line width and the squared LSF Gaussian avoids biases and allows for lower SNR data to be used reliably.
△ Less
Submitted 2 February, 2024;
originally announced February 2024.
-
Design and development of opto-neural processors for simulation of neural networks trained in image detection for potential implementation in hybrid robotics
Authors:
Sanjana Shetty
Abstract:
Neural networks have been employed for a wide range of processing applications like image processing, motor control, object detection and many others. Living neural networks offer advantages of lower power consumption, faster processing, and biological realism. Optogenetics offers high spatial and temporal control over biological neurons and presents potential in training live neural networks. Thi…
▽ More
Neural networks have been employed for a wide range of processing applications like image processing, motor control, object detection and many others. Living neural networks offer advantages of lower power consumption, faster processing, and biological realism. Optogenetics offers high spatial and temporal control over biological neurons and presents potential in training live neural networks. This work proposes a simulated living neural network trained indirectly by backpropagating STDP based algorithms using precision activation by optogenetics achieving accuracy comparable to traditional neural network training algorithms.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Performance Analysis of Fixed Broadband Wireless Access in mmWave Band in 5G
Authors:
Soumya Banerjee,
Sarada Prasad Gochhayat,
Sachin Shetty
Abstract:
An end-to-end fiber-based network holds the potential to provide multi-gigabit fixed access to end-users. However, deploying fiber access, especially in areas where fiber is non-existent, can be time-consuming and costly, resulting in delayed returns for Operators. This work investigates transmission data from fixed broadband wireless access in the mmWave band in 5G. Given the growing interest in…
▽ More
An end-to-end fiber-based network holds the potential to provide multi-gigabit fixed access to end-users. However, deploying fiber access, especially in areas where fiber is non-existent, can be time-consuming and costly, resulting in delayed returns for Operators. This work investigates transmission data from fixed broadband wireless access in the mmWave band in 5G. Given the growing interest in this domain, understanding the transmission characteristics of the data becomes crucial. While existing datasets for the mmWave band are available, they are often generated from simulated environments. In this study, we introduce a dataset compiled from real-world transmission data collected from the Fixed Broadband Wireless Access in mmWave Band device (RWM6050). The aim is to facilitate self-configuration based on transmission characteristics. To achieve this, we propose an online machine learning-based approach for real-time training and classification of transmission characteristics. Additionally, we present two advanced temporal models for more accurate classifications. Our results demonstrate the ability to detect transmission angle and distance directly from the analysis of transmission data with very high accuracy, reaching up to 99% accuracy on the combined classification task. Finally, we outline promising future research directions based on the collected data.
△ Less
Submitted 28 November, 2023;
originally announced December 2023.
-
MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning
Authors:
Soumya Banerjee,
Sandip Roy,
Sayyed Farid Ahamed,
Devin Quinn,
Marc Vucovich,
Dhruv Nandakumar,
Kevin Choi,
Abdul Rahman,
Edward Bowen,
Sachin Shetty
Abstract:
The membership inference attack (MIA) is a popular paradigm for compromising the privacy of a machine learning (ML) model. MIA exploits the natural inclination of ML models to overfit upon the training data. MIAs are trained to distinguish between training and testing prediction confidence to infer membership information. Federated Learning (FL) is a privacy-preserving ML paradigm that enables mul…
▽ More
The membership inference attack (MIA) is a popular paradigm for compromising the privacy of a machine learning (ML) model. MIA exploits the natural inclination of ML models to overfit upon the training data. MIAs are trained to distinguish between training and testing prediction confidence to infer membership information. Federated Learning (FL) is a privacy-preserving ML paradigm that enables multiple clients to train a unified model without disclosing their private data. In this paper, we propose an enhanced Membership Inference Attack with the Batch-wise generated Attack Dataset (MIA-BAD), a modification to the MIA approach. We investigate that the MIA is more accurate when the attack dataset is generated batch-wise. This quantitatively decreases the attack dataset while qualitatively improving it. We show how training an ML model through FL, has some distinct advantages and investigate how the threat introduced with the proposed MIA-BAD approach can be mitigated with FL approaches. Finally, we demonstrate the qualitative effects of the proposed MIA-BAD methodology by conducting extensive experiments with various target datasets, variable numbers of federated clients, and training batch sizes.
△ Less
Submitted 28 November, 2023;
originally announced December 2023.
-
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
Authors:
Ryutaro Tanno,
David G. T. Barrett,
Andrew Sellergren,
Sumedh Ghaisas,
Sumanth Dathathri,
Abigail See,
Johannes Welbl,
Karan Singhal,
Shekoofeh Azizi,
Tao Tu,
Mike Schaekermann,
Rhys May,
Roy Lee,
SiWai Man,
Zahra Ahmed,
Sara Mahdavi,
Yossi Matias,
Joelle Barral,
Ali Eslami,
Danielle Belgrave,
Vivek Natarajan,
Shravya Shetty,
Pushmeet Kohli,
Po-Sen Huang,
Alan Karthikesalingam
, et al. (1 additional authors not shown)
Abstract:
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, restricts access to expert care and imposes heavy workloads, contributing to avoidable errors and delays in report delivery. While recent progress in automated report generation with vision-language models offer clear pote…
▽ More
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment. The worldwide shortage of radiologists, however, restricts access to expert care and imposes heavy workloads, contributing to avoidable errors and delays in report delivery. While recent progress in automated report generation with vision-language models offer clear potential in ameliorating the situation, the path to real-world adoption has been stymied by the challenge of evaluating the clinical quality of AI-generated reports. In this study, we build a state-of-the-art report generation system for chest radiographs, $\textit{Flamingo-CXR}$, by fine-tuning a well-known vision-language foundation model on radiology data. To evaluate the quality of the AI-generated reports, a group of 16 certified radiologists provide detailed evaluations of AI-generated and human written reports for chest X-rays from an intensive care setting in the United States and an inpatient setting in India. At least one radiologist (out of two per case) preferred the AI report to the ground truth report in over 60$\%$ of cases for both datasets. Amongst the subset of AI-generated reports that contain errors, the most frequently cited reasons were related to the location and finding, whereas for human written reports, most mistakes were related to severity and finding. This disparity suggested potential complementarity between our AI system and human experts, prompting us to develop an assistive scenario in which Flamingo-CXR generates a first-draft report, which is subsequently revised by a clinician. This is the first demonstration of clinician-AI collaboration for report writing, and the resultant reports are assessed to be equivalent or preferred by at least one radiologist to reports written by experts alone in 80$\%$ of in-patient cases and 60$\%$ of intensive care cases.
△ Less
Submitted 20 December, 2023; v1 submitted 30 November, 2023;
originally announced November 2023.
-
Anonymous Jamming Detection in 5G with Bayesian Network Model Based Inference Analysis
Authors:
Ying Wang,
Shashank Jere,
Soumya Banerjee,
Lingjia Liu,
Sachin Shetty,
Shehadi Dayekh
Abstract:
Jamming and intrusion detection are critical in 5G research, aiming to maintain reliability, prevent user experience degradation, and avoid infrastructure failure. This paper introduces an anonymous jamming detection model for 5G based on signal parameters from the protocol stacks. The system uses supervised and unsupervised learning for real-time, high-accuracy detection of jamming, including unk…
▽ More
Jamming and intrusion detection are critical in 5G research, aiming to maintain reliability, prevent user experience degradation, and avoid infrastructure failure. This paper introduces an anonymous jamming detection model for 5G based on signal parameters from the protocol stacks. The system uses supervised and unsupervised learning for real-time, high-accuracy detection of jamming, including unknown types. Supervised models reach an AUC of 0.964 to 1, compared to LSTM models with an AUC of 0.923 to 1. However, the need for data annotation limits the supervised approach. To address this, an unsupervised auto-encoder-based anomaly detection is presented with an AUC of 0.987. The approach is resistant to adversarial training samples. For transparency and domain knowledge injection, a Bayesian network-based causation analysis is introduced.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Detecting Natural Language Biases with Prompt-based Learning
Authors:
Md Abdul Aowal,
Maliha T Islam,
Priyanka Mary Mammen,
Sandesh Shetty
Abstract:
In this project, we want to explore the newly emerging field of prompt engineering and apply it to the downstream task of detecting LM biases. More concretely, we explore how to design prompts that can indicate 4 different types of biases: (1) gender, (2) race, (3) sexual orientation, and (4) religion-based. Within our project, we experiment with different manually crafted prompts that can draw ou…
▽ More
In this project, we want to explore the newly emerging field of prompt engineering and apply it to the downstream task of detecting LM biases. More concretely, we explore how to design prompts that can indicate 4 different types of biases: (1) gender, (2) race, (3) sexual orientation, and (4) religion-based. Within our project, we experiment with different manually crafted prompts that can draw out the subtle biases that may be present in the language model. We apply these prompts to multiple variations of popular and well-recognized models: BERT, RoBERTa, and T5 to evaluate their biases. We provide a comparative analysis of these models and assess them using a two-fold method: use human judgment to decide whether model predictions are biased and utilize model-level judgment (through further prompts) to understand if a model can self-diagnose the biases of its own prediction.
△ Less
Submitted 11 September, 2023;
originally announced September 2023.
-
ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders
Authors:
Shawn Xu,
Lin Yang,
Christopher Kelly,
Marcin Sieniek,
Timo Kohlberger,
Martin Ma,
Wei-Hung Weng,
Atilla Kiraly,
Sahar Kazemzadeh,
Zakkai Melamed,
Jungyeon Park,
Patricia Strachan,
Yun Liu,
Chuck Lau,
Preeti Singh,
Christina Chen,
Mozziyar Etemadi,
Sreenivasa Raju Kalidindi,
Yossi Matias,
Katherine Chou,
Greg S. Corrado,
Shravya Shetty,
Daniel Tse,
Shruthi Prabhakara,
Daniel Golden
, et al. (3 additional authors not shown)
Abstract:
In this work, we present an approach, which we call Embeddings for Language/Image-aligned X-Rays, or ELIXR, that leverages a language-aligned image encoder combined or grafted onto a fixed LLM, PaLM 2, to perform a broad range of chest X-ray tasks. We train this lightweight adapter architecture using images paired with corresponding free-text radiology reports from the MIMIC-CXR dataset. ELIXR ach…
▽ More
In this work, we present an approach, which we call Embeddings for Language/Image-aligned X-Rays, or ELIXR, that leverages a language-aligned image encoder combined or grafted onto a fixed LLM, PaLM 2, to perform a broad range of chest X-ray tasks. We train this lightweight adapter architecture using images paired with corresponding free-text radiology reports from the MIMIC-CXR dataset. ELIXR achieved state-of-the-art performance on zero-shot chest X-ray (CXR) classification (mean AUC of 0.850 across 13 findings), data-efficient CXR classification (mean AUCs of 0.893 and 0.898 across five findings (atelectasis, cardiomegaly, consolidation, pleural effusion, and pulmonary edema) for 1% (~2,200 images) and 10% (~22,000 images) training data), and semantic search (0.76 normalized discounted cumulative gain (NDCG) across nineteen queries, including perfect retrieval on twelve of them). Compared to existing data-efficient methods including supervised contrastive learning (SupCon), ELIXR required two orders of magnitude less data to reach similar performance. ELIXR also showed promise on CXR vision-language tasks, demonstrating overall accuracies of 58.7% and 62.5% on visual question answering and report quality assurance tasks, respectively. These results suggest that ELIXR is a robust and versatile approach to CXR AI.
△ Less
Submitted 7 September, 2023; v1 submitted 2 August, 2023;
originally announced August 2023.
-
Multimodal LLMs for health grounded in individual-specific data
Authors:
Anastasiya Belyaeva,
Justin Cosentino,
Farhad Hormozdiari,
Krish Eswaran,
Shravya Shetty,
Greg Corrado,
Andrew Carroll,
Cory Y. McLean,
Nicholas A. Furlotte
Abstract:
Foundation large language models (LLMs) have shown an impressive ability to solve tasks across a wide range of fields including health. To effectively solve personalized health tasks, LLMs need the ability to ingest a diversity of data modalities that are relevant to an individual's health status. In this paper, we take a step towards creating multimodal LLMs for health that are grounded in indivi…
▽ More
Foundation large language models (LLMs) have shown an impressive ability to solve tasks across a wide range of fields including health. To effectively solve personalized health tasks, LLMs need the ability to ingest a diversity of data modalities that are relevant to an individual's health status. In this paper, we take a step towards creating multimodal LLMs for health that are grounded in individual-specific data by developing a framework (HeLM: Health Large Language Model for Multimodal Understanding) that enables LLMs to use high-dimensional clinical modalities to estimate underlying disease risk. HeLM encodes complex data modalities by learning an encoder that maps them into the LLM's token embedding space and for simple modalities like tabular data by serializing the data into text. Using data from the UK Biobank, we show that HeLM can effectively use demographic and clinical features in addition to high-dimensional time-series data to estimate disease risk. For example, HeLM achieves an AUROC of 0.75 for asthma prediction when combining tabular and spirogram data modalities compared with 0.49 when only using tabular data. Overall, we find that HeLM outperforms or performs at parity with classical machine learning approaches across a selection of eight binary traits. Furthermore, we investigate the downstream uses of this model such as its generalizability to out-of-distribution traits and its ability to power conversations around individual health and wellness.
△ Less
Submitted 20 July, 2023; v1 submitted 18 July, 2023;
originally announced July 2023.
-
Functional light diffusers based on hybrid CsPbBr$_3$/SiO$_2$ aero-framework structures for laser light illumination and conversion
Authors:
Lena M. Saure,
Jonas Lumma,
Niklas Kohlmann,
Torge Hartig,
Ercules E. S. Teotonio,
Shwetha Shetty,
Narayanan Ravishankar,
Lorenz Kienle,
Franz Faupel,
Stefan Schröder,
Rainer Adelung,
Huayna Terraschke,
Fabian Schütt
Abstract:
The new generation of laser-based solid-state lighting (SSL) white light sources requires new material systems capable of withstanding, diffusing and converting high intensity laser light. State-of-the-art systems use a blue light emitting diode (LED) or laser diode (LD) in combination with color conversion materials, such as yellow emitting Ce-doped phosphors or red and green emitting quantum dot…
▽ More
The new generation of laser-based solid-state lighting (SSL) white light sources requires new material systems capable of withstanding, diffusing and converting high intensity laser light. State-of-the-art systems use a blue light emitting diode (LED) or laser diode (LD) in combination with color conversion materials, such as yellow emitting Ce-doped phosphors or red and green emitting quantum dots (QD), to produce white light. However, for laser-based high-brightness illumination in particular, thermal management is a major challenge, and in addition, a light diffuser is required to diffuse the highly focused laser beam. Here, we present a hybrid material system that simultaneously enables efficient, uniform light distribution and color conversion of a blue LD, while ensuring good thermal management even at high laser powers of up to 5W. A highly open porous (> 99%) framework structure of hollow SiO$_2$ microtubes is utilized as an efficient light diffuser that can drastically reduce speckle contrast. By further functionalizing the microtubes with halide perovskite QDs (SiO$_2$@CsPbBr$_3$ as model system) color conversion from UV to visible light is achieved. Under laser illumination, the open porous structure prevents heat accumulation and thermal quenching of the QDs. By depositing an ultrathin (~ 5.5 nm) film of poly(ethylene glycol dimethyl acrylate) (pEGDMA) via initiated chemical vapor deposition (iCVD), the luminescent stability of the QDs against moisture is enhanced. The demonstrated hybrid material system paves the way for the design of advanced and functional laser light diffusers and converters that can meet the challenges associated with laser-based SSL applications.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
Trustworthy Artificial Intelligence Framework for Proactive Detection and Risk Explanation of Cyber Attacks in Smart Grid
Authors:
Md. Shirajum Munir,
Sachin Shetty,
Danda B. Rawat
Abstract:
The rapid growth of distributed energy resources (DERs), such as renewable energy sources, generators, consumers, and prosumers in the smart grid infrastructure, poses significant cybersecurity and trust challenges to the grid controller. Consequently, it is crucial to identify adversarial tactics and measure the strength of the attacker's DER. To enable a trustworthy smart grid controller, this w…
▽ More
The rapid growth of distributed energy resources (DERs), such as renewable energy sources, generators, consumers, and prosumers in the smart grid infrastructure, poses significant cybersecurity and trust challenges to the grid controller. Consequently, it is crucial to identify adversarial tactics and measure the strength of the attacker's DER. To enable a trustworthy smart grid controller, this work investigates a trustworthy artificial intelligence (AI) mechanism for proactive identification and explanation of the cyber risk caused by the control/status message of DERs. Thus, proposing and developing a trustworthy AI framework to facilitate the deployment of any AI algorithms for detecting potential cyber threats and analyzing root causes based on Shapley value interpretation while dynamically quantifying the risk of an attack based on Ward's minimum variance formula. The experiment with a state-of-the-art dataset establishes the proposed framework as a trustworthy AI by fulfilling the capabilities of reliability, fairness, explainability, transparency, reproducibility, and accountability.
△ Less
Submitted 11 June, 2023;
originally announced June 2023.
-
Predicting Cardiovascular Disease Risk using Photoplethysmography and Deep Learning
Authors:
Wei-Hung Weng,
Sebastien Baur,
Mayank Daswani,
Christina Chen,
Lauren Harrell,
Sujay Kakarmath,
Mariam Jabara,
Babak Behsaz,
Cory Y. McLean,
Yossi Matias,
Greg S. Corrado,
Shravya Shetty,
Shruthi Prabhakara,
Yun Liu,
Goodarz Danaei,
Diego Ardila
Abstract:
Cardiovascular diseases (CVDs) are responsible for a large proportion of premature deaths in low- and middle-income countries. Early CVD detection and intervention is critical in these populations, yet many existing CVD risk scores require a physical examination or lab measurements, which can be challenging in such health systems due to limited accessibility. Here we investigated the potential to…
▽ More
Cardiovascular diseases (CVDs) are responsible for a large proportion of premature deaths in low- and middle-income countries. Early CVD detection and intervention is critical in these populations, yet many existing CVD risk scores require a physical examination or lab measurements, which can be challenging in such health systems due to limited accessibility. Here we investigated the potential to use photoplethysmography (PPG), a sensing technology available on most smartphones that can potentially enable large-scale screening at low cost, for CVD risk prediction. We developed a deep learning PPG-based CVD risk score (DLS) to predict the probability of having major adverse cardiovascular events (MACE: non-fatal myocardial infarction, stroke, and cardiovascular death) within ten years, given only age, sex, smoking status and PPG as predictors. We compared the DLS with the office-based refit-WHO score, which adopts the shared predictors from WHO and Globorisk scores (age, sex, smoking status, height, weight and systolic blood pressure) but refitted on the UK Biobank (UKB) cohort. In UKB cohort, DLS's C-statistic (71.1%, 95% CI 69.9-72.4) was non-inferior to office-based refit-WHO score (70.9%, 95% CI 69.7-72.2; non-inferiority margin of 2.5%, p<0.01). The calibration of the DLS was satisfactory, with a 1.8% mean absolute calibration error. Adding DLS features to the office-based score increased the C-statistic by 1.0% (95% CI 0.6-1.4). DLS predicts ten-year MACE risk comparable with the office-based refit-WHO score. It provides a proof-of-concept and suggests the potential of a PPG-based approach strategies for community-based primary prevention in resource-limited regions.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
CancerGPT: Few-shot Drug Pair Synergy Prediction using Large Pre-trained Language Models
Authors:
Tianhao Li,
Sandesh Shetty,
Advaith Kamath,
Ajay Jaiswal,
Xianqian Jiang,
Ying Ding,
Yejin Kim
Abstract:
Large pre-trained language models (LLMs) have been shown to have significant potential in few-shot learning across various fields, even with minimal training data. However, their ability to generalize to unseen tasks in more complex fields, such as biology, has yet to be fully evaluated. LLMs can offer a promising alternative approach for biological inference, particularly in cases where structure…
▽ More
Large pre-trained language models (LLMs) have been shown to have significant potential in few-shot learning across various fields, even with minimal training data. However, their ability to generalize to unseen tasks in more complex fields, such as biology, has yet to be fully evaluated. LLMs can offer a promising alternative approach for biological inference, particularly in cases where structured data and sample size are limited, by extracting prior knowledge from text corpora. Our proposed few-shot learning approach uses LLMs to predict the synergy of drug pairs in rare tissues that lack structured data and features. Our experiments, which involved seven rare tissues from different cancer types, demonstrated that the LLM-based prediction model achieved significant accuracy with very few or zero samples. Our proposed model, the CancerGPT (with $\sim$ 124M parameters), was even comparable to the larger fine-tuned GPT-3 model (with $\sim$ 175B parameters). Our research is the first to tackle drug pair synergy prediction in rare tissues with limited data. We are also the first to utilize an LLM-based prediction model for biological reaction prediction tasks.
△ Less
Submitted 17 April, 2023;
originally announced April 2023.
-
Hybrid aeromaterials for enhanced and rapid volumetric photothermal response
Authors:
Lena M. Saure,
Niklas Kohlmann,
Haoyi Qiu,
Shwetha Shetty,
Ali Shaygan Nia,
Narayanan Ravishankar,
Xinliang Feng,
Alexander Szameit,
Lorenz Kienle,
Rainer Adelung,
Fabian Schütt
Abstract:
Conversion of light into heat is essential for a broad range of technologies such as solar thermal heating, catalysis and desalination. Three-dimensional (3D) carbon nanomaterial-based aerogels have shown to hold great promise as photothermal transducer materials. However, till now, their light-to-heat conversion is limited by surface-near absorption, resulting in a strong heat localization only a…
▽ More
Conversion of light into heat is essential for a broad range of technologies such as solar thermal heating, catalysis and desalination. Three-dimensional (3D) carbon nanomaterial-based aerogels have shown to hold great promise as photothermal transducer materials. However, till now, their light-to-heat conversion is limited by surface-near absorption, resulting in a strong heat localization only at the illuminated surface region, while most of the aerogel volume remains unused. We present an innovative fabrication concept for highly porous (>99.9%) photothermal hybrid aeromaterials, that enable an ultra-rapid and volumetric photothermal response with an enhancement by a factor of around 2.5 compared to the pristine variant. The hybrid aeromaterial is based on strongly light-scattering framework structures composed of interconnected hollow silicon dioxide (SiO${_2}$) microtubes, which are functionalized with extremely low amounts (in order of a few $μ$g cm${^-}$${^3}$) of reduced graphene oxide (rGO) nanosheets, acting as photothermal agents. Tailoring the density of rGO within the framework structure enables us to control both, light scattering and light absorption, and thus the volumetric photothermal response. We further show that by rapid and repeatable gas activation these transducer materials expand the field of photothermal applications, like untethered light-powered and -controlled microfluidic pumps and soft pneumatic actuators.
△ Less
Submitted 12 June, 2023; v1 submitted 24 March, 2023;
originally announced March 2023.
-
Understanding metric-related pitfalls in image analysis validation
Authors:
Annika Reinke,
Minu D. Tizabi,
Michael Baumgartner,
Matthias Eisenmann,
Doreen Heckmann-Nötzel,
A. Emre Kavur,
Tim Rädsch,
Carole H. Sudre,
Laura Acion,
Michela Antonelli,
Tal Arbel,
Spyridon Bakas,
Arriel Benis,
Matthew Blaschko,
Florian Buettner,
M. Jorge Cardoso,
Veronika Cheplygina,
Jianxu Chen,
Evangelia Christodoulou,
Beth A. Cimini,
Gary S. Collins,
Keyvan Farahani,
Luciana Ferrer,
Adrian Galdran,
Bram van Ginneken
, et al. (53 additional authors not shown)
Abstract:
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibilit…
▽ More
Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.
△ Less
Submitted 23 February, 2024; v1 submitted 3 February, 2023;
originally announced February 2023.
-
ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds
Authors:
Daniel Seita,
Yufei Wang,
Sarthak J. Shetty,
Edward Yao Li,
Zackory Erickson,
David Held
Abstract:
Point clouds are a widely available and canonical data modality which convey the 3D geometry of a scene. Despite significant progress in classification and segmentation from point clouds, policy learning from such a modality remains challenging, and most prior works in imitation learning focus on learning policies from images or state information. In this paper, we propose a novel framework for le…
▽ More
Point clouds are a widely available and canonical data modality which convey the 3D geometry of a scene. Despite significant progress in classification and segmentation from point clouds, policy learning from such a modality remains challenging, and most prior works in imitation learning focus on learning policies from images or state information. In this paper, we propose a novel framework for learning policies from point clouds for robotic manipulation with tools. We use a novel neural network, ToolFlowNet, which predicts dense per-point flow on the tool that the robot controls, and then uses the flow to derive the transformation that the robot should execute. We apply this framework to imitation learning of challenging deformable object manipulation tasks with continuous movement of tools, including scooping and pouring, and demonstrate significantly improved performance over baselines which do not use flow. We perform 50 physical scooping experiments with ToolFlowNet and attain 82% scooping success. See https://tinyurl.com/toolflownet for supplementary material.
△ Less
Submitted 16 November, 2022;
originally announced November 2022.
-
Student-centric Model of Learning Management System Activity and Academic Performance: from Correlation to Causation
Authors:
Varun Mandalapu,
Lujie Karen Chen,
Sushruta Shetty,
Zhiyuan Chen,
Jiaqi Gong
Abstract:
In recent years, there is a lot of interest in modeling students' digital traces in Learning Management System (LMS) to understand students' learning behavior patterns including aspects of meta-cognition and self-regulation, with the ultimate goal to turn those insights into actionable information to support students to improve their learning outcomes. In achieving this goal, however, there are tw…
▽ More
In recent years, there is a lot of interest in modeling students' digital traces in Learning Management System (LMS) to understand students' learning behavior patterns including aspects of meta-cognition and self-regulation, with the ultimate goal to turn those insights into actionable information to support students to improve their learning outcomes. In achieving this goal, however, there are two main issues that need to be addressed given the existing literature. Firstly, most of the current work is course-centered (i.e. models are built from data for a specific course) rather than student-centered; secondly, a vast majority of the models are correlational rather than causal. Those issues make it challenging to identify the most promising actionable factors for intervention at the student level where most of the campus-wide academic support is designed for. In this paper, we explored a student-centric analytical framework for LMS activity data that can provide not only correlational but causal insights mined from observational data. We demonstrated this approach using a dataset of 1651 computing major students at a public university in the US during one semester in the Fall of 2019. This dataset includes students' fine-grained LMS interaction logs and administrative data, e.g. demographics and academic performance. In addition, we expand the repository of LMS behavior indicators to include those that can characterize the time-of-the-day of login (e.g. chronotype). Our analysis showed that student login volume, compared with other login behavior indicators, is both strongly correlated and causally linked to student academic performance, especially among students with low academic performance. We envision that those insights will provide convincing evidence for college student support groups to launch student-centered and targeted interventions that are effective and scalable.
△ Less
Submitted 29 March, 2023; v1 submitted 27 October, 2022;
originally announced October 2022.
-
Neuro-symbolic Explainable Artificial Intelligence Twin for Zero-touch IoE in Wireless Network
Authors:
Md. Shirajum Munir,
Ki Tae Kim,
Apurba Adhikary,
Walid Saad,
Sachin Shetty,
Seong-Bae Park,
Choong Seon Hong
Abstract:
Explainable artificial intelligence (XAI) twin systems will be a fundamental enabler of zero-touch network and service management (ZSM) for sixth-generation (6G) wireless networks. A reliable XAI twin system for ZSM requires two composites: an extreme analytical ability for discretizing the physical behavior of the Internet of Everything (IoE) and rigorous methods for characterizing the reasoning…
▽ More
Explainable artificial intelligence (XAI) twin systems will be a fundamental enabler of zero-touch network and service management (ZSM) for sixth-generation (6G) wireless networks. A reliable XAI twin system for ZSM requires two composites: an extreme analytical ability for discretizing the physical behavior of the Internet of Everything (IoE) and rigorous methods for characterizing the reasoning of such behavior. In this paper, a novel neuro-symbolic explainable artificial intelligence twin framework is proposed to enable trustworthy ZSM for a wireless IoE. The physical space of the XAI twin executes a neural-network-driven multivariate regression to capture the time-dependent wireless IoE environment while determining unconscious decisions of IoE service aggregation. Subsequently, the virtual space of the XAI twin constructs a directed acyclic graph (DAG)-based Bayesian network that can infer a symbolic reasoning score over unconscious decisions through a first-order probabilistic language model. Furthermore, a Bayesian multi-arm bandits-based learning problem is proposed for reducing the gap between the expected explained score and the current obtained score of the proposed neuro-symbolic XAI twin. To address the challenges of extensible, modular, and stateless management functions in ZSM, the proposed neuro-symbolic XAI twin framework consists of two learning systems: 1) an implicit learner that acts as an unconscious learner in physical space, and 2) an explicit leaner that can exploit symbolic reasoning based on implicit learner decisions and prior evidence. Experimental results show that the proposed neuro-symbolic XAI twin can achieve around 96.26% accuracy while guaranteeing from 18% to 44% more trust score in terms of reasoning and closed-loop automation.
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
Tensor Train for Global Optimization Problems in Robotics
Authors:
Suhan Shetty,
Teguh Lembono,
Tobias Loew,
Sylvain Calinon
Abstract:
The convergence of many numerical optimization techniques is highly dependent on the initial guess given to the solver. To address this issue, we propose a novel approach that utilizes tensor methods to initialize existing optimization solvers near global optima. Our method does not require access to a database of good solutions. We first transform the cost function, which depends on both task par…
▽ More
The convergence of many numerical optimization techniques is highly dependent on the initial guess given to the solver. To address this issue, we propose a novel approach that utilizes tensor methods to initialize existing optimization solvers near global optima. Our method does not require access to a database of good solutions. We first transform the cost function, which depends on both task parameters and optimization variables, into a probability density function. Unlike existing approaches, the joint probability distribution of the task parameters and optimization variables is approximated using the Tensor Train model, which enables efficient conditioning and sampling. We treat the task parameters as random variables, and for a given task, we generate samples for decision variables from the conditional distribution to initialize the optimization solver. Our method can produce multiple solutions (when they exist) faster than existing methods. We first evaluate the approach on benchmark functions for numerical optimization that are hard to solve using gradient-based optimization solvers with a naive initialization. The results show that the proposed method can generate samples close to global optima and from multiple modes. We then demonstrate the generality and relevance of our framework to robotics by applying it to inverse kinematics with obstacles and motion planning problems with a 7-DoF manipulator.
△ Less
Submitted 22 November, 2023; v1 submitted 10 June, 2022;
originally announced June 2022.
-
Metrics reloaded: Recommendations for image analysis validation
Authors:
Lena Maier-Hein,
Annika Reinke,
Patrick Godau,
Minu D. Tizabi,
Florian Buettner,
Evangelia Christodoulou,
Ben Glocker,
Fabian Isensee,
Jens Kleesiek,
Michal Kozubek,
Mauricio Reyes,
Michael A. Riegler,
Manuel Wiesenfarth,
A. Emre Kavur,
Carole H. Sudre,
Michael Baumgartner,
Matthias Eisenmann,
Doreen Heckmann-Nötzel,
Tim Rädsch,
Laura Acion,
Michela Antonelli,
Tal Arbel,
Spyridon Bakas,
Arriel Benis,
Matthew Blaschko
, et al. (49 additional authors not shown)
Abstract:
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflect the domain interest, thus failing to adequately measure scientific progress and hindering translation of ML techniques into practice. To overcome this, our large international ex…
▽ More
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. Particularly in automatic biomedical image analysis, chosen performance metrics often do not reflect the domain interest, thus failing to adequately measure scientific progress and hindering translation of ML techniques into practice. To overcome this, our large international expert consortium created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. The framework was developed in a multi-stage Delphi process and is based on the novel concept of a problem fingerprint - a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), data set and algorithm output. Based on the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as a classification task at image, object or pixel level, namely image-level classification, object detection, semantic segmentation, and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool, which also provides a point of access to explore weaknesses, strengths and specific recommendations for the most common validation metrics. The broad applicability of our framework across domains is demonstrated by an instantiation for various biological and medical image analysis use cases.
△ Less
Submitted 23 February, 2024; v1 submitted 3 June, 2022;
originally announced June 2022.
-
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions
Authors:
Difan Liu,
Sandesh Shetty,
Tobias Hinz,
Matthew Fisher,
Richard Zhang,
Taesung Park,
Evangelos Kalogerakis
Abstract:
We present ASSET, a neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map. Our architecture is based on a transformer with a novel attention mechanism. Our key idea is to sparsify the transformer's attention matrix at high resolutions, guided by dense attention extracted at lower image resolutions. While previous…
▽ More
We present ASSET, a neural architecture for automatically modifying an input high-resolution image according to a user's edits on its semantic segmentation map. Our architecture is based on a transformer with a novel attention mechanism. Our key idea is to sparsify the transformer's attention matrix at high resolutions, guided by dense attention extracted at lower image resolutions. While previous attention mechanisms are computationally too expensive for handling high-resolution images or are overly constrained within specific image regions hampering long-range interactions, our novel attention mechanism is both computationally efficient and effective. Our sparsified attention mechanism is able to capture long-range interactions and context, leading to synthesizing interesting phenomena in scenes, such as reflections of landscapes onto water or flora consistent with the rest of the landscape, that were not possible to generate reliably with previous convnets and transformer approaches. We present qualitative and quantitative results, along with user studies, demonstrating the effectiveness of our method.
△ Less
Submitted 24 May, 2022;
originally announced May 2022.
-
Enabling faster and more reliable sonographic assessment of gestational age through machine learning
Authors:
Chace Lee,
Angelica Willis,
Christina Chen,
Marcin Sieniek,
Akib Uddin,
Jonny Wong,
Rory Pilgrim,
Katherine Chou,
Daniel Tse,
Shravya Shetty,
Ryan G. Gomes
Abstract:
Fetal ultrasounds are an essential part of prenatal care and can be used to estimate gestational age (GA). Accurate GA assessment is important for providing appropriate prenatal care throughout pregnancy and identifying complications such as fetal growth disorders. Since derivation of GA from manual fetal biometry measurements (head, abdomen, femur) are operator-dependent and time-consuming, there…
▽ More
Fetal ultrasounds are an essential part of prenatal care and can be used to estimate gestational age (GA). Accurate GA assessment is important for providing appropriate prenatal care throughout pregnancy and identifying complications such as fetal growth disorders. Since derivation of GA from manual fetal biometry measurements (head, abdomen, femur) are operator-dependent and time-consuming, there have been a number of research efforts focused on using artificial intelligence (AI) models to estimate GA using standard biometry images, but there is still room to improve the accuracy and reliability of these AI systems for widescale adoption. To improve GA estimates, without significant change to provider workflows, we leverage AI to interpret standard plane ultrasound images as well as 'fly-to' ultrasound videos, which are 5-10s videos automatically recorded as part of the standard of care before the still image is captured. We developed and validated three AI models: an image model using standard plane images, a video model using fly-to videos, and an ensemble model (combining both image and video). All three were statistically superior to standard fetal biometry-based GA estimates derived by expert sonographers, the ensemble model has the lowest mean absolute error (MAE) compared to the clinical standard fetal biometry (mean difference: -1.51 $\pm$ 3.96 days, 95% CI [-1.9, -1.1]) on a test set that consisted of 404 participants. We showed that our models outperform standard biometry by a more substantial margin on fetuses that were small for GA. Our AI models have the potential to empower trained operators to estimate GA with higher accuracy while reducing the amount of time required and user variability in measurement acquisition.
△ Less
Submitted 22 March, 2022;
originally announced March 2022.
-
AI system for fetal ultrasound in low-resource settings
Authors:
Ryan G. Gomes,
Bellington Vwalika,
Chace Lee,
Angelica Willis,
Marcin Sieniek,
Joan T. Price,
Christina Chen,
Margaret P. Kasaro,
James A. Taylor,
Elizabeth M. Stringer,
Scott Mayer McKinney,
Ntazana Sindano,
George E. Dahl,
William Goodnight III,
Justin Gilmer,
Benjamin H. Chi,
Charles Lau,
Terry Spitz,
T Saensuksopa,
Kris Liu,
Jonny Wong,
Rory Pilgrim,
Akib Uddin,
Greg Corrado,
Lily Peng
, et al. (4 additional authors not shown)
Abstract:
Despite considerable progress in maternal healthcare, maternal and perinatal deaths remain high in low-to-middle income countries. Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption. We developed and validated an artificial intelligence (AI) system that uses novice-acquired "blind sweep" ultrasound videos to…
▽ More
Despite considerable progress in maternal healthcare, maternal and perinatal deaths remain high in low-to-middle income countries. Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption. We developed and validated an artificial intelligence (AI) system that uses novice-acquired "blind sweep" ultrasound videos to estimate gestational age (GA) and fetal malpresentation. We further addressed obstacles that may be encountered in low-resourced settings. Using a simplified sweep protocol with real-time AI feedback on sweep quality, we have demonstrated the generalization of model performance to minimally trained novice ultrasound operators using low cost ultrasound devices with on-device AI integration. The GA model was non-inferior to standard fetal biometry estimates with as few as two sweeps, and the fetal malpresentation model had high AUC-ROCs across operators and devices. Our AI models have the potential to assist in upleveling the capabilities of lightly trained ultrasound operators in low resource settings.
△ Less
Submitted 18 March, 2022;
originally announced March 2022.
-
FedSyn: Synthetic Data Generation using Federated Learning
Authors:
Monik Raj Behera,
Sudhir Upadhyay,
Suresh Shetty,
Sudha Priyadarshini,
Palka Patel,
Ker Farn Lee
Abstract:
As Deep Learning algorithms continue to evolve and become more sophisticated, they require massive datasets for model training and efficacy of models. Some of those data requirements can be met with the help of existing datasets within the organizations. Current Machine Learning practices can be leveraged to generate synthetic data from an existing dataset. Further, it is well established that div…
▽ More
As Deep Learning algorithms continue to evolve and become more sophisticated, they require massive datasets for model training and efficacy of models. Some of those data requirements can be met with the help of existing datasets within the organizations. Current Machine Learning practices can be leveraged to generate synthetic data from an existing dataset. Further, it is well established that diversity in generated synthetic data relies on (and is perhaps limited by) statistical properties of available dataset within a single organization or entity. The more diverse an existing dataset is, the more expressive and generic synthetic data can be. However, given the scarcity of underlying data, it is challenging to collate big data in one organization. The diverse, non-overlapping dataset across distinct organizations provides an opportunity for them to contribute their limited distinct data to a larger pool that can be leveraged to further synthesize. Unfortunately, this raises data privacy concerns that some institutions may not be comfortable with.
This paper proposes a novel approach to generate synthetic data - FedSyn. FedSyn is a collaborative, privacy preserving approach to generate synthetic data among multiple participants in a federated and collaborative network. FedSyn creates a synthetic data generation model, which can generate synthetic data consisting of statistical distribution of almost all the participants in the network. FedSyn does not require access to the data of an individual participant, hence protecting the privacy of participant's data. The proposed technique in this paper leverages federated machine learning and generative adversarial network (GAN) as neural network architecture for synthetic data generation. The proposed method can be extended to many machine learning problem classes in finance, health, governance, technology and many more.
△ Less
Submitted 5 April, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Paving the Way towards 800 Gbps Quantum-Secured Optical Channel Deployment in Mission-Critical Environments
Authors:
Marco Pistoia,
Omar Amer,
Monik R. Behera,
Joseph A. Dolphin,
James F. Dynes,
Benny John,
Paul A. Haigh,
Yasushi Kawakura,
David H. Kramer,
Jeffrey Lyon,
Navid Moazzami,
Tulasi D. Movva,
Antigoni Polychroniadou,
Suresh Shetty,
Greg Sysak,
Farzam Toudeh-Fallah,
Sudhir Upadhyay,
Robert I. Woodward,
Andrew J. Shields
Abstract:
This article describes experimental research studies conducted towards understanding the implementation aspects of high-capacity quantum-secured optical channels in mission-critical metro-scale operational environments using Quantum Key Distribution (QKD) technology. To the best of our knowledge, this is the first time that an 800 Gbps quantum-secured optical channel -- along with several other De…
▽ More
This article describes experimental research studies conducted towards understanding the implementation aspects of high-capacity quantum-secured optical channels in mission-critical metro-scale operational environments using Quantum Key Distribution (QKD) technology. To the best of our knowledge, this is the first time that an 800 Gbps quantum-secured optical channel -- along with several other Dense Wavelength Division Multiplexed (DWDM) channels on the C-band and multiplexed with the QKD channel on the O-band -- was established at distances up to 100 km, with secret key-rates relevant for practical industry use cases. In addition, during the course of these trials, transporting a blockchain application over this established channel was utilized as a demonstration of securing a financial transaction in transit over a quantum-secured optical channel. The findings of this research pave the way towards the deployment of QKD-secured optical channels in high-capacity, metro-scale, mission-critical operational environments, such as Inter-Data Center Interconnects.
△ Less
Submitted 2 March, 2023; v1 submitted 15 February, 2022;
originally announced February 2022.
-
Succinct Differentiation of Disparate Boosting Ensemble Learning Methods for Prognostication of Polycystic Ovary Syndrome Diagnosis
Authors:
Abhishek Gupta,
Sannidhi Shetty,
Raunak Joshi,
Ronald Melwin Laban
Abstract:
Prognostication of medical problems using the clinical data by leveraging the Machine Learning techniques with stellar precision is one of the most important real world challenges at the present time. Considering the medical problem of Polycystic Ovary Syndrome also known as PCOS is an emerging problem in women aged from 15 to 49. Diagnosing this disorder by using various Boosting Ensemble Methods…
▽ More
Prognostication of medical problems using the clinical data by leveraging the Machine Learning techniques with stellar precision is one of the most important real world challenges at the present time. Considering the medical problem of Polycystic Ovary Syndrome also known as PCOS is an emerging problem in women aged from 15 to 49. Diagnosing this disorder by using various Boosting Ensemble Methods is something we have presented in this paper. A detailed and compendious differentiation between Adaptive Boost, Gradient Boosting Machine, XGBoost and CatBoost with their respective performance metrics highlighting the hidden anomalies in the data and its effects on the result is something we have presented in this paper. Metrics like Confusion Matrix, Precision, Recall, F1 Score, FPR, RoC Curve and AUC have been used in this paper.
△ Less
Submitted 13 August, 2022; v1 submitted 2 January, 2022;
originally announced January 2022.
-
Effectiveness of Multi-Physics Numerical Model in Simulating Accelerated Corrosion with Spatial and Temporal Non Uniformity
Authors:
Shanmukha Shetty,
Sauvik Banerjee,
Siddharth Tallur,
Yogesh M. Desai
Abstract:
This study is motivated by the need to develop an efficient numerical model to simulate non-uniform interfacial degradation of reinforcing steel in concrete in an accelerated corrosion setup. In this study, a multi physics finite element (FE) model is presented that takes into consideration the spatial and temporal non uniformity of corrosion induced degradation in rebar, and eliminates the assump…
▽ More
This study is motivated by the need to develop an efficient numerical model to simulate non-uniform interfacial degradation of reinforcing steel in concrete in an accelerated corrosion setup. In this study, a multi physics finite element (FE) model is presented that takes into consideration the spatial and temporal non uniformity of corrosion induced degradation in rebar, and eliminates the assumption of uniform mass loss and its linear variation with time as per available literature that uses classical approach of Faraday's law. The model is validated experimentally with accelerated corrosion setup designed to induce partial corrosion. Further, the possibility of extending this model to monitor natural corrosion is discussed with required modifications. Unlike previous studies, pore saturation (PS) is continuously monitored and its existing experimental correlations with electrolyte conductivity and oxygen diffusivity in the vicinity of partial corrosion are adopted so that the model can be extended to simulate natural corrosion. These evaluations can be made completely nondestructive and in real time to capture the influence of local environment. The proposed methodology also captures the effect of differential aeration pertaining to local exposure. Therefore, the challenges in incorporating influence of local environment by the use of alternative parameters such as relative humidity from real climate change predictions are eliminated. It is shown that the multi-physics model is effective and convenient to simulate the non-uniform time dependent mass loss with acceptable accuracy and that its capability can be extended to monitor natural non uniform corrosion on a space time frame.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
Avoid Estimating the Unknown Function in a Semiparametric Nonignorable Propensity Model
Authors:
Samidha Shetty,
Yanyuan Ma,
Jiwei Zhao
Abstract:
We study the problem of estimating a functional or a parameter in the context where outcome is subject to nonignorable missingness. We completely avoid modeling the regression relation, while allowing the propensity to be modeled by a semiparametric logistic relation where the dependence on covariates is unspecified. We discover a surprising phenomenon in that the estimation of the parameter in th…
▽ More
We study the problem of estimating a functional or a parameter in the context where outcome is subject to nonignorable missingness. We completely avoid modeling the regression relation, while allowing the propensity to be modeled by a semiparametric logistic relation where the dependence on covariates is unspecified. We discover a surprising phenomenon in that the estimation of the parameter in the propensity model as well as the functional estimation can be carried out without assessing the missingness dependence on covariates. This allows us to propose a general class of estimators for both model parameter estimation and functional estimation, including estimating the outcome mean. The robustness of the estimators are nonstandard and are established rigorously through theoretical derivations, and are supported by simulations and a data application.
△ Less
Submitted 10 August, 2021;
originally announced August 2021.
-
Federated Learning using Smart Contracts on Blockchains, based on Reward Driven Approach
Authors:
Monik Raj Behera,
Sudhir Upadhyay,
Suresh Shetty
Abstract:
Over the recent years, Federated machine learning continues to gain interest and momentum where there is a need to draw insights from data while preserving the data provider's privacy. However, one among other existing challenges in the adoption of federated learning has been the lack of fair, transparent and universally agreed incentivization schemes for rewarding the federated learning contribut…
▽ More
Over the recent years, Federated machine learning continues to gain interest and momentum where there is a need to draw insights from data while preserving the data provider's privacy. However, one among other existing challenges in the adoption of federated learning has been the lack of fair, transparent and universally agreed incentivization schemes for rewarding the federated learning contributors. Smart contracts on a blockchain network provide transparent, immutable and independently verifiable proofs by all participants of the network. We leverage this open and transparent nature of smart contracts on a blockchain to define incentivization rules for the contributors, which is based on a novel scalar quantity - federated contribution. Such a smart contract based reward-driven model has the potential to revolutionize the federated learning adoption in enterprises. Our contribution is two-fold: first is to show how smart contract based blockchain can be a very natural communication channel for federated learning. Second, leveraging this infrastructure, we can show how an intuitive measure of each agents' contribution can be built and integrated with the life cycle of the training and reward process.
△ Less
Submitted 25 March, 2022; v1 submitted 19 July, 2021;
originally announced July 2021.
-
Deep learning for detecting pulmonary tuberculosis via chest radiography: an international study across 10 countries
Authors:
Sahar Kazemzadeh,
Jin Yu,
Shahar Jamshy,
Rory Pilgrim,
Zaid Nabulsi,
Christina Chen,
Neeral Beladia,
Charles Lau,
Scott Mayer McKinney,
Thad Hughes,
Atilla Kiraly,
Sreenivasa Raju Kalidindi,
Monde Muyoyeta,
Jameson Malemela,
Ting Shih,
Greg S. Corrado,
Lily Peng,
Katherine Chou,
Po-Hsuan Cameron Chen,
Yun Liu,
Krish Eswaran,
Daniel Tse,
Shravya Shetty,
Shruthi Prabhakara
Abstract:
Tuberculosis (TB) is a top-10 cause of death worldwide. Though the WHO recommends chest radiographs (CXRs) for TB screening, the limited availability of CXR interpretation is a barrier. We trained a deep learning system (DLS) to detect active pulmonary TB using CXRs from 9 countries across Africa, Asia, and Europe, and utilized large-scale CXR pretraining, attention pooling, and noisy student semi…
▽ More
Tuberculosis (TB) is a top-10 cause of death worldwide. Though the WHO recommends chest radiographs (CXRs) for TB screening, the limited availability of CXR interpretation is a barrier. We trained a deep learning system (DLS) to detect active pulmonary TB using CXRs from 9 countries across Africa, Asia, and Europe, and utilized large-scale CXR pretraining, attention pooling, and noisy student semi-supervised learning. Evaluation was on (1) a combined test set spanning China, India, US, and Zambia, and (2) an independent mining population in South Africa. Given WHO targets of 90% sensitivity and 70% specificity, the DLS's operating point was prespecified to favor sensitivity over specificity. On the combined test set, the DLS's ROC curve was above all 9 India-based radiologists, with an AUC of 0.90 (95%CI 0.87-0.92). The DLS's sensitivity (88%) was higher than the India-based radiologists (75% mean sensitivity), p<0.001 for superiority; and its specificity (79%) was non-inferior to the radiologists (84% mean specificity), p=0.004. Similar trends were observed within HIV positive and sputum smear positive sub-groups, and in the South Africa test set. We found that 5 US-based radiologists (where TB isn't endemic) were more sensitive and less specific than the India-based radiologists (where TB is endemic). The DLS also remained non-inferior to the US-based radiologists. In simulations, using the DLS as a prioritization tool for confirmatory testing reduced the cost per positive case detected by 40-80% compared to using confirmatory testing alone. To conclude, our DLS generalized to 5 countries, and merits prospective evaluation to assist cost-effective screening efforts in radiologist-limited settings. Operating point flexibility may permit customization of the DLS to account for site-specific factors such as TB prevalence, demographics, clinical resources, and customary practice patterns.
△ Less
Submitted 29 October, 2021; v1 submitted 16 May, 2021;
originally announced May 2021.
-
Common Limitations of Image Processing Metrics: A Picture Story
Authors:
Annika Reinke,
Minu D. Tizabi,
Carole H. Sudre,
Matthias Eisenmann,
Tim Rädsch,
Michael Baumgartner,
Laura Acion,
Michela Antonelli,
Tal Arbel,
Spyridon Bakas,
Peter Bankhead,
Arriel Benis,
Matthew Blaschko,
Florian Buettner,
M. Jorge Cardoso,
Jianxu Chen,
Veronika Cheplygina,
Evangelia Christodoulou,
Beth Cimini,
Gary S. Collins,
Sandy Engelhardt,
Keyvan Farahani,
Luciana Ferrer,
Adrian Galdran,
Bram van Ginneken
, et al. (68 additional authors not shown)
Abstract:
While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using spe…
▽ More
While the importance of automatic image analysis is continuously increasing, recent meta-research revealed major flaws with respect to algorithm validation. Performance metrics are particularly key for meaningful, objective, and transparent performance assessment and validation of the used automatic algorithms, but relatively little attention has been given to the practical pitfalls when using specific metrics for a given image analysis task. These are typically related to (1) the disregard of inherent metric properties, such as the behaviour in the presence of class imbalance or small target structures, (2) the disregard of inherent data set properties, such as the non-independence of the test cases, and (3) the disregard of the actual biomedical domain interest that the metrics should reflect. This living dynamically document has the purpose to illustrate important limitations of performance metrics commonly applied in the field of image analysis. In this context, it focuses on biomedical image analysis problems that can be phrased as image-level classification, semantic segmentation, instance segmentation, or object detection task. The current version is based on a Delphi process on metrics conducted by an international consortium of image analysis experts from more than 60 institutions worldwide.
△ Less
Submitted 6 December, 2023; v1 submitted 12 April, 2021;
originally announced April 2021.
-
Reproducibility, Replicability and Beyond: Assessing Production Readiness of Aspect Based Sentiment Analysis in the Wild
Authors:
Rajdeep Mukherjee,
Shreyas Shetty,
Subrata Chattopadhyay,
Subhadeep Maji,
Samik Datta,
Pawan Goyal
Abstract:
With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor…
▽ More
With the exponential growth of online marketplaces and user-generated content therein, aspect-based sentiment analysis has become more important than ever. In this work, we critically review a representative sample of the models published during the past six years through the lens of a practitioner, with an eye towards deployment in production. First, our rigorous empirical evaluation reveals poor reproducibility: an average 4-5% drop in test accuracy across the sample. Second, to further bolster our confidence in empirical evaluation, we report experiments on two challenging data slices, and observe a consistent 12-55% drop in accuracy. Third, we study the possibility of transfer across domains and observe that as little as 10-25% of the domain-specific training dataset, when used in conjunction with datasets from other domains within the same locale, largely closes the gap between complete cross-domain and complete in-domain predictive performance. Lastly, we open-source two large-scale annotated review corpora from a large e-commerce portal in India in order to aid the study of replicability and transfer, with the hope that it will fuel further growth of the field.
△ Less
Submitted 23 January, 2021;
originally announced January 2021.
-
Ergodic Exploration using Tensor Train: Applications in Insertion Tasks
Authors:
Suhan Shetty,
João Silvério,
Sylvain Calinon
Abstract:
In robotics, ergodic control extends the tracking principle by specifying a probability distribution over an area to cover instead of a trajectory to track. The original problem is formulated as a spectral multiscale coverage problem, typically requiring the spatial distribution to be decomposed as Fourier series. This approach does not scale well to control problems requiring exploration in searc…
▽ More
In robotics, ergodic control extends the tracking principle by specifying a probability distribution over an area to cover instead of a trajectory to track. The original problem is formulated as a spectral multiscale coverage problem, typically requiring the spatial distribution to be decomposed as Fourier series. This approach does not scale well to control problems requiring exploration in search space of more than 2 dimensions. To address this issue, we propose the use of tensor trains, a recent low-rank tensor decomposition technique from the field of multilinear algebra. The proposed solution is efficient, both computationally and storage-wise, hence making it suitable for its online implementation in robotic systems. The approach is applied to a peg-in-hole insertion task requiring full 6D end-effector poses, implemented with a 7-axis Franka Emika Panda robot. In this experiment, ergodic exploration allows the task to be achieved without requiring the use of force/torque sensors.
△ Less
Submitted 17 May, 2021; v1 submitted 12 January, 2021;
originally announced January 2021.
-
Weight-Based Exploration for Unmanned Aerial Teams Searching for Multiple Survivors
Authors:
Sarthak J. Shetty,
Debasish Ghose
Abstract:
During floods, reaching survivors in the shortest possible time is a priority for rescue teams. Given their ability to explore difficult terrain in short spans of time, Unmanned Aerial Vehicles (UAVs) have become an increasingly valuable aid to search and rescue operations. Traditionally, UAVs utilize exhaustive lawnmower exploration patterns to locate stranded survivors, without any information r…
▽ More
During floods, reaching survivors in the shortest possible time is a priority for rescue teams. Given their ability to explore difficult terrain in short spans of time, Unmanned Aerial Vehicles (UAVs) have become an increasingly valuable aid to search and rescue operations. Traditionally, UAVs utilize exhaustive lawnmower exploration patterns to locate stranded survivors, without any information regarding the survivor's whereabouts. In real life disaster scenarios however, on-ground observers provide valuable information to the rescue effort, such as the survivor's last known location and heading. In earlier work, a Weight Based Exploration (WBE) model, which utilizes this information to generate a prioritized list of waypoints to aid the UAV in its search mission, was proposed. This approach was shown to be effective for a single UAV locating a single survivor. In this paper, we extend the WBE model to a team of UAVs locating multiple survivors. The model initially partitions the search environment amongst the UAVs using Voronoi cells. The UAVs then utilize the WBE model to locate survivors in their partitions. We test this model with varying survivor locations and headings. We demonstrate the scalability of the model developed by testing the model with aerial teams comprising several UAVs.
△ Less
Submitted 21 December, 2020;
originally announced December 2020.
-
SDSS-IV MaNGA: Modeling the Spectral Line Spread Function to Sub-Percent Accuracy
Authors:
David R. Law,
Kyle B. Westfall,
Matthew A. Bershady,
Michele Cappellari,
Renbin Yan,
Francesco Belfiore,
Dmitry Bizyaev,
Joel R. Brownstein,
Yanping Chen,
Brian Cherinka,
Niv Drory,
Daniel Lazarz,
Shravan Shetty
Abstract:
The SDSS-IV Mapping Nearby Galaxies at APO (MaNGA) program has been operating from 2014-2020, and has now observed a sample of 9,269 galaxies in the low redshift universe (z ~ 0.05) with integral-field spectroscopy. With rest-optical (λλ0.36 - 1.0 um) spectral resolution R ~ 2000 the instrumental spectral line-spread function (LSF) typically has 1sigma width of about 70 km/s, which poses a challen…
▽ More
The SDSS-IV Mapping Nearby Galaxies at APO (MaNGA) program has been operating from 2014-2020, and has now observed a sample of 9,269 galaxies in the low redshift universe (z ~ 0.05) with integral-field spectroscopy. With rest-optical (λλ0.36 - 1.0 um) spectral resolution R ~ 2000 the instrumental spectral line-spread function (LSF) typically has 1sigma width of about 70 km/s, which poses a challenge for the study of the typically 20-30 km/s velocity dispersion of the ionized gas in present-day disk galaxies. In this contribution, we present a major revision of the MaNGA data pipeline architecture, focusing particularly on a variety of factors impacting the effective LSF (e.g., undersampling, spectral rectification, and data cube construction). Through comparison with external assessments of the MaNGA data provided by substantially higher-resolution R ~ 10,000 instruments we demonstrate that the revised MPL-10 pipeline measures the instrumental line spread function sufficiently accurately (<= 0.6% systematic, 2% random around the wavelength of Halpha) that it enables reliable measurements of astrophysical velocity dispersions sigma_Halpha ~ 20 km/s for spaxels with emission lines detected at SNR > 50. Velocity dispersions derived from [O II], Hbeta, [O III], [N II], and [S II] are consistent with those derived from Halpha to within about 2% at sigma_Halpha > 30 km/s. Although the impact of these changes to the estimated LSF will be minimal at velocity dispersions greater than about 100 km/s, scientific results from previous data releases that are based on dispersions far below the instrumental resolution should be reevaulated.
△ Less
Submitted 13 November, 2020; v1 submitted 9 November, 2020;
originally announced November 2020.
-
Deep Learning for Distinguishing Normal versus Abnormal Chest Radiographs and Generalization to Unseen Diseases
Authors:
Zaid Nabulsi,
Andrew Sellergren,
Shahar Jamshy,
Charles Lau,
Edward Santos,
Atilla P. Kiraly,
Wenxing Ye,
Jie Yang,
Rory Pilgrim,
Sahar Kazemzadeh,
Jin Yu,
Sreenivasa Raju Kalidindi,
Mozziyar Etemadi,
Florencia Garcia-Vicente,
David Melnick,
Greg S. Corrado,
Lily Peng,
Krish Eswaran,
Daniel Tse,
Neeral Beladia,
Yun Liu,
Po-Hsuan Cameron Chen,
Shravya Shetty
Abstract:
Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible conditi…
▽ More
Chest radiography (CXR) is the most widely-used thoracic clinical imaging modality and is crucial for guiding the management of cardiothoracic conditions. The detection of specific CXR findings has been the main focus of several artificial intelligence (AI) systems. However, the wide range of possible CXR abnormalities makes it impractical to build specific systems to detect every possible condition. In this work, we developed and evaluated an AI system to classify CXRs as normal or abnormal. For development, we used a de-identified dataset of 248,445 patients from a multi-city hospital network in India. To assess generalizability, we evaluated our system using 6 international datasets from India, China, and the United States. Of these datasets, 4 focused on diseases that the AI was not trained to detect: 2 datasets with tuberculosis and 2 datasets with coronavirus disease 2019. Our results suggest that the AI system generalizes to new patient populations and abnormalities. In a simulated workflow where the AI system prioritized abnormal cases, the turnaround time for abnormal cases reduced by 7-28%. These results represent an important step towards evaluating whether AI can be safely used to flag cases in a general setting where previously unseen abnormalities exist.
△ Less
Submitted 29 October, 2021; v1 submitted 21 October, 2020;
originally announced October 2020.
-
An SDN-IoT-based Framework for Future Smart Cities: Addressing Perspective
Authors:
Uttam Ghosh,
Pushpita Chatterjee,
Sachin Shetty,
Raja Datta
Abstract:
In this Chapter, a software-defined network (SDN)-based framework for future smart cities has been proposed and discussed. It also comprises a distributed addressing scheme to facilitate the allocation of addresses to devices in the smart city dynamically. The framework is dynamic and modules can be added and omitted by a centralized controlling unit without disturbing the other components of the…
▽ More
In this Chapter, a software-defined network (SDN)-based framework for future smart cities has been proposed and discussed. It also comprises a distributed addressing scheme to facilitate the allocation of addresses to devices in the smart city dynamically. The framework is dynamic and modules can be added and omitted by a centralized controlling unit without disturbing the other components of the framework and other modules may be updated accordingly. In the proposed addressing scheme, a new Internet of Things (IoT) device will receive an IP address from one of their existing neighboring peer devices. This allows devices in the city to act as a proxy and generate a set of unique IP addresses from their own IP addresses, which can then be assigned to new (joining) devices; hence, reducing addressing overhead and latency, as well as avoiding the need to send broadcast messages during the address allocation process. Thus, it achieves considerable bandwidth and energy savings for the IoT devices.
△ Less
Submitted 22 July, 2020;
originally announced July 2020.
-
Stellar Population Synthesis with Distinct Kinematics: Multi-Age Asymmetric Drift in SDSS-IV MaNGA Galaxies
Authors:
Shravan Shetty,
Matthew A. Bershady,
Kyle B. Westfall,
Michele Cappellari,
Niv Drory,
David R. Law,
Renbin Yan,
Kevin Bundy
Abstract:
We present the first asymmetric drift (AD) measurements for unresolved stellar populations of different characteristic ages above and below 1.5 Gyr. These measurements sample the age-velocity relation (AVR) in galaxy disks. In this first paper we develop two efficient algorithms to extract AD on a spaxel-by-spaxel basis from optical integral-field spectroscopic (IFS) data-cubes. The algorithms app…
▽ More
We present the first asymmetric drift (AD) measurements for unresolved stellar populations of different characteristic ages above and below 1.5 Gyr. These measurements sample the age-velocity relation (AVR) in galaxy disks. In this first paper we develop two efficient algorithms to extract AD on a spaxel-by-spaxel basis from optical integral-field spectroscopic (IFS) data-cubes. The algorithms apply different spectral templates, one using simple stellar populations and the other a stellar library; their comparison allows us to assess systematic errors in derived multi-component velocities, such as template-mismatch. We test algorithm reliability using mock spectra and Monte Carlo Markov Chains on real data from the MaNGA survey in SDSS-IV. We quantify random and systematic errors in AD as a function of signal-to-noise and stellar population properties with the aim of applying this technique to large subsets of the MaNGA galaxy sample. As a demonstration of our methods, we apply them to an initial sample of seven galaxies with comparable stellar mass and color to the Milky Way. We find a wide range of distinct AD radial profiles for young and old stellar populations.
△ Less
Submitted 3 June, 2020;
originally announced June 2020.
-
A Precise Benchmark for Cluster Scaling Relations: Fundamental Plane, Mass Plane and IMF in the Coma Cluster from Dynamical Models
Authors:
Shravan Shetty,
Michele Cappellari,
Richard M. McDermid,
Davor Krajnovic,
P. T. de Zeeuw,
Roger L. Davies,
Chiaki Kobayashi
Abstract:
We study a sample of 148 early-type galaxies in the Coma cluster using SDSS photometry and spectra, and calibrate our results using detailed dynamical models for a subset of these galaxies, to create a precise benchmark for dynamical scaling relations in high-density environments. For these galaxies, we successfully measured global galaxy properties, modeled stellar populations, and created dynami…
▽ More
We study a sample of 148 early-type galaxies in the Coma cluster using SDSS photometry and spectra, and calibrate our results using detailed dynamical models for a subset of these galaxies, to create a precise benchmark for dynamical scaling relations in high-density environments. For these galaxies, we successfully measured global galaxy properties, modeled stellar populations, and created dynamical models, and support the results using detailed dynamical models of 16 galaxies, including the two most massive cluster galaxies, using data taken with the SAURON IFU. By design, the study provides minimal scatter in derived scaling relations due to the small uncertainty in the relative distances of galaxies compared to the cluster distance. Our results demonstrate low ($\leq$55% for 90$^{th}$ percentile) dark matter fractions in the inner 1$R_{\rm e}$ ~of galaxies. Owing to the study design, we produce the tightest, to our knowledge, IMF-$σ_e$ relation of galaxies, with a slope consistent with that seen in local galaxies. Leveraging our dynamical models, we transform the classical Fundamental Plane of the galaxies to the Mass Plane. We find that the coefficients of the mass plane are close to predictions from the virial theorem, and have significantly lower scatter compared to the Fundamental plane. We show that Coma galaxies occupy similar locations in the (M$_*$ - $R_{\rm e}$) and (M$_*$ - $σ_e$) relations as local field galaxies but are older. This, and the fact we find only three slow rotators in the cluster, is consistent with the scenario of hierarchical galaxy formation and expectations of the kinematic morphology-density relation.
△ Less
Submitted 16 April, 2020;
originally announced April 2020.
-
Implementation of Survivor Detection Strategies Using Drones
Authors:
Sarthak J. Shetty,
Rahul Ravichandran,
Lima Agnel Tony,
N. Sai Abhinay,
Kaushik Das,
Debasish Ghose
Abstract:
Survivors stranded during floods tend to seek refuge on dry land. It is important to search for these survivors and help them reach safety as quickly as possible. The terrain in such situations however, is heavily damaged and restricts the movement of emergency personnel towards these survivors. Therefore, it is advantageous to utilize Unmanned Aerial Vehicles (UAVs) in cooperation with on-ground…
▽ More
Survivors stranded during floods tend to seek refuge on dry land. It is important to search for these survivors and help them reach safety as quickly as possible. The terrain in such situations however, is heavily damaged and restricts the movement of emergency personnel towards these survivors. Therefore, it is advantageous to utilize Unmanned Aerial Vehicles (UAVs) in cooperation with on-ground first responders to aid search and rescue efforts. In this article we demonstrate an implementation and improvement of the weight-based path planning algorithm using an off-the-shelf UAV. The coordinates of the survivor and their heading is reported by an on-ground observer to the UAV to generate a weighted map of the surroundings for exploration. Each coordinate in the map is assigned a weight which dictates the priority of exploration. These waypoints are then sorted on the basis of their weights to arrive at an ordered list for exploration by the UAV. We developed the model in MATLAB, followed by prototyping on Robot Operating System (ROS) using a 3DR Iris quadcopter. We tested the model on an off-the-shelf UAV by utilizing the MAVROS and MAVLINK capabilities of ROS. During the implementation of the algorithm on the UAV, several additional factors such as unreliable GPS signals and limited field of view which could effect the performance of the model were in effect, despite which the algorithm performed fairly well. We compared our model with conventional algorithms described in the literature, and showed that our implementation outperforms them.
△ Less
Submitted 4 April, 2020; v1 submitted 27 March, 2020;
originally announced March 2020.