-
Inpainting Pathology in Lumbar Spine MRI with Latent Diffusion
Authors:
Colin Hansen,
Simas Glinskis,
Ashwin Raju,
Micha Kornreich,
JinHyeong Park,
Jayashri Pawar,
Richard Herzog,
Li Zhang,
Benjamin Odry
Abstract:
Data driven models for automated diagnosis in radiology suffer from insufficient and imbalanced datasets due to low representation of pathology in a population and the cost of expert annotations. Datasets can be bolstered through data augmentation. However, even when utilizing a full suite of transformations during model training, typical data augmentations do not address variations in human anato…
▽ More
Data driven models for automated diagnosis in radiology suffer from insufficient and imbalanced datasets due to low representation of pathology in a population and the cost of expert annotations. Datasets can be bolstered through data augmentation. However, even when utilizing a full suite of transformations during model training, typical data augmentations do not address variations in human anatomy. An alternative direction is to synthesize data using generative models, which can potentially craft datasets with specific attributes. While this holds promise, commonly used generative models such as Generative Adversarial Networks may inadvertently produce anatomically inaccurate features. On the other hand, diffusion models, which offer greater stability, tend to memorize training data, raising concerns about privacy and generative diversity. Alternatively, inpainting has the potential to augment data through directly inserting pathology in medical images. However, this approach introduces a new challenge: accurately merging the generated pathological features with the surrounding anatomical context. While inpainting is a well established method for addressing simple lesions, its application to pathologies that involve complex structural changes remains relatively unexplored. We propose an efficient method for inpainting pathological features onto healthy anatomy in MRI through voxelwise noise scheduling in a latent diffusion model. We evaluate the method's ability to insert disc herniation and central canal stenosis in lumbar spine sagittal T2 MRI, and it achieves superior Frechet Inception Distance compared to state-of-the-art methods.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Does complimentary information from multispectral imaging improve face presentation attack detection?
Authors:
Narayan Vetrekar,
Raghavendra Ramachandra,
Sushma Venkatesh,
Jyoti D. Pawar,
R. S. Gad
Abstract:
Presentation Attack Detection (PAD) has been extensively studied, particularly in the visible spectrum. With the advancement of sensing technology beyond the visible range, multispectral imaging has gained significant attention in this direction. We present PAD based on multispectral images constructed for eight different presentation artifacts resulted from three different artifact species. In th…
▽ More
Presentation Attack Detection (PAD) has been extensively studied, particularly in the visible spectrum. With the advancement of sensing technology beyond the visible range, multispectral imaging has gained significant attention in this direction. We present PAD based on multispectral images constructed for eight different presentation artifacts resulted from three different artifact species. In this work, we introduce Face Presentation Attack Multispectral (FPAMS) database to demonstrate the significance of employing multispectral imaging. The goal of this work is to study complementary information that can be combined in two different ways (image fusion and score fusion) from multispectral imaging to improve the face PAD. The experimental evaluation results present an extensive qualitative analysis of 61650 sample multispectral images collected for bonafide and artifacts. The PAD based on the score fusion and image fusion method presents superior performance, demonstrating the significance of employing multispectral imaging to detect presentation artifacts.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Empirical Analysis of Oral and Nasal Vowels of Konkani
Authors:
Swapnil Fadte,
Edna Vaz,
Atul Kr. Ojha,
Ramdas Karmali,
Jyoti D. Pawar
Abstract:
Konkani is a highly nasalised language which makes it unique among Indo-Aryan languages. This work investigates the acoustic-phonetic properties of Konkani oral and nasal vowels. For this study, speech samples from six speakers (3 male and 3 female) were collected. A total of 74 unique sentences were used as a part of the recording script, 37 each for oral and nasal vowels, respectively. The final…
▽ More
Konkani is a highly nasalised language which makes it unique among Indo-Aryan languages. This work investigates the acoustic-phonetic properties of Konkani oral and nasal vowels. For this study, speech samples from six speakers (3 male and 3 female) were collected. A total of 74 unique sentences were used as a part of the recording script, 37 each for oral and nasal vowels, respectively. The final data set consisted of 1135 vowel phonemes. A comparative F1-F2 plot of Konkani oral and nasal vowels is presented with an experimental result and formant analysis. The average F1, F2 and F3 values are also reported for the first time through experimentation for all nasal and oral vowels. This study can be helpful for the linguistic research on vowels and speech synthesis systems specific to the Konkani language.
△ Less
Submitted 17 May, 2023;
originally announced May 2023.
-
An innovative Deep Learning Based Approach for Accurate Agricultural Crop Price Prediction
Authors:
Mayank Ratan Bhardwaj,
Jaydeep Pawar,
Abhijnya Bhat,
Deepanshu,
Inavamsi Enaganti,
Kartik Sagar,
Y. Narahari
Abstract:
Accurate prediction of agricultural crop prices is a crucial input for decision-making by various stakeholders in agriculture: farmers, consumers, retailers, wholesalers, and the Government. These decisions have significant implications including, most importantly, the economic well-being of the farmers. In this paper, our objective is to accurately predict crop prices using historical price infor…
▽ More
Accurate prediction of agricultural crop prices is a crucial input for decision-making by various stakeholders in agriculture: farmers, consumers, retailers, wholesalers, and the Government. These decisions have significant implications including, most importantly, the economic well-being of the farmers. In this paper, our objective is to accurately predict crop prices using historical price information, climate conditions, soil type, location, and other key determinants of crop prices. This is a technically challenging problem, which has been attempted before. In this paper, we propose an innovative deep learning based approach to achieve increased accuracy in price prediction. The proposed approach uses graph neural networks (GNNs) in conjunction with a standard convolutional neural network (CNN) model to exploit geospatial dependencies in prices. Our approach works well with noisy legacy data and produces a performance that is at least 20% better than the results available in the literature. We are able to predict prices up to 30 days ahead. We choose two vegetables, potato (stable price behavior) and tomato (volatile price behavior) and work with noisy public data available from Indian agricultural markets.
△ Less
Submitted 15 April, 2023;
originally announced April 2023.
-
Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reports
Authors:
Arijit Sehanobish,
McCullen Sandora,
Nabila Abraham,
Jayashri Pawar,
Danielle Torres,
Anasuya Das,
Murray Becker,
Richard Herzog,
Benjamin Odry,
Ron Vianu
Abstract:
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. However, training or fine-tuning these models for individual tasks can be time consuming and resource intensive. Thus, a lot of current research is focused on using transformers for multi-task learning (Raffel et al.,2020) and how to group the tasks to help a multi-task model to learn effect…
▽ More
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. However, training or fine-tuning these models for individual tasks can be time consuming and resource intensive. Thus, a lot of current research is focused on using transformers for multi-task learning (Raffel et al.,2020) and how to group the tasks to help a multi-task model to learn effective representations that can be shared across tasks (Standley et al., 2020; Fifty et al., 2021). In this work, we show that a single multi-tasking model can match the performance of task specific models when the task specific models show similar representations across all of their hidden layers and their gradients are aligned, i.e. their gradients follow the same direction. We hypothesize that the above observations explain the effectiveness of multi-task learning. We validate our observations on our internal radiologist-annotated datasets on the cervical and lumbar spine. Our method is simple and intuitive, and can be used in a wide range of NLP problems.
△ Less
Submitted 5 May, 2022;
originally announced May 2022.
-
Efficient Extraction of Pathologies from C-Spine Radiology Reports using Multi-Task Learning
Authors:
Arijit Sehanobish,
Nathaniel Brown,
Ishita Daga,
Jayashri Pawar,
Danielle Torres,
Anasuya Das,
Murray Becker,
Richard Herzog,
Benjamin Odry,
Ron Vianu
Abstract:
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. Generally, if one has multiple tasks on a given dataset, one may finetune different models or use task specific adapters. In this work, we show that a multi-task model can beat or achieve the performance of multiple BERT-based models finetuned on various tasks and various task specific adapt…
▽ More
Pretrained Transformer based models finetuned on domain specific corpora have changed the landscape of NLP. Generally, if one has multiple tasks on a given dataset, one may finetune different models or use task specific adapters. In this work, we show that a multi-task model can beat or achieve the performance of multiple BERT-based models finetuned on various tasks and various task specific adapter augmented BERT-based models. We validate our method on our internal radiologist's report dataset on cervical spine. We hypothesize that the tasks are semantically close and related and thus multitask learners are powerful classifiers. Our work opens the scope of using our method to radiologist's reports on various body parts.
△ Less
Submitted 9 April, 2022;
originally announced April 2022.
-
Learning Robust Real-Time Cultural Transmission without Human Data
Authors:
Cultural General Intelligence Team,
Avishkar Bhoopchand,
Bethanie Brownfield,
Adrian Collister,
Agustin Dal Lago,
Ashley Edwards,
Richard Everett,
Alexandre Frechette,
Yanko Gitahy Oliveira,
Edward Hughes,
Kory W. Mathewson,
Piermaria Mendolicchio,
Julia Pawar,
Miruna Pislar,
Alex Platonov,
Evan Senter,
Sukhdeep Singh,
Alexander Zacherl,
Lei M. Zhang
Abstract:
Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in arti…
▽ More
Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in artificially intelligent agents. Our agents succeed at real-time cultural transmission from humans in novel contexts without using any pre-collected human data. We identify a surprisingly simple set of ingredients sufficient for generating cultural transmission and develop an evaluation methodology for rigorously assessing it. This paves the way for cultural evolution as an algorithm for developing artificial general intelligence.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
Dynamic Risk Assessment for Geologic CO2 Sequestration
Authors:
Bailian Chen,
Dylan R. Harp,
Yingqi Zhang,
Curtis M. Oldenburg,
Rajesh J. Pawar
Abstract:
At a geologic CO2 sequestration (GCS) site, geologic uncertainty usually leads to large uncertainty in the predictions of properties that influence metrics for leakage risk assessment, such as CO2 saturations and pressures in potentially leaky wellbores, CO2/brine leakage rates, and leakage consequences such as changes in drinking water quality in groundwater aquifers. The large uncertainty in the…
▽ More
At a geologic CO2 sequestration (GCS) site, geologic uncertainty usually leads to large uncertainty in the predictions of properties that influence metrics for leakage risk assessment, such as CO2 saturations and pressures in potentially leaky wellbores, CO2/brine leakage rates, and leakage consequences such as changes in drinking water quality in groundwater aquifers. The large uncertainty in these risk-related system properties and risk metrics can lead to over-conservative risk management decisions to ensure safe operations of GCS sites. The objective of this work is to develop a novel approach based on dynamic risk assessment to effectively reduce the uncertainty in the predicted risk-related system properties and risk metrics. We demonstrate our framework for dynamic risk assessment on two case studies: a 3D synthetic example and a synthetic field example based on the Rock Springs Uplift (RSU) storage site in Wyoming, USA. Results show that the NRAP-Open-IAM risk assessment tool coupled with a conformance evaluation can be used to effectively quantify and reduce the uncertainty in the predictions of risk-related system properties and risk metrics in GCS.
△ Less
Submitted 26 September, 2021;
originally announced September 2021.
-
A Robust Deep Learning Workflow to Predict Multiphase Flow Behavior during Geological CO2 Sequestration Injection and Post-Injection Periods
Authors:
Bicheng Yan,
Bailian Chen,
Dylan Robert Harp,
Rajesh J. Pawar
Abstract:
This paper contributes to the development and evaluation of a deep learning workflow that accurately and efficiently predicts the temporal-spatial evolution of pressure and CO2 plumes during injection and post-injection periods of geologic CO2 sequestration (GCS) operations. Based on a Fourier Neuron Operator, the deep learning workflow takes input variables or features including rock properties,…
▽ More
This paper contributes to the development and evaluation of a deep learning workflow that accurately and efficiently predicts the temporal-spatial evolution of pressure and CO2 plumes during injection and post-injection periods of geologic CO2 sequestration (GCS) operations. Based on a Fourier Neuron Operator, the deep learning workflow takes input variables or features including rock properties, well operational controls and time steps, and predicts the state variables of pressure and CO2 saturation. To further improve the predictive fidelity, separate deep learning models are trained for CO2 injection and post-injection periods due the difference in primary driving force of fluid flow and transport during these two phases. We also explore different combinations of features to predict the state variables. We use a realistic example of CO2 injection and storage in a 3D heterogeneous saline aquifer, and apply the deep learning workflow that is trained from physics-based simulation data and emulate the physics process. Through this numerical experiment, we demonstrate that using two separate deep learning models to distinguish post-injection from injection period generates the most accurate prediction of pressure, and a single deep learning model of the whole GCS process including the cumulative injection volume of CO2 as a deep learning feature, leads to the most accurate prediction of CO2 saturation. For the post-injection period, it is key to use cumulative CO2 injection volume to inform the deep learning models about the total carbon storage when predicting either pressure or saturation. The deep learning workflow not only provides high predictive fidelity across temporal and spatial scales, but also offers a speedup of 250 times compared to full physics reservoir simulation, and thus will be a significant predictive tool for engineers to manage the long term process of GCS.
△ Less
Submitted 15 July, 2021;
originally announced July 2021.
-
Improving Deep Learning Performance for Predicting Large-Scale Porous-Media Flow through Feature Coarsening
Authors:
Bicheng Yan,
Dylan Robert Harp,
Bailian Chen,
Rajesh J. Pawar
Abstract:
Physics-based simulation for fluid flow in porous media is a computational technology to predict the temporal-spatial evolution of state variables (e.g. pressure) in porous media, and usually requires high computational expense due to its nonlinearity and the scale of the study domain. This letter describes a deep learning (DL) workflow to predict the pressure evolution as fluid flows in large-sca…
▽ More
Physics-based simulation for fluid flow in porous media is a computational technology to predict the temporal-spatial evolution of state variables (e.g. pressure) in porous media, and usually requires high computational expense due to its nonlinearity and the scale of the study domain. This letter describes a deep learning (DL) workflow to predict the pressure evolution as fluid flows in large-scale 3D heterogeneous porous media. In particular, we apply feature coarsening technique to extract the most representative information and perform the training and prediction of DL at the coarse scale, and further recover the resolution at the fine scale by 2D piecewise cubic interpolation. We validate the DL approach that is trained from physics-based simulation data to predict pressure field in a field-scale 3D geologic CO_2 storage reservoir. We evaluate the impact of feature coarsening on DL performance, and observe that the feature coarsening can not only decrease training time by >74% and reduce memory consumption by >75%, but also maintains temporal error <1.5%. Besides, the DL workflow provides predictive efficiency with ~1400 times speedup compared to physics-based simulation.
△ Less
Submitted 8 May, 2021;
originally announced May 2021.
-
A Gradient-based Deep Neural Network Model for Simulating Multiphase Flow in Porous Media
Authors:
Bicheng Yan,
Dylan Robert Harp,
Rajesh J. Pawar
Abstract:
Simulation of multiphase flow in porous media is crucial for the effective management of subsurface energy and environment related activities. The numerical simulators used for modeling such processes rely on spatial and temporal discretization of the governing partial-differential equations (PDEs) into algebraic systems via numerical methods. These simulators usually require dedicated software de…
▽ More
Simulation of multiphase flow in porous media is crucial for the effective management of subsurface energy and environment related activities. The numerical simulators used for modeling such processes rely on spatial and temporal discretization of the governing partial-differential equations (PDEs) into algebraic systems via numerical methods. These simulators usually require dedicated software development and maintenance, and suffer low efficiency from a runtime and memory standpoint. Therefore, developing cost-effective, data-driven models can become a practical choice since deep learning approaches are considered to be universal approximations. In this paper, we describe a gradient-based deep neural network (GDNN) constrained by the physics related to multiphase flow in porous media. We tackle the nonlinearity of flow in porous media induced by rock heterogeneity, fluid properties and fluid-rock interactions by decomposing the nonlinear PDEs into a dictionary of elementary differential operators. We use a combination of operators to handle rock spatial heterogeneity and fluid flow by advection. Since the augmented differential operators are inherently related to the physics of fluid flow, we treat them as first principles prior knowledge to regularize the GDNN training. We use the example of pressure management at geologic CO2 storage sites, where CO2 is injected in saline aquifers and brine is produced, and apply GDNN to construct a predictive model that is trained from physics-based simulation data and emulates the physics process. We demonstrate that GDNN can effectively predict the nonlinear patterns of subsurface responses including the temporal-spatial evolution of the pressure and saturation plumes. GDNN has great potential to tackle challenging problems that are governed by highly nonlinear physics and enables development of data-driven models with higher fidelity.
△ Less
Submitted 29 April, 2021;
originally announced May 2021.
