-
In-situ Water quality monitoring in Oil and Gas operations
Authors:
Satish Kumar,
Rui Kou,
Henry Hill,
Jake Lempges,
Eric Qian,
Vikram Jayaram
Abstract:
From agriculture to mining, to energy, surface water quality monitoring is an essential task. As oil and gas operators work to reduce the consumption of freshwater, it is increasingly important to actively manage fresh and non-fresh water resources over the long term. For large-scale monitoring, manual sampling at many sites has become too time-consuming and unsustainable, given the sheer number o…
▽ More
From agriculture to mining, to energy, surface water quality monitoring is an essential task. As oil and gas operators work to reduce the consumption of freshwater, it is increasingly important to actively manage fresh and non-fresh water resources over the long term. For large-scale monitoring, manual sampling at many sites has become too time-consuming and unsustainable, given the sheer number of dispersed ponds, small lakes, playas, and wetlands over a large area. Therefore, satellite-based environmental monitoring presents great potential. Many existing satellite-based monitoring studies utilize index-based methods to monitor large water bodies such as rivers and oceans. However, these existing methods fail when monitoring small ponds-the reflectance signal received from small water bodies is too weak to detect. To address this challenge, we propose a new Water Quality Enhanced Index (WQEI) Model, which is designed to enable users to determine contamination levels in water bodies with weak reflectance patterns. Our results show that 1) WQEI is a good indicator of water turbidity validated with 1200 water samples measured in the laboratory, and 2) by applying our method to commonly available satellite data (e.g. LandSat8), one can achieve high accuracy water quality monitoring efficiently in large regions. This provides a tool for operators to optimize the quality of water stored within surface storage ponds and increasing the readiness and availability of non-fresh water.
△ Less
Submitted 22 July, 2023; v1 submitted 20 January, 2023;
originally announced January 2023.
-
ClearBuds: Wireless Binaural Earbuds for Learning-Based Speech Enhancement
Authors:
Ishan Chatterjee,
Maruchi Kim,
Vivek Jayaram,
Shyamnath Gollakota,
Ira Kemelmacher-Shlizerman,
Shwetak Patel,
Steven M. Seitz
Abstract:
We present ClearBuds, the first hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and background cancellation, operating in real-time and on a mobile phone. Clear-Buds bridges state-of-the-art deep learning for blind audio source separation and in…
▽ More
We present ClearBuds, the first hardware and software system that utilizes a neural network to enhance speech streamed from two wireless earbuds. Real-time speech enhancement for wireless earbuds requires high-quality sound separation and background cancellation, operating in real-time and on a mobile phone. Clear-Buds bridges state-of-the-art deep learning for blind audio source separation and in-ear mobile systems by making two key technical contributions: 1) a new wireless earbud design capable of operating as a synchronized, binaural microphone array, and 2) a lightweight dual-channel speech enhancement neural network that runs on a mobile device. Our neural network has a novel cascaded architecture that combines a time-domain conventional neural network with a spectrogram-based frequency masking neural network to reduce the artifacts in the audio output. Results show that our wireless earbuds achieve a synchronization error less than 64 microseconds and our network has a runtime of 21.4 milliseconds on an accompanying mobile phone. In-the-wild evaluation with eight users in previously unseen indoor and outdoor multipath scenarios demonstrates that our neural network generalizes to learn both spatial and acoustic cues to perform noise suppression and background speech removal. In a user-study with 37 participants who spent over 15.4 hours rating 1041 audio samples collected in-the-wild, our system achieves improved mean opinion score and background noise suppression.
Project page with demos: https://clearbuds.cs.washington.edu
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets
Authors:
Xiaoxi Wei,
A. Aldo Faisal,
Moritz Grosse-Wentrup,
Alexandre Gramfort,
Sylvain Chevallier,
Vinay Jayaram,
Camille Jeunet,
Stylianos Bakas,
Siegfried Ludwig,
Konstantinos Barmpas,
Mehdi Bahri,
Yannis Panagakis,
Nikolaos Laskaris,
Dimitrios A. Adamos,
Stefanos Zafeiriou,
William C. Duong,
Stephen M. Gordon,
Vernon J. Lawhern,
Maciej Śliwowski,
Vincent Rouanne,
Piotr Tempczyk
Abstract:
Transfer learning and meta-learning offer some of the most promising avenues to unlock the scalability of healthcare and consumer technologies driven by biosignal data. This is because current methods cannot generalise well across human subjects' data and handle learning from different heterogeneously collected data sets, thus limiting the scale of training data. On the other side, developments in…
▽ More
Transfer learning and meta-learning offer some of the most promising avenues to unlock the scalability of healthcare and consumer technologies driven by biosignal data. This is because current methods cannot generalise well across human subjects' data and handle learning from different heterogeneously collected data sets, thus limiting the scale of training data. On the other side, developments in transfer learning would benefit significantly from a real-world benchmark with immediate practical application. Therefore, we pick electroencephalography (EEG) as an exemplar for what makes biosignal machine learning hard. We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI), that have to be solved in the face of low signal-to-noise ratios, major variability among subjects, differences in the data recording sessions and techniques, and even between the specific BCI tasks recorded in the dataset. Task 1 is centred on the field of medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets. The BEETL competition with its over 30 competing teams and its 3 winning entries brought attention to the potential of deep transfer learning and combinations of set theory and conventional machine learning techniques to overcome the challenges. The results set a new state-of-the-art for the real-world BEETL benchmark.
△ Less
Submitted 14 February, 2022;
originally announced February 2022.
-
Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics
Authors:
Vivek Jayaram,
John Thickstun
Abstract:
This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This appr…
▽ More
This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global log-likelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.
△ Less
Submitted 16 December, 2021; v1 submitted 17 May, 2021;
originally announced May 2021.
-
The Cone of Silence: Speech Separation by Localization
Authors:
Teerapat Jenrungrot,
Vivek Jayaram,
Steve Seitz,
Ira Kemelmacher-Shlizerman
Abstract:
Given a multi-microphone recording of an unknown number of speakers talking concurrently, we simultaneously localize the sources and separate the individual speakers. At the core of our method is a deep network, in the waveform domain, which isolates sources within an angular region $θ\pm w/2$, given an angle of interest $θ$ and angular window size $w$. By exponentially decreasing $w$, we can perf…
▽ More
Given a multi-microphone recording of an unknown number of speakers talking concurrently, we simultaneously localize the sources and separate the individual speakers. At the core of our method is a deep network, in the waveform domain, which isolates sources within an angular region $θ\pm w/2$, given an angle of interest $θ$ and angular window size $w$. By exponentially decreasing $w$, we can perform a binary search to localize and separate all sources in logarithmic time. Our algorithm allows for an arbitrary number of potentially moving speakers at test time, including more speakers than seen during training. Experiments demonstrate state-of-the-art performance for both source separation and source localization, particularly in high levels of background noise.
△ Less
Submitted 12 October, 2020;
originally announced October 2020.
-
Background Matting: The World is Your Green Screen
Authors:
Soumyadip Sengupta,
Vivek Jayaram,
Brian Curless,
Steve Seitz,
Ira Kemelmacher-Shlizerman
Abstract:
We propose a method for creating a matte -- the per-pixel foreground color and alpha -- of a person by taking photos or videos in an everyday setting with a handheld camera. Most existing matting methods require a green screen background or a manually created trimap to produce a good matte. Automatic, trimap-free methods are appearing, but are not of comparable quality. In our trimap free approach…
▽ More
We propose a method for creating a matte -- the per-pixel foreground color and alpha -- of a person by taking photos or videos in an everyday setting with a handheld camera. Most existing matting methods require a green screen background or a manually created trimap to produce a good matte. Automatic, trimap-free methods are appearing, but are not of comparable quality. In our trimap free approach, we ask the user to take an additional photo of the background without the subject at the time of capture. This step requires a small amount of foresight but is far less time-consuming than creating a trimap. We train a deep network with an adversarial loss to predict the matte. We first train a matting network with supervised loss on ground truth data with synthetic composites. To bridge the domain gap to real imagery with no labeling, we train another matting network guided by the first network and by a discriminator that judges the quality of composites. We demonstrate results on a wide variety of photos and videos and show significant improvement over the state of the art.
△ Less
Submitted 9 April, 2020; v1 submitted 1 April, 2020;
originally announced April 2020.
-
Real-Time Camera Pose Estimation for Sports Fields
Authors:
Leonardo Citraro,
Pablo Márquez-Neila,
Stefano Savarè,
Vivek Jayaram,
Charles Dubout,
Félix Renaut,
Andrés Hasfura,
Horesh Ben Shitrit,
Pascal Fua
Abstract:
Given an image sequence featuring a portion of a sports field filmed by a moving and uncalibrated camera, such as the one of the smartphones, our goal is to compute automatically in real time the focal length and extrinsic camera parameters for each image in the sequence without using a priori knowledges of the position and orientation of the camera. To this end, we propose a novel framework that…
▽ More
Given an image sequence featuring a portion of a sports field filmed by a moving and uncalibrated camera, such as the one of the smartphones, our goal is to compute automatically in real time the focal length and extrinsic camera parameters for each image in the sequence without using a priori knowledges of the position and orientation of the camera. To this end, we propose a novel framework that combines accurate localization and robust identification of specific keypoints in the image by using a fully convolutional deep architecture. Our algorithm exploits both the field lines and the players' image locations, assuming their ground plane positions to be given, to achieve accuracy and robustness that is beyond the current state of the art. We will demonstrate its effectiveness on challenging soccer, basketball, and volleyball benchmark datasets.
△ Less
Submitted 31 March, 2020;
originally announced March 2020.
-
Source Separation with Deep Generative Priors
Authors:
Vivek Jayaram,
John Thickstun
Abstract:
Despite substantial progress in signal source separation, results for richly structured data continue to contain perceptible artifacts. In contrast, recent deep generative models can produce authentic samples in a variety of domains that are indistinguishable from samples of the data distribution. This paper introduces a Bayesian approach to source separation that uses generative models as priors…
▽ More
Despite substantial progress in signal source separation, results for richly structured data continue to contain perceptible artifacts. In contrast, recent deep generative models can produce authentic samples in a variety of domains that are indistinguishable from samples of the data distribution. This paper introduces a Bayesian approach to source separation that uses generative models as priors over the components of a mixture of sources, and noise-annealed Langevin dynamics to sample from the posterior distribution of sources given a mixture. This decouples the source separation problem from generative modeling, enabling us to directly use cutting-edge generative models as priors. The method achieves state-of-the-art performance for MNIST digit separation. We introduce new methodology for evaluating separation quality on richer datasets, providing quantitative evaluation of separation results on CIFAR-10. We also provide qualitative results on LSUN.
△ Less
Submitted 21 September, 2020; v1 submitted 18 February, 2020;
originally announced February 2020.
-
Tangent space spatial filters for interpretable and efficient Riemannian classification
Authors:
Jiachen Xu,
Moritz Grosse-Wentrup,
Vinay Jayaram
Abstract:
Methods based on Riemannian geometry have proven themselves to be good models for decoding in brain-computer interfacing (BCI). However, one major drawback of these methods is that it is not possible to determine what aspect of the signal the classifier is built on, leaving open the possibility that artifacts drive classification performance. In areas where artifactual control is problematic, spec…
▽ More
Methods based on Riemannian geometry have proven themselves to be good models for decoding in brain-computer interfacing (BCI). However, one major drawback of these methods is that it is not possible to determine what aspect of the signal the classifier is built on, leaving open the possibility that artifacts drive classification performance. In areas where artifactual control is problematic, specifically neurofeedback and BCIs in patient populations, this has led people to continue to rely on spatial filters as a way of generating features that are provably brain-related. Furthermore, these methods also suffer from the curse of dimensionality and are almost infeasible in high-density online BCI systems. To tackle these drawbacks, we introduce here a method for computing spatial filters from any linear function in the Riemannian tangent space, which allows for more efficient classification as well as the removal of artifact sources from classifiers built on Riemannian methods. We first prove a fundamental relationship between certain tangent spaces and spatial filtering methods, including an explanation of common spatial patterns within this framework, and then validate our proposed approach using an open-access BCI analysis framework.
△ Less
Submitted 23 September, 2019;
originally announced September 2019.
-
MOABB: Trustworthy algorithm benchmarking for BCIs
Authors:
Vinay Jayaram,
Alexandre Barachant
Abstract:
BCI algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. By building on recent advances in software for…
▽ More
BCI algorithm development has long been hampered by two major issues: small sample sets and a lack of reproducibility. We offer a solution to both of these problems via a software suite that streamlines both the issues of finding and preprocessing data in a reliable manner, as well as that of using a consistent interface for machine learning methods. By building on recent advances in software for signal analysis implemented in the MNE toolkit, and the unified framework for machine learning offered by the scikit-learn project, we offer a system that can improve BCI algorithm development. This system is fully open-source under the BSD licence and available at https://github.com/NeuroTechX/moabb. To validate our efforts, we analyze a set of state-of-the-art decoding algorithms across 12 open access datasets, with over 250 subjects. Our analysis confirms that different datasets can result in very different results for identical processing pipelines, highlighting the need for trustworthy algorithm benchmarking in the field of BCIs, and further that many previously validated methods do not hold up when applied across different datasets, which has wide-reaching implications for practical BCIs.
△ Less
Submitted 16 May, 2018;
originally announced May 2018.
-
Transfer Learning in Brain-Computer Interfaces
Authors:
Vinay Jayaram,
Morteza Alamgir,
Yasemin Altun,
Bernhard Schölkopf,
Moritz Grosse-Wentrup
Abstract:
The performance of brain-computer interfaces (BCIs) improves with the amount of available training data, the statistical distribution of this data, however, varies across subjects as well as across sessions within individual subjects, limiting the transferability of training data or trained models between them. In this article, we review current transfer learning techniques in BCIs that exploit sh…
▽ More
The performance of brain-computer interfaces (BCIs) improves with the amount of available training data, the statistical distribution of this data, however, varies across subjects as well as across sessions within individual subjects, limiting the transferability of training data or trained models between them. In this article, we review current transfer learning techniques in BCIs that exploit shared structure between training data of multiple subjects and/or sessions to increase performance. We then present a framework for transfer learning in the context of BCIs that can be applied to any arbitrary feature space, as well as a novel regression estimation method that is specifically designed for the structure of a system based on the electroencephalogram (EEG). We demonstrate the utility of our framework and method on subject-to-subject transfer in a motor-imagery paradigm as well as on session-to-session transfer in one patient diagnosed with amyotrophic lateral sclerosis (ALS), showing that it is able to outperform other comparable methods on an identical dataset.
△ Less
Submitted 1 December, 2015;
originally announced December 2015.
