-
Learning to Abstract Visuomotor Mappings using Meta-Reinforcement Learning
Authors:
Carlos A. Velazquez-Vargas,
Isaac Ray Christian,
Jordan A. Taylor,
Sreejan Kumar
Abstract:
We investigated the human capacity to acquire multiple visuomotor mappings for de novo skills. Using a grid navigation paradigm, we tested whether contextual cues implemented as different "grid worlds", allow participants to learn two distinct key-mappings more efficiently. Our results indicate that when contextual information is provided, task performance is significantly better. The same held tr…
▽ More
We investigated the human capacity to acquire multiple visuomotor mappings for de novo skills. Using a grid navigation paradigm, we tested whether contextual cues implemented as different "grid worlds", allow participants to learn two distinct key-mappings more efficiently. Our results indicate that when contextual information is provided, task performance is significantly better. The same held true for meta-reinforcement learning agents that differed in whether or not they receive contextual information when performing the task. We evaluated their accuracy in predicting human performance in the task and analyzed their internal representations. The results indicate that contextual cues allow the formation of separate representations in space and time when using different visuomotor mappings, whereas the absence of them favors sharing one representation. While both strategies can allow learning of multiple visuomotor mappings, we showed contextual cues provide a computational advantage in terms of how many mappings can be learned.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
AI system for fetal ultrasound in low-resource settings
Authors:
Ryan G. Gomes,
Bellington Vwalika,
Chace Lee,
Angelica Willis,
Marcin Sieniek,
Joan T. Price,
Christina Chen,
Margaret P. Kasaro,
James A. Taylor,
Elizabeth M. Stringer,
Scott Mayer McKinney,
Ntazana Sindano,
George E. Dahl,
William Goodnight III,
Justin Gilmer,
Benjamin H. Chi,
Charles Lau,
Terry Spitz,
T Saensuksopa,
Kris Liu,
Jonny Wong,
Rory Pilgrim,
Akib Uddin,
Greg Corrado,
Lily Peng
, et al. (4 additional authors not shown)
Abstract:
Despite considerable progress in maternal healthcare, maternal and perinatal deaths remain high in low-to-middle income countries. Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption. We developed and validated an artificial intelligence (AI) system that uses novice-acquired "blind sweep" ultrasound videos to…
▽ More
Despite considerable progress in maternal healthcare, maternal and perinatal deaths remain high in low-to-middle income countries. Fetal ultrasound is an important component of antenatal care, but shortage of adequately trained healthcare workers has limited its adoption. We developed and validated an artificial intelligence (AI) system that uses novice-acquired "blind sweep" ultrasound videos to estimate gestational age (GA) and fetal malpresentation. We further addressed obstacles that may be encountered in low-resourced settings. Using a simplified sweep protocol with real-time AI feedback on sweep quality, we have demonstrated the generalization of model performance to minimally trained novice ultrasound operators using low cost ultrasound devices with on-device AI integration. The GA model was non-inferior to standard fetal biometry estimates with as few as two sweeps, and the fetal malpresentation model had high AUC-ROCs across operators and devices. Our AI models have the potential to assist in upleveling the capabilities of lightly trained ultrasound operators in low resource settings.
△ Less
Submitted 18 March, 2022;
originally announced March 2022.
-
Online Convex Optimization with Binary Constraints
Authors:
Antoine Lesage-Landry,
Joshua A. Taylor,
Duncan S. Callaway
Abstract:
We consider online optimization with binary decision variables and convex loss functions. We design a new algorithm, binary online gradient descent (bOGD) and bound its expected dynamic regret. We provide a regret bound that holds for any time horizon and a specialized bound for finite time horizons. First, we present the regret as the sum of the relaxed, continuous round optimum tracking error an…
▽ More
We consider online optimization with binary decision variables and convex loss functions. We design a new algorithm, binary online gradient descent (bOGD) and bound its expected dynamic regret. We provide a regret bound that holds for any time horizon and a specialized bound for finite time horizons. First, we present the regret as the sum of the relaxed, continuous round optimum tracking error and the rounding error of our update in which the former asymptomatically decreases with time under certain conditions. Then, we derive a finite-time bound that is sublinear in time and linear in the cumulative variation of the relaxed, continuous round optima. We apply bOGD to demand response with thermostatically controlled loads, in which binary constraints model discrete on/off settings. We also model uncertainty and varying load availability, which depend on temperature deadbands, lockout of cooling units and manual overrides. We test the performance of bOGD in several simulations based on demand response. The simulations corroborate that the use of randomization in bOGD does not significantly degrade performance while making the problem more tractable.
△ Less
Submitted 19 February, 2021; v1 submitted 5 May, 2020;
originally announced May 2020.
-
Predictive Online Convex Optimization
Authors:
Antoine Lesage-Landry,
Iman Shames,
Joshua A. Taylor
Abstract:
We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the loads' behavior, can be used to improve on predictions made with only past observations. Specifically, we introduce an additional predictive step that follows…
▽ More
We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the loads' behavior, can be used to improve on predictions made with only past observations. Specifically, we introduce an additional predictive step that follows the standard online convex optimization step when certain conditions on the estimated gradient and descent direction are met. We show that under these conditions and without any assumptions on the predictability of the environment, the predictive update strictly improves on the performance of the standard update. We give two types of predictive update for various family of loss functions. We provide a regret bound for each of our predictive online convex optimization algorithms. Finally, we apply our framework to an example based on demand response which demonstrates its superior performance to a standard online convex optimization algorithm.
△ Less
Submitted 29 November, 2019; v1 submitted 15 May, 2019;
originally announced May 2019.
-
Price and capacity competition in balancing markets with energy storage
Authors:
Josh A. Taylor,
Johanna L. Mathieu,
Duncan S. Callaway,
Kameshwar Poolla
Abstract:
Energy storage can absorb variability from the rising number of wind and solar power producers. Storage is different from the conventional generators that have traditionally balanced supply and demand on fast time scales due to its hard energy capacity constraints, dynamic coupling, and low marginal costs. These differences are leading system operators to propose new mechanisms for enabling storag…
▽ More
Energy storage can absorb variability from the rising number of wind and solar power producers. Storage is different from the conventional generators that have traditionally balanced supply and demand on fast time scales due to its hard energy capacity constraints, dynamic coupling, and low marginal costs. These differences are leading system operators to propose new mechanisms for enabling storage to participate in reserve and real-time energy markets. The persistence of market power and gaming in electricity markets suggests that these changes will expose new vulnerabilities.
We develop a new model of strategic behavior among storages in energy balancing markets. Our model is a two-stage game that generalizes a classic model of capacity followed by Bertrand-Edgeworth price competition by explicitly modeling storage dynamics and uncertainty in the pricing stage. By applying the model to balancing markets with storage, we are able to compare capacity and energy-based pricing schemes, and to analyze the dynamic effects of the market horizon and energy losses due to leakage. Our first key finding is that capacity pricing leads to higher prices and higher capacity commitments, and that energy pricing leads to lower, randomized prices and lower capacity commitments. Second, we find that a longer market horizon and higher physical efficiencies lead to lower prices by inducing the storage to compete to have their states of charge cycled more frequently.
△ Less
Submitted 1 July, 2015; v1 submitted 23 March, 2015;
originally announced March 2015.
