Showing 1–28 of 28 results for author: Viola, F

Ultra-thin transistors and circuits for conformable electronics

Authors: Federico Parenti, Riccardo Sargeni, Elisabetta Dimaggio, Francesco Pieri, Filippo Fabbri, Tommaso Losi, Fabrizio Antonio Viola, Arindam Bala, Zhenyu Wang, Andras Kis, Mario Caironi, Gianluca Fiori

Abstract: Adapting electronics to perfectly conform to non-planar and rough surfaces, such as human skin, is a very challenging task which, if solved, could open up new applications in fields of high economic and scientific interest ranging from health to robotics, wearable electronics, human machine interface and Internet of Things. The key to success lies in defining a technology that can lead to the fabr… ▽ More Adapting electronics to perfectly conform to non-planar and rough surfaces, such as human skin, is a very challenging task which, if solved, could open up new applications in fields of high economic and scientific interest ranging from health to robotics, wearable electronics, human machine interface and Internet of Things. The key to success lies in defining a technology that can lead to the fabrication of ultra-thin devices while exploiting materials that are ultimately thin, with high mechanical flexibility and excellent electrical properties. Here, we report a hybrid approach for the definition of high-performance, ultra-thin and conformable electronic devices and circuits, based on the integration of ultimately thin semiconducting transition metal dichalcogenides (TMDC), i.e., MoS2, with organic gate dielectric material, i.e., polyvinyl formal (PVF) combined with the ink-jet printing of conductive PEDOT:PSS ink for electrodes definition. Through this cost-effective, fully bottom-up and solution-based approach, transistors and simple digital and analogue circuits are fabricated by a sequential stacking of ultrathin (nanometer) layers on a few micron thick polyimide substrate, which guarantees the high flexibility mandatory for the targeted applications. △ Less

Submitted 24 June, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

Authors: Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, Andrea Tacchetti, Colin Gaffney, Samira Daruki, Olcan Sercinoglu, Zach Gleicher, Juliette Love , et al. (1092 additional authors not shown)

Abstract: In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February… ▽ More In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content. △ Less

Submitted 14 June, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

Gemini: A Family of Highly Capable Multimodal Models

Authors: Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee , et al. (1325 additional authors not shown)

Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr… ▽ More This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI. △ Less

Submitted 17 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

A revised gap-averaged Floquet analysis of Faraday waves in Hele-Shaw cells

Authors: Alessandro Bongarzone, Baptiste Jouron, Francesco Viola, François Gallaire

Abstract: Existing theoretical analyses of Faraday waves in Hele-Shaw cells rely on the Darcy approximation and assume a parabolic flow profile in the narrow direction. However, Darcy's model is known to be inaccurate when convective or unsteady inertial effects are important. In this work, we propose a gap-averaged Floquet theory accounting for inertial effects induced by the unsteady terms in the Navier-S… ▽ More Existing theoretical analyses of Faraday waves in Hele-Shaw cells rely on the Darcy approximation and assume a parabolic flow profile in the narrow direction. However, Darcy's model is known to be inaccurate when convective or unsteady inertial effects are important. In this work, we propose a gap-averaged Floquet theory accounting for inertial effects induced by the unsteady terms in the Navier-Stokes equations, a scenario that corresponds to a pulsatile flow where the fluid motion reduces to a two-dimensional oscillating Poiseuille flow, similarly to the Womersley flow in arteries. When gap-averaging the linearized Navier-Stokes equation, this results in a modified damping coefficient, which is a function of the ratio between the Stokes boundary layer thickness and the cell's gap, and whose complex value depends on the frequency of the wave response specific to each unstable parametric region. We first revisit the standard case of horizontally infinite rectangular Hele-Shaw cells by also accounting for a dynamic contact angle model. A comparison with existing experiments shows the predictive improvement brought by the present theory and points out how the standard gap-averaged model often underestimates the Faraday threshold. The analysis is then extended to the less conventional case of thin annuli. A series of dedicated experiments for this configuration highlights how Darcy's thin-gap approximation overlooks a frequency detuning that is essential to correctly predict the locations of the Faraday tongues in the frequency-amplitude parameter plane. These findings are well rationalized and captured by the present model. △ Less

Submitted 20 June, 2023; originally announced June 2023.

Sub-harmonic parametric instability in nearly-brimful circular-cylinders: a weakly nonlinear analysis

Authors: Alessandro Bongarzone, Francesco Viola, Simone Camarri, François Gallaire

Abstract: In lab-scale Faraday experiments, meniscus waves respond harmonically to small-amplitude forcing without threshold, hence potentially cloaking the instability onset of parametric waves. Their suppression can be achieved by resorting to a contact line pinned at the container brim with static contact angle $θ_s=90^{\circ}$ (brimful condition). However, tunable meniscus waves are desired in some appl… ▽ More In lab-scale Faraday experiments, meniscus waves respond harmonically to small-amplitude forcing without threshold, hence potentially cloaking the instability onset of parametric waves. Their suppression can be achieved by resorting to a contact line pinned at the container brim with static contact angle $θ_s=90^{\circ}$ (brimful condition). However, tunable meniscus waves are desired in some applications as those of liquid-based biosensors, where they can be controlled adjusting the shape of the static meniscus by slightly under/over-filling the vessel ($θ_s\ne90^{\circ}$) while keeping the contact line fixed at the brim. Here, we refer to this wetting condition as nearly-brimful. Although classic inviscid theories based on Floquet analysis have been reformulated for the case of a pinned contact line (Kidambi 2013), accounting for (i) viscous dissipation and (ii) static contact angle effects, including meniscus waves, makes such analyses practically intractable and a comprehensive theoretical framework is still lacking. Aiming at filling this gap, in this work we formalize a weakly nonlinear analysis via multiple timescale method capable to predict the impact of (i) and (ii) on the instability onset of viscous sub-harmonic standing waves in both brimful and nearly-brimful circular-cylinders. Notwithstanding that the form of the resulting amplitude equation is in fact analogous to that obtained by symmetry arguments (Douady 1990), the normal form coefficients are here computed numerically from first principles, thus allowing us to rationalize and systematically quantify the modifications on the Faraday tongues and on the associated bifurcation diagrams induced by the interaction of meniscus and sub-harmonic parametric waves. △ Less

Submitted 28 February, 2022; originally announced February 2022.

A fast computational model for the electrophysiology of the whole human heart

Authors: Giulio Del Corso, Roberto Verzicco, Francesco Viola

Abstract: In this study we present a novel computational model for unprecedented simulations of the whole cardiac electrophysiology. According to the heterogeneous electrophysiologic properties of the heart, the whole cardiac geometry is decomposed into a set of coupled conductive media having different topology and electrical conductivities: (i) a network of slender bundles comprising a fast conduction atr… ▽ More In this study we present a novel computational model for unprecedented simulations of the whole cardiac electrophysiology. According to the heterogeneous electrophysiologic properties of the heart, the whole cardiac geometry is decomposed into a set of coupled conductive media having different topology and electrical conductivities: (i) a network of slender bundles comprising a fast conduction atrial network, the AV-node and the ventricular bundles; (ii) the Purkinje network; and (iii) the atrial and ventricular myocardium. The propagation of the action potential in these conductive media is governed by the bidomain/monodomain equations, which are discretized in space using an in-house finite volume method and coupled to three different cellular models, the Courtemanche model [1] for the atrial myocytes, the Stewart model [2] for the Purkinje Network and the ten Tusscher-Panfilov model [3] for the ventricular myocytes. The developed numerical model correctly reproduces the cardiac electrophysiology of the whole human heart in healthy and pathologic conditions and it can be tailored to study and optimize resynchronization therapies or invasive surgical procedures. Importantly, the whole solver is GPU-accelerated using CUDA Fortran providing an unprecedented speedup, thus opening the way for systematic parametric studies and uncertainty quantification analyses. △ Less

Submitted 23 December, 2021; originally announced December 2021.

Podracer architectures for scalable Reinforcement Learning

Authors: Matteo Hessel, Manuel Kroiss, Aidan Clark, Iurii Kemaev, John Quan, Thomas Keck, Fabio Viola, Hado van Hasselt

Abstract: Supporting state-of-the-art AI research requires balancing rapid prototyping, ease of use, and quick iteration, with the ability to deploy experiments at a scale traditionally associated with production systems.Deep learning frameworks such as TensorFlow, PyTorch and JAX allow users to transparently make use of accelerators, such as TPUs and GPUs, to offload the more computationally intensive part… ▽ More Supporting state-of-the-art AI research requires balancing rapid prototyping, ease of use, and quick iteration, with the ability to deploy experiments at a scale traditionally associated with production systems.Deep learning frameworks such as TensorFlow, PyTorch and JAX allow users to transparently make use of accelerators, such as TPUs and GPUs, to offload the more computationally intensive parts of training and inference in modern deep learning systems. Popular training pipelines that use these frameworks for deep learning typically focus on (un-)supervised learning. How to best train reinforcement learning (RL) agents at scale is still an active research area. In this report we argue that TPUs are particularly well suited for training RL agents in a scalable, efficient and reproducible way. Specifically we describe two architectures designed to make the best use of the resources available on a TPU Pod (a special configuration in a Google data center that features multiple TPU devices connected to each other by extremely low latency communication channels). △ Less

Submitted 13 April, 2021; originally announced April 2021.

Muesli: Combining Improvements in Policy Optimization

Authors: Matteo Hessel, Ivo Danihelka, Fabio Viola, Arthur Guez, Simon Schmitt, Laurent Sifre, Theophane Weber, David Silver, Hado van Hasselt

Abstract: We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss. The update (henceforth Muesli) matches MuZero's state-of-the-art performance on Atari. Notably, Muesli does so without using deep search: it acts directly with a policy network and has computation speed comparable to model-free baselines. The Atari results are complemented by ex… ▽ More We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss. The update (henceforth Muesli) matches MuZero's state-of-the-art performance on Atari. Notably, Muesli does so without using deep search: it acts directly with a policy network and has computation speed comparable to model-free baselines. The Atari results are complemented by extensive ablations, and by additional results on continuous control and 9x9 Go. △ Less

Submitted 31 March, 2022; v1 submitted 13 April, 2021; originally announced April 2021.

FSEI-GPU: GPU accelerated simulations of the fluid-structure-electrophysiology interaction in the left heart

Authors: Francesco Viola, Vamsi Spandan, Valentina Meschini, Joshua Romero, Massimiliano Fatica, Marco D. de Tullio, Roberto Verzicco

Abstract: The reliability of cardiovascular computational models depends on the accurate solution of the hemodynamics, the realistic characterization of the hyperelastic and electric properties of the tissues along with the correct description of their interaction. The resulting fluid-structure-electrophysiology interaction (FSEI) thus requires an immense computational power, usually available in large supe… ▽ More The reliability of cardiovascular computational models depends on the accurate solution of the hemodynamics, the realistic characterization of the hyperelastic and electric properties of the tissues along with the correct description of their interaction. The resulting fluid-structure-electrophysiology interaction (FSEI) thus requires an immense computational power, usually available in large supercomputing centers, and requires long time to obtain results even if multi-CPU processors are used (MPI acceleration). In recent years, graphics processing units (GPUs) have emerged as a convenient platform for high performance computing, as they allow for considerable reductions of the time-to-solution. This approach is particularly appealing if the tool has to support medical decisions that require solutions within reduced times and possibly obtained by local computational resources. Accordingly, our multi-physics solver has been ported to GPU architectures using CUDA Fortran to tackle fast and accurate hemodynamics simulations of the human heart without resorting to large-scale supercomputers. This work describes the use of CUDA to accelerate the FSEI on heterogeneous clusters, where both the CPUs and GPUs are used in synergistically with minor modifications of the original source code. The resulting GPU accelerated code solves a single heartbeat within a few hours (from three to ten depending on the grid resolution) running on premises computing facility made of few GPU cards, which can be easily installed in a medical laboratory or in a hospital, thus opening towards a systematic computational fluid dynamics (CFD) aided diagnostic. △ Less

Submitted 4 May, 2021; v1 submitted 28 March, 2021; originally announced March 2021.

Effects of stenotic aortic valve on the left heart hemodynamics: a fluid-structure-electrophysiology approach

Authors: Francesco Viola, Valentina Meschini, Roberto Verzicco

Abstract: The aortic valve is a three-leaflet passive structure that, driven by pressure differences between the left ventricle and the aorta, opens and closes during the heartbeat to ensure the correct stream direction and flow rate. In elderly individuals or because of particular pathologies, the valve leaflets can stiffen thus impairing the valve functioning and, in turn, the pumping efficiency of the he… ▽ More The aortic valve is a three-leaflet passive structure that, driven by pressure differences between the left ventricle and the aorta, opens and closes during the heartbeat to ensure the correct stream direction and flow rate. In elderly individuals or because of particular pathologies, the valve leaflets can stiffen thus impairing the valve functioning and, in turn, the pumping efficiency of the heart. Using a multi-physics left heart model accounting for the electrophysiology, the active contraction of the myocardium, the hemodynamics and the related fluid-structure-interaction, we have investigated the changes in the flow features for different severities of the aortic valve stenosis. We have found that, in addition to the increase of the transvalvular pressure drop and of the systolic jet velocity, a stenotic aortic valve significantly alters the wall shear stresses and their spatial distribution over the aortic arch and valve leaflets, which may induce a remodelling process of the ventricular myocardium. The numerical results from the multi-physics model are fully consistent with the clinical experience, thus further opening the way for computational engineering aided medical diagnostic. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Direct numerical simulation of flapping flags in grid-induced turbulence

Authors: Stefano Olivieri, Francesco Viola, Andrea Mazzino, Marco E. Rosti

Abstract: A fully-resolved direct-numerical-simulation (DNS) approach for investigating flexible bodies forced by a turbulent incoming flow is designed to study the flapping motion of a flexible flag at moderate Reynolds number. The incoming turbulent flow is generated by placing a passive grid at the inlet of the numerical domain and the turbulence level of the flow impacting the flag can be controlled by… ▽ More A fully-resolved direct-numerical-simulation (DNS) approach for investigating flexible bodies forced by a turbulent incoming flow is designed to study the flapping motion of a flexible flag at moderate Reynolds number. The incoming turbulent flow is generated by placing a passive grid at the inlet of the numerical domain and the turbulence level of the flow impacting the flag can be controlled by changing its downstream distance from the grid. The computational framework is based on the immersed boundary method for dealing with arbitrary geometries and implemented using a graphics-processing-unit (GPU) accelerated parallelisation to increase the computational efficiency. The grid-induced turbulent flow is first characterised by means of the comparison with well-known results for decaying turbulence and a scale-by-scale analysis. Then, the flag-in-the-wind problem is revisited by exploring the effect of the turbulence intensity on self-sustained flapping. Whilst the latter is still manifesting under strong fluctuations, the main features of the oscillation (including its amplitude and frequency) are altered by turbulence, whose fingerprint can also be qualitatively detected by spectral analysis. Besides their relevance for advancing the fundamental understanding of fluid-structure interaction in turbulence, these findings have potential impact for related applications, e.g., aeroelastic energy harvesting or flow control techniques. △ Less

Submitted 20 July, 2021; v1 submitted 15 March, 2021; originally announced March 2021.

Counterfactual Credit Assignment in Model-Free Reinforcement Learning

Authors: Thomas Mesnard, Théophane Weber, Fabio Viola, Shantanu Thakoor, Alaa Saade, Anna Harutyunyan, Will Dabney, Tom Stepleton, Nicolas Heess, Arthur Guez, Éric Moulines, Marcus Hutter, Lars Buesing, Rémi Munos

Abstract: Credit assignment in reinforcement learning is the problem of measuring an action's influence on future rewards. In particular, this requires separating skill from luck, i.e. disentangling the effect of an action on rewards from that of external factors and subsequent actions. To achieve this, we adapt the notion of counterfactuals from causality theory to a model-free RL setup. The key idea is to… ▽ More Credit assignment in reinforcement learning is the problem of measuring an action's influence on future rewards. In particular, this requires separating skill from luck, i.e. disentangling the effect of an action on rewards from that of external factors and subsequent actions. To achieve this, we adapt the notion of counterfactuals from causality theory to a model-free RL setup. The key idea is to condition value functions on future events, by learning to extract relevant information from a trajectory. We formulate a family of policy gradient algorithms that use these future-conditional value functions as baselines or critics, and show that they are provably low variance. To avoid the potential bias from conditioning on future information, we constrain the hindsight information to not contain information about the agent's actions. We demonstrate the efficacy and validity of our algorithm on a number of illustrative and challenging problems. △ Less

Submitted 14 December, 2021; v1 submitted 18 November, 2020; originally announced November 2020.

On the role of planning in model-based deep reinforcement learning

Authors: Jessica B. Hamrick, Abram L. Friesen, Feryal Behbahani, Arthur Guez, Fabio Viola, Sims Witherspoon, Thomas Anthony, Lars Buesing, Petar Veličković, Théophane Weber

Abstract: Model-based planning is often thought to be necessary for deep, careful reasoning and generalization in artificial agents. While recent successes of model-based reinforcement learning (MBRL) with deep function approximation have strengthened this hypothesis, the resulting diversity of model-based methods has also made it difficult to track which components drive success and why. In this paper, we… ▽ More Model-based planning is often thought to be necessary for deep, careful reasoning and generalization in artificial agents. While recent successes of model-based reinforcement learning (MBRL) with deep function approximation have strengthened this hypothesis, the resulting diversity of model-based methods has also made it difficult to track which components drive success and why. In this paper, we seek to disentangle the contributions of recent methods by focusing on three questions: (1) How does planning benefit MBRL agents? (2) Within planning, what choices drive performance? (3) To what extent does planning improve generalization? To answer these questions, we study the performance of MuZero (Schrittwieser et al., 2019), a state-of-the-art MBRL algorithm with strong connections and overlapping components with many other MBRL algorithms. We perform a number of interventions and ablations of MuZero across a wide range of environments, including control tasks, Atari, and 9x9 Go. Our results suggest the following: (1) Planning is most useful in the learning process, both for policy updates and for providing a more useful data distribution. (2) Using shallow trees with simple Monte-Carlo rollouts is as performant as more complex methods, except in the most difficult reasoning tasks. (3) Planning alone is insufficient to drive strong generalization. These results indicate where and how to utilize planning in reinforcement learning settings, and highlight a number of open questions for future MBRL research. △ Less

Submitted 17 March, 2021; v1 submitted 8 November, 2020; originally announced November 2020.

Comments: Published at ICLR 2021

Comparison of Evolving Granular Classifiers applied to Anomaly Detection for Predictive Maintenance in Computing Centers

Authors: Leticia Decker, Daniel Leite, Fabio Viola, Daniele Bonacorsi

Abstract: Log-based predictive maintenance of computing centers is a main concern regarding the worldwide computing grid that supports the CERN (European Organization for Nuclear Research) physics experiments. A log, as event-oriented adhoc information, is quite often given as unstructured big data. Log data processing is a time-consuming computational task. The goal is to grab essential information from a… ▽ More Log-based predictive maintenance of computing centers is a main concern regarding the worldwide computing grid that supports the CERN (European Organization for Nuclear Research) physics experiments. A log, as event-oriented adhoc information, is quite often given as unstructured big data. Log data processing is a time-consuming computational task. The goal is to grab essential information from a continuously changeable grid environment to construct a classification model. Evolving granular classifiers are suited to learn from time-varying log streams and, therefore, perform online classification of the severity of anomalies. We formulated a 4-class online anomaly classification problem, and employed time windows between landmarks and two granular computing methods, namely, Fuzzy-set-Based evolving Modeling (FBeM) and evolving Granular Neural Network (eGNN), to model and monitor logging activity rate. The results of classification are of utmost importance for predictive maintenance because priority can be given to specific time intervals in which the classifier indicates the existence of high or medium severity anomalies. △ Less

Submitted 8 April, 2020; originally announced May 2020.

Comments: 8 pages, 8 figures, IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS 2020)

Neural Communication Systems with Bandwidth-limited Channel

Authors: Karen Ullrich, Fabio Viola, Danilo Jimenez Rezende

Abstract: Reliably transmitting messages despite information loss due to a noisy channel is a core problem of information theory. One of the most important aspects of real world communication, e.g. via wifi, is that it may happen at varying levels of information transfer. The bandwidth-limited channel models this phenomenon. In this study we consider learning coding with the bandwidth-limited channel (BWLC)… ▽ More Reliably transmitting messages despite information loss due to a noisy channel is a core problem of information theory. One of the most important aspects of real world communication, e.g. via wifi, is that it may happen at varying levels of information transfer. The bandwidth-limited channel models this phenomenon. In this study we consider learning coding with the bandwidth-limited channel (BWLC). Recently, neural communication models such as variational autoencoders have been studied for the task of source compression. We build upon this work by studying neural communication systems with the BWLC. Specifically,we find three modelling choices that are relevant under expected information loss. First, instead of separating the sub-tasks of compression (source coding) and error correction (channel coding), we propose to model both jointly. Framing the problem as a variational learning problem, we conclude that joint systems outperform their separate counterparts when coding is performed by flexible learnable function approximators such as neural networks. To facilitate learning, we introduce a differentiable and computationally efficient version of the bandwidth-limited channel. Second, we propose a design to model missing information with a prior, and incorporate this into the channel model. Finally, sampling from the joint model is improved by introducing auxiliary latent variables in the decoder. Experimental results justify the validity of our design decisions through improved distortion and FID scores. △ Less

Submitted 1 April, 2020; v1 submitted 30 March, 2020; originally announced March 2020.

Value-driven Hindsight Modelling

Authors: Arthur Guez, Fabio Viola, Théophane Weber, Lars Buesing, Steven Kapturowski, Doina Precup, David Silver, Nicolas Heess

Abstract: Value estimation is a critical component of the reinforcement learning (RL) paradigm. The question of how to effectively learn value predictors from data is one of the major problems studied by the RL community, and different approaches exploit structure in the problem domain in different ways. Model learning can make use of the rich transition structure present in sequences of observations, but t… ▽ More Value estimation is a critical component of the reinforcement learning (RL) paradigm. The question of how to effectively learn value predictors from data is one of the major problems studied by the RL community, and different approaches exploit structure in the problem domain in different ways. Model learning can make use of the rich transition structure present in sequences of observations, but this approach is usually not sensitive to the reward function. In contrast, model-free methods directly leverage the quantity of interest from the future, but receive a potentially weak scalar signal (an estimate of the return). We develop an approach for representation learning in RL that sits in between these two extremes: we propose to learn what to model in a way that can directly help value prediction. To this end, we determine which features of the future trajectory provide useful information to predict the associated return. This provides tractable prediction targets that are directly relevant for a task, and can thus accelerate learning the value function. The idea can be understood as reasoning, in hindsight, about which aspects of the future observations could help past value prediction. We show how this can help dramatically even in simple policy evaluation settings. We then test our approach at scale in challenging domains, including on 57 Atari 2600 games. △ Less

Submitted 20 October, 2020; v1 submitted 19 February, 2020; originally announced February 2020.

Comments: 9 pages + reference + appendix. NeurIPS 2020 version

Causally Correct Partial Models for Reinforcement Learning

Authors: Danilo J. Rezende, Ivo Danihelka, George Papamakarios, Nan Rosemary Ke, Ray Jiang, Theophane Weber, Karol Gregor, Hamza Merzic, Fabio Viola, Jane Wang, Jovana Mitrovic, Frederic Besse, Ioannis Antonoglou, Lars Buesing

Abstract: In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can be computationally expensive or even intractable if the observations are high-dimensional (e.g. images). For this reason, previous works have considered partial models, which model only part of the observation. In this pa… ▽ More In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can be computationally expensive or even intractable if the observations are high-dimensional (e.g. images). For this reason, previous works have considered partial models, which model only part of the observation. In this paper, we show that partial models can be causally incorrect: they are confounded by the observations they don't model, and can therefore lead to incorrect planning. To address this, we introduce a general family of partial models that are provably causally correct, yet remain fast because they do not need to fully model future observations. △ Less

Submitted 7 February, 2020; originally announced February 2020.

TF-Replicator: Distributed Machine Learning for Researchers

Authors: Peter Buchlovsky, David Budden, Dominik Grewe, Chris Jones, John Aslanides, Frederic Besse, Andy Brock, Aidan Clark, Sergio Gómez Colmenarejo, Aedan Pope, Fabio Viola, Dan Belov

Abstract: We describe TF-Replicator, a framework for distributed machine learning designed for DeepMind researchers and implemented as an abstraction over TensorFlow. TF-Replicator simplifies writing data-parallel and model-parallel research code. The same models can be effortlessly deployed to different cluster architectures (i.e. one or many machines containing CPUs, GPUs or TPU accelerators) using synchr… ▽ More We describe TF-Replicator, a framework for distributed machine learning designed for DeepMind researchers and implemented as an abstraction over TensorFlow. TF-Replicator simplifies writing data-parallel and model-parallel research code. The same models can be effortlessly deployed to different cluster architectures (i.e. one or many machines containing CPUs, GPUs or TPU accelerators) using synchronous or asynchronous training regimes. To demonstrate the generality and scalability of TF-Replicator, we implement and benchmark three very different models: (1) A ResNet-50 for ImageNet classification, (2) a SN-GAN for class-conditional ImageNet image generation, and (3) a D4PG reinforcement learning agent for continuous control. Our results show strong scalability performance without demanding any distributed systems expertise of the user. The TF-Replicator programming model will be open-sourced as part of TensorFlow 2.0 (see https://github.com/tensorflow/community/pull/25). △ Less

Submitted 1 February, 2019; originally announced February 2019.

Taming VAEs

Authors: Danilo Jimenez Rezende, Fabio Viola

Abstract: In spite of remarkable progress in deep latent variable generative modeling, training still remains a challenge due to a combination of optimization and generalization issues. In practice, a combination of heuristic algorithms (such as hand-crafted annealing of KL-terms) is often used in order to achieve the desired results, but such solutions are not robust to changes in model architecture or dat… ▽ More In spite of remarkable progress in deep latent variable generative modeling, training still remains a challenge due to a combination of optimization and generalization issues. In practice, a combination of heuristic algorithms (such as hand-crafted annealing of KL-terms) is often used in order to achieve the desired results, but such solutions are not robust to changes in model architecture or dataset. The best settings can often vary dramatically from one problem to another, which requires doing expensive parameter sweeps for each new case. Here we develop on the idea of training VAEs with additional constraints as a way to control their behaviour. We first present a detailed theoretical analysis of constrained VAEs, expanding our understanding of how these models work. We then introduce and analyze a practical algorithm termed Generalized ELBO with Constrained Optimization, GECO. The main advantage of GECO for the machine learning practitioner is a more intuitive, yet principled, process of tuning the loss. This involves defining of a set of constraints, which typically have an explicit relation to the desired model performance, in contrast to tweaking abstract hyper-parameters which implicitly affect the model behavior. Encouraging experimental results in several standard datasets indicate that GECO is a very robust and effective tool to balance reconstruction and compression constraints. △ Less

Submitted 1 October, 2018; originally announced October 2018.

Learning models for visual 3D localization with implicit mapping

Authors: Dan Rosenbaum, Frederic Besse, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami

Abstract: We consider learning based methods for visual localization that do not require the construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level. We propose to use a generative approach based on Generative Query Networks (GQNs, Eslami et al. 2018), asking the following questions: 1) Can GQN c… ▽ More We consider learning based methods for visual localization that do not require the construction of explicit maps in the form of point clouds or voxels. The goal is to learn an implicit representation of the environment at a higher, more abstract level. We propose to use a generative approach based on Generative Query Networks (GQNs, Eslami et al. 2018), asking the following questions: 1) Can GQN capture more complex scenes than those it was originally demonstrated on? 2) Can GQN be used for localization in those scenes? To study this approach we consider procedurally generated Minecraft worlds, for which we can generate images of complex 3D scenes along with camera pose coordinates. We first show that GQNs, enhanced with a novel attention mechanism can capture the structure of 3D scenes in Minecraft, as evidenced by their samples. We then apply the models to the localization problem, comparing the results to a discriminative baseline, and comparing the ways each approach captures the task uncertainty. △ Less

Submitted 12 December, 2018; v1 submitted 4 July, 2018; originally announced July 2018.

Consistent Generative Query Networks

Authors: Ananya Kumar, S. M. Ali Eslami, Danilo J. Rezende, Marta Garnelo, Fabio Viola, Edward Lockhart, Murray Shanahan

Abstract: Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes these drawbacks by generating a latent representation from an arbitrary set of fram… ▽ More Stochastic video prediction models take in a sequence of image frames, and generate a sequence of consecutive future image frames. These models typically generate future frames in an autoregressive fashion, which is slow and requires the input and output frames to be consecutive. We introduce a model that overcomes these drawbacks by generating a latent representation from an arbitrary set of frames that can then be used to simultaneously and efficiently sample temporally consistent frames at arbitrary time-points. For example, our model can "jump" and directly sample frames at the end of the video, without sampling intermediate frames. Synthetic video evaluations confirm substantial gains in speed and functionality without loss in fidelity. We also apply our framework to a 3D scene reconstruction dataset. Here, our model is conditioned on camera location and can sample consistent sets of images for what an occluded region of a 3D scene might look like, even if there are multiple possibilities for what that region might contain. Reconstructions and videos are available at https://bit.ly/2O4Pc4R. △ Less

Submitted 21 April, 2019; v1 submitted 5 July, 2018; originally announced July 2018.

Encoding Spatial Relations from Natural Language

Authors: Tiago Ramalho, Tomáš Kočiský, Frederic Besse, S. M. Ali Eslami, Gábor Melis, Fabio Viola, Phil Blunsom, Karl Moritz Hermann

Abstract: Natural language processing has made significant inroads into learning the semantics of words through distributional approaches, however representations learnt via these methods fail to capture certain kinds of information implicit in the real world. In particular, spatial relations are encoded in a way that is inconsistent with human spatial reasoning and lacking invariance to viewpoint changes.… ▽ More Natural language processing has made significant inroads into learning the semantics of words through distributional approaches, however representations learnt via these methods fail to capture certain kinds of information implicit in the real world. In particular, spatial relations are encoded in a way that is inconsistent with human spatial reasoning and lacking invariance to viewpoint changes. We present a system capable of capturing the semantics of spatial relations such as behind, left of, etc from natural language. Our key contributions are a novel multi-modal objective based on generating images of scenes from their textual descriptions, and a new dataset on which to train it. We demonstrate that internal representations are robust to meaning preserving transformations of descriptions (paraphrase invariance), while viewpoint invariance is an emergent property of the system. △ Less

Submitted 5 July, 2018; v1 submitted 4 July, 2018; originally announced July 2018.

Neural Processes

Authors: Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, Yee Whye Teh

Abstract: A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexibl… ▽ More A neural network (NN) is a parameterised function that can be tuned via gradient descent to approximate a labelled collection of data with high precision. A Gaussian process (GP), on the other hand, is a probabilistic model that defines a distribution over possible functions, and is updated in light of data via the rules of probabilistic inference. GPs are probabilistic, data-efficient and flexible, however they are also computationally intensive and thus limited in their applicability. We introduce a class of neural latent variable models which we call Neural Processes (NPs), combining the best of both worlds. Like GPs, NPs define distributions over functions, are capable of rapid adaptation to new observations, and can estimate the uncertainty in their predictions. Like NNs, NPs are computationally efficient during training and evaluation but also learn to adapt their priors to data. We demonstrate the performance of NPs on a range of learning tasks, including regression and optimisation, and compare and contrast with related models in the literature. △ Less

Submitted 4 July, 2018; originally announced July 2018.

Suppression of von Kármán vortex streets past porous rectangular cylinders

Authors: Pier Giuseppe Ledda, Lorenzo Siconolfi, Francesco Viola, François Gallaire, Simone Camarri

Abstract: Although the stability properties of the wake past impervious bluff bodies have been widely examined in the literature, similar analyses regarding the flow around and through porous ones are still lacking. In this work, the effect of the porosity and permeability on the wake patterns of porous rectangular cylinders is numerically investigated at low to moderate Reynolds numbers in the framework of… ▽ More Although the stability properties of the wake past impervious bluff bodies have been widely examined in the literature, similar analyses regarding the flow around and through porous ones are still lacking. In this work, the effect of the porosity and permeability on the wake patterns of porous rectangular cylinders is numerically investigated at low to moderate Reynolds numbers in the framework of direct numerical simulation combined with local and global stability analyses. A modified Darcy-Brinkman formulation is employed here so as to describe the flow behavior inside the porous media, where also the convective terms are retained to correctly account for the inertial effects at high values of permeability. Different aspect ratios of the cylinder are considered, varying the thickness-to-height ratios, t/d, from 0.01 (flat plate) to 1.0 (square cylinder). The results show that the permeability of the bodies has a strong effect in modifying the characteristics of the wakes and of the associated flow instabilities, while the porosity weakly affects the resulting flow patterns. In particular, the fluid flows through the porous bodies and, thus, as the permeability is progressively increased, the recirculation regions, initially attached to the rear part of the bodies, at first detach from the body and, eventually, disappear even in the near wakes. Global stability analyses lead to the identification of critical values of the permeability above which any linear instability is prevented. Moreover, a different scaling of the non-dimensional permeability allows to identify a general threshold for all the configurations here studied that ensures the suppression of vortex shedding, at least in the considered parameter space. △ Less

Submitted 30 May, 2018; originally announced May 2018.

Comments: 31 pages and 17 figures

Journal ref: Phys. Rev. Fluids 3, 103901 (2018)

Generative Temporal Models with Spatial Memory for Partially Observed Environments

Authors: Marco Fraccaro, Danilo Jimenez Rezende, Yori Zwols, Alexander Pritzel, S. M. Ali Eslami, Fabio Viola

Abstract: In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic environments, due to the difficulty of training such models in larger, potentially p… ▽ More In model-based reinforcement learning, generative and temporal models of environments can be leveraged to boost agent performance, either by tuning the agent's representations during training or via use as part of an explicit planning mechanism. However, their application in practice has been limited to simplistic environments, due to the difficulty of training such models in larger, potentially partially-observed and 3D environments. In this work we introduce a novel action-conditioned generative model of such challenging environments. The model features a non-parametric spatial memory system in which we store learned, disentangled representations of the environment. Low-dimensional spatial updates are computed using a state-space model that makes use of knowledge on the prior dynamics of the moving agent, and high-dimensional visual observations are modelled with a Variational Auto-Encoder. The result is a scalable architecture capable of performing coherent predictions over hundreds of time steps across a range of partially observed 2D and 3D environments. △ Less

Submitted 19 July, 2018; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: ICML 2018

Learning and Querying Fast Generative Models for Reinforcement Learning

Authors: Lars Buesing, Theophane Weber, Sebastien Racaniere, S. M. Ali Eslami, Danilo Rezende, David P. Reichert, Fabio Viola, Frederic Besse, Karol Gregor, Demis Hassabis, Daan Wierstra

Abstract: A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative models that learn and operate on compact state representations, so-called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish th… ▽ More A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative models that learn and operate on compact state representations, so-called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment from raw pixels. The computational speed-up of state-space models while maintaining high accuracy makes their application in RL feasible: We demonstrate that agents which query these models for decision making outperform strong model-free baselines on the game MSPACMAN, demonstrating the potential of using learned environment models for planning. △ Less

Submitted 8 February, 2018; originally announced February 2018.

The Kinetics Human Action Video Dataset

Authors: Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, Andrew Zisserman

Abstract: We describe the DeepMind Kinetics human action video dataset. The dataset contains 400 human action classes, with at least 400 video clips for each action. Each clip lasts around 10s and is taken from a different YouTube video. The actions are human focussed and cover a broad range of classes including human-object interactions such as playing instruments, as well as human-human interactions such… ▽ More We describe the DeepMind Kinetics human action video dataset. The dataset contains 400 human action classes, with at least 400 video clips for each action. Each clip lasts around 10s and is taken from a different YouTube video. The actions are human focussed and cover a broad range of classes including human-object interactions such as playing instruments, as well as human-human interactions such as shaking hands. We describe the statistics of the dataset, how it was collected, and give some baseline performance figures for neural network architectures trained and tested for human action classification on this dataset. We also carry out a preliminary analysis of whether imbalance in the dataset leads to bias in the classifiers. △ Less

Submitted 19 May, 2017; originally announced May 2017.

Learning to Navigate in Complex Environments

Authors: Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J. Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, Dharshan Kumaran, Raia Hadsell

Abstract: Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents. In this work we formulate the navigation question as a reinforcement learning problem and show that data efficiency and task performance can be dramatically improved by relying on additional auxiliary tasks leveraging multimodal sensory inputs. In particular we consider jointly lea… ▽ More Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents. In this work we formulate the navigation question as a reinforcement learning problem and show that data efficiency and task performance can be dramatically improved by relying on additional auxiliary tasks leveraging multimodal sensory inputs. In particular we consider jointly learning the goal-driven reinforcement learning problem with auxiliary depth prediction and loop closure classification tasks. This approach can learn to navigate from raw sensory input in complicated 3D mazes, approaching human-level performance even under conditions where the goal location changes frequently. We provide detailed analysis of the agent behaviour, its ability to localise, and its network activity dynamics, showing that the agent implicitly learns key navigation abilities. △ Less

Submitted 13 January, 2017; v1 submitted 11 November, 2016; originally announced November 2016.

Comments: 11 pages, 5 appendix pages, 11 figures, 3 tables, under review as a conference paper at ICLR 2017