Academia.edu no longer supports Internet Explorer.
To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser.
2017
Convolutional autoregressive models have recently demonstrated state-of-the-art performance on a number of generation tasks. While fast, parallel training methods have been crucial for their success, generation is typically implemented in a naive fashion where redundant computations are unnecessarily repeated. This results in slow generation, making such models infeasible for production environments. In this work, we describe a method to speed up generation in convolutional autoregressive models. The key idea is to cache hidden states to avoid redundant computation. We apply our fast generation method to the Wavenet and PixelCNN++ models and achieve up to $21\times$ and $183\times$ speedups respectively.
2021 •
In standard generative deep learning models, such as autoencoders or GANs, the size of the parameter set is proportional to the complexity of the generated data distribution. A significant challenge is to deploy resource-hungry deep learning models in devices with limited memory to prevent system upgrade costs. To combat this, we propose a novel framework called generative optimization networks (GON) that is similar to GANs, but does not use a generator, significantly reducing its memory footprint. GONs use a single discriminator network and run optimization in the input space to generate new data samples, achieving an effective compromise between training time and memory consumption. GONs are most suited for data generation problems in limited memory settings. Here we illustrate their use for the problem of anomaly detection in memory-constrained edge devices arising from attacks or intrusion events. Specifically, we use a GON to calculate a reconstructionbased anomaly score for in...
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
RepVGG: Making VGG-style ConvNets Great Again2021 •
2021 •
Deep Learning has revolutionized the fields of computer vision, natural language understanding, speech recognition, information retrieval and more. However, with the progressive improvements in deep learning models, their number of parameters, latency, resources required to train, etc. have all have increased significantly. Consequently, it has become important to pay attention to these footprint metrics of a model as well, not just its quality. We present and motivate the problem of efficiency in deep learning, followed by a thorough survey of the five core areas of model efficiency (spanning modeling techniques, infrastructure, and hardware) and the seminal work there. We also present an experiment-based guide along with code, for practitioners to optimize their model training and deployment. We believe this is the first comprehensive survey in the efficient deep learning space that covers the landscape of model efficiency from modeling techniques to hardware support. Our hope is ...
ArXiv
TGANv2: Efficient Training of Large Models for Video Generation with Multiple Subsampling Layers2018 •
In this paper, we propose a novel method to efficiently train a Generative Adversarial Network (GAN) on high dimensional samples. The key idea is to introduce a differentiable subsampling layer which appropriately reduces the dimensionality of intermediate feature maps in the generator during training. In general, generators require large memory and computational costs in the latter stages of the network as the feature maps become larger, though the latter stages have relatively fewer parameters than the earlier stages. It makes training large models for video generation difficult due to the limited computational resource. We solve this problem by introducing a method that gradually reduces the dimensionality of feature maps in the generator with multiple subsampling layers. We also propose a network (Temporal GAN v2) with such layers and perform video generation experiments. As a consequence, our model trained on the UCF101 dataset at $192 \times 192$ pixels achieves an Inception S...
2018 •
Autoregressive generative models consistently achieve the best results in density estimation tasks involving high dimensional data, such as images or audio. They pose density estimation as a sequence modeling task, where a recurrent neural network (RNN) models the conditional distribution over the next element conditioned on all previous elements. In this paradigm, the bottleneck is the extent to which the RNN can model long-range dependencies, and the most successful approaches rely on causal convolutions, which offer better access to earlier parts of the sequence than conventional RNNs. Taking inspiration from recent work in meta reinforcement learning, where dealing with long-range dependencies is also essential, we introduce a new generative model architecture that combines causal convolutions with self attention. In this note, we describe the resulting model and present state-of-the-art log-likelihood results on CIFAR-10 (2.85 bits per dim) and $32 \times 32$ ImageNet (3.80 bit...
2018 International Conference on Field-Programmable Technology (FPT)
Memory-Efficient Architecture for Accelerating Generative Networks on FPGA2018 •
Generative adversarial networks (GANs) are a class of artificial intelligence algorithms used in unsupervised machine learning, implemented by a system of two neural networks: a generative network (generator) and a discriminative network (discriminator). These two networks compete with each other to perform better at their respective tasks. The generator is typically a deconvolutional neural network and the discriminator is a convolutional neural network (CNN). Deconvolution performs a fundamentally new type of mathematical operation which differs from convolution. While the FPGA-based CNN accelerators have been widely studied in prior work, the acceleration of deconvolutional networks on FPGA is rarely explored. This paper proposes a novel parametrized deconvolutional architecture based on an FPGA-friendly method, in contrast to the transposed convolution implementation in CPUs and GPUs. Hardware design templates which map this architecture to FPGAs are provided with configurable d...
Journal of Control, Automation and Electrical Systems
Generating Stochastic Processes Through Convolutional Neural Networks2020 •
The present work establishes the use of convolutional neural networks as a generative model for stochastic processes that are widely present in industrial automation and system modelling such as fault detection, computer vision and sensor data analysis. This enables researchers from a broad range of fields—as in medical imaging, robotics and control engineering—to develop a general tool for artificial data generation and simulation without the need to identify or assume a specific system structure or estimate its parameters. We demonstrate the approach as a generative model on top of data retrieved from a wide set of classic, simplest to the most complex, deterministic and stochastic data generation processes of technological importance—from damped oscillators to autoregressive conditional heteroskedastic and jump-diffusion models. Also, anonparametric estimation and forecast was carried out for the traditional benchmark “Fisher River” time-series dataset, yielding the superior mean absolute prediction error results compared to a standard ARIMA model. This approach can have potential applications as an alternative to simulation tools such as Gibbs sampling and Monte Carlo-based methods, in the enhancement of the understanding of generative adversarial networks (GANs) and in data simulation for training Reinforcement Learning algorithms.
arXiv (Cornell University)
CacheNet: A Model Caching Framework for Deep Learning Inference on the Edge2020 •
Advances in Artificial Intelligence
Memory-Efficient Backpropagation for Recurrent Neural Networks2019 •
2024 •
Rencontre Assyriologique Internationale
Society on the Edge: A Social Network Analysis of the Central Babylonian Fortress of Dūr-Abiešuḫ in the Late Old Babylonian Period (1711–1595 BCE).2024 •
Studi in onore di Sergio Gensini, a cura di F. Ciappi
L'organizzazione del territorio nella Toscana comunale (XIII-XV secolo)2013 •
2021 •
Communication and Critical/Cultural Studies
Negotiating Rhetorics of Diversity Through Performances of Propriety: A Quare Autocritography2024 •
Research on humanities and social sciences
Implications of Retirement on Health and Longevity2016 •
The Open Neurology Journal
Validation Study of the Mini-Mental State Examination in Urdu Language for Pakistani Population2015 •
2015 •
Zenodo (CERN European Organization for Nuclear Research)
La transformation digitale de la fonction contrôle de gestion : une garantie de sa performance ?2023 •
1999 •
Dental Research Journal
Isolation of Enterococcus faecalis in the saliva samples of patient candidates for liver transplantation2019 •
Erich Mendelsohn. Nuove riflessioni/New reflections, a cura di David Palterer (Tre Lune Edizioni, Mantova)
Mendelsohn e il programma dell'architettura. Berlino 1919-2004 •
JURNAL ILMU DAN TEKNOLOGI PERIKANAN TANGKAP
Pengaruh jenis umpan terhadap hasil tangkapan pancing dasar sekitar perairan Desa Bajo Kabupaten Minahasa Selatan Provinsi Sulawesi Utara2014 •