generative adversarial networks (GANs)

Analysis and modeling of crime text report data has important applications, including refinement of crime classifications, clustering of documents, and feature extraction for spatiotemporal forecasts. Having better neural network representations
of crime text data may facilitate all of these tasks. This paper evaluates the ability of generative adversarial network models to represent crime text data and generate realistic crime reports. We compare four state-of-the-art GAN algorithms in terms of quantitative metrics such as coherence, embedding similarity, negative log-likelihood, and qualitatively based on inspection of generated text. We discuss current challenges with crime text
representation and directions for future research

Histopathological evaluation and Gleason grading on Hematoxylin and Eosin
(H&E) stained specimens is the clinical standard in grading prostate cancer. Recently,
deep learning models have been trained to assist pathologists in detecting
prostate cancer. However, these predictions could be improved further regarding
variations in morphology, staining and differences across scanners. An approach to
tackle such problems is to employ conditional GANs for style transfer. A total of
52 prostatectomies from 48 patients were scanned with two different scanners. Data
was split into 40 images for training and 12 images for testing and all images were
divided into overlapping 256x256 patches.
A segmentation model was trained using images from scanner A, and the model
was tested on images from both scanner A and B. Next, GANs were trained to perform
style transfer from scanner A to scanner B. The training was performed using
unpaired training images and different types of Unsupervised Image to Image Translation
GANs (CycleGAN and UNIT). Beside the common CycleGAN architecture,
a modified version was also tested, adding Kullback Leibler (KL) divergence in the
loss function. Then, the segmentation model was tested on the augmented images
from scanner B.
The models were evaluated on 2,000 randomly selected patches of 256x256 pixels
from 10 prostatectomies. The resulting predictions were evaluated both qualitatively
and quantitatively. All proposed methods outperformed in AUC, in the best
case the improvement was of 16%. However, only CycleGAN trained on a large
dataset demonstrated to be capable to improve the segmentation tool performance,
preserving tissue morphology and obtaining higher results in all the evaluation measurements.
All the models were analyzed and, finally, the significance of the difference
between the segmentation model performance on style transferred images and
on untransferred images was assessed, using statistical tests.

In this study, we introduce a novel unsupervised countermeasure for smart grid power systems, based on generative adversarial networks (GANs). Given the pivotal role of smart grid systems (SGSs) in urban life, their security is of particular importance. In recent years, however, advances in the field of machine learning, have raised concerns about cyber attacks on these systems. Power systems, among the most important components of urban infrastructure, have, for example, been widely attacked by adversaries. Attackers disrupt power systems using false data injection attacks (FDIA), resulting in a breach of availability, integrity, or confidential principles of the system. Our model simulates possible attacks on power systems using multiple generators in a step-by-step interaction with a discriminator in the training phase. As a consequence, our system is robust to unseen attacks. Moreover, the proposed model considerably reduces the well-known mode collapse problem of GAN-based models. Our method is general and it can be potentially employed in a wide range of one of one-class classification tasks. The proposed model has low computational complexity and outperforms baseline systems about 14% and 41% in terms of accuracy on the highly imbalanced publicly available industrial control system (ICS) cyber attack power system dataset.

Deep learning's breakthrough in the field of artificial intelligence has resulted in the creation of a slew of deep learning models. One of these is the Generative Adversarial Network, which has only recently emerged. The goal of GAN is to use unsupervised learning to analyse the distribution of data and create more accurate results. The GAN allows the learning of deep representations in the absence of substantial labelled training information. Computer vision, language and video processing, and image synthesis are just a few of the applications that might benefit from these representations. The purpose of this research is to get the reader conversant with the GAN framework as well as to provide the background information on Generative Adversarial Networks, including the structure of both the generator and discriminator, as well as the various GAN variants along with their respective architectures. Applications of GANs are also discussed with examples.

Recently, generative adversarial networks (GANs) have become a research focus of artificial intelligence. Inspired by two-player zero-sum game, GANs comprise a generator and a discriminator, both trained under the adversarial learning idea. The goal of GANs is to estimate the potential distribution of real data samples and generate new samples from that distribution. Since their initiation, GANs have been widely studied due to their enormous prospect for applications, including image and vision computing, speech and language processing, etc. In this review paper, we summarize the state of the art of GANs and look into the future. Firstly, we survey GANs' proposal background, theoretic and implementation models, and application fields. Then, we discuss GANs' advantages and disadvantages, and their development trends. In particular, we investigate the relation between GANs and parallel intelligence, with the conclusion that GANs have a great potential in parallel systems research in terms of virtual-real interaction and integration. Clearly, GANs can provide substantial algorithmic support for parallel intelligence.

Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for both objectives. We present a novel, complete deep learning framework for multi-person localisation and tracking. In this context we first introduce a light weight sequential Generative Adversarial Network architecture for person localisation, which overcomes issues related to occlusions and noisy detections, typically found in a multi person environment. In the proposed tracking framework we build upon recent advances in pedestrian trajectory prediction approaches and propose a novel data association scheme based on predicted trajectories. This removes the need for computationally expensive person re-identification systems based on appearance features and generates human like trajectories with minimal fragmentation. The proposed method is evaluated on multiple public benchmarks including both static and dynamic cameras and is capable of generating outstanding performance, especially among other recently proposed deep neural network based approaches.

Also available for review at https://tinyurl.com/StrangeVisitor.

We recognize myths for what they indicate about the values and beliefs of the cultures who were the authors and audiences of those myths; many of the earliest communicators of myth were telling stories about subjects they counted as people (somewhat) like themselves. Yet, despite our modern sophistication, today’s consumers of narrative still often treat fictions as though they are real, thinking and feeling people.  We are introduced to Superman, for instance, as a fiction, but circumstance often leads consumers to discursively treat that invented hero as a human subject.  Now, though, we verge on a technological future in which just such sympathies might become functionally useful. In our modern science fiction like Star Trek: The Next Generation and Westworld, we have already begun to imagine how we will come to treat our narrative inventions of tomorrow. Strange Visitor: Subjectivity, Simulation, and the Future of the First Superhero begins to ask what our actual responsibilities will become when fictions can be rendered with volition.  In its opening chapters, the text studies how we humanize a figure like Superman in our own perceptions, and how that treatment--along with his continuing narrative development--makes him a prime candidate for an experiment in complex, highly verisimilitudinous simulation.  Superman is a central point of consideration here simply because there is so much of him to consider. He may be the most content-dense fiction ever created, but the innumerable opportunities consumers have had to get to know the character mean that he is also eminently familiar.  Superman has maintained his presence in our media and discourses, across the decades, as consumer populations have continued to grow and our communications structures have commensurately evolved and expanded. Superman is the subject of this study not because he has become so popular, but because he has managed to stay that way at a particularly sharp inflection point in the history of human communications. 

But, the Man of Steel is often generalized as infallible and, therefore, rather dull.  The first part of this work--an expansive psychological assessment--troubles such a presumption, though, by examining the complex mind that has accrued within decades of narrative, across media, and which must be carefully navigated in any exhaustive analysis.  This first section also considers Superman’s relationship with his media consumers: from how we use him to form childhood bonds with one another, to how we respond to his continuing humanization.  For, Superman is powerfully determined by how audiences actually treat any given instance of the figure as Superman.  Today, we find that we can engage with particularly realistic media, encouraging us even more urgently to invest in the verisimilitude and humanity of the characters and places being presented, and doing so often makes a narrative more appealing.  Accordingly, Superman’s relationship with his own media is particularly convenient to an examination of the intersection between fictional narrative, high technology, and identity negotiation.  To understand Superman's potent humanization, the first half of this text presents an assessment of his identity from multiple essential perspectives, including those of his content creators and consumers, as well as the complex judgments of himself and his fictional society as they have been presented in his narratives. Diegetic evidence is considered in historical and psychological capacities in order to assess how Superman’s media has been constructed--practically from the beginning--to encourage consumers to invest themselves in the figure’s humanity and relatability.

In its second part, Strange Visitor moves from relevant questions of identity into a more practical application of those insights to current and emerging technologies.  We already enjoy vastly interactive ways of engaging with our fictions, and we can anticipate continuing advancements in the verisimilitude, interactability, and spontaneity of artificial intelligences.  Accordingly, fewer and fewer holes in the experiential tapestry of narrative will have to be ignored as both storytelling techniques and rendering technologies continue to evolve.  Soon, what we practically treat as a ‘human’ subject may have more to do with that figure’s specific engageability than with its biology or physical consistency.  Given the view of the detailed project proposal that constitutes the second part of this work, spontaneous, humanlike, autonomous simulations culled from fictional worlds could very well become reality within the foreseeable future.  Emerging from our science fiction dreams, several considerable, real-world technological initiatives would see us reconstruct the intellects of great thinkers, such as Gandhi, Curie, or Hawking. But, reconstructing and functionalizing the consciousness of a real person--living or dead--is an ethically dubious proposition from the start.  Instead of an original personality, though--seemingly ‘complete’ at first, but limited in contextual understanding--what if researchers built a simulated subject from the material of a robustly developed fiction?  If the first of our deep, humanlike engagements with such technology will be the most important, Superman seems a fitting choice, given how we have already embraced him on personal and cultural levels.

But, as real technologies advance toward that imagined outcome, we must also ask how the practical, livable terms of fictionality would change. How would our own responsibilities evolve as we produced and engaged with such personalities?  The prototypical solution I propose here--referred to as the Metropolis Simulation System--could be considered one of Andrew M. Butler’s necessary explorations into the role of the simulacrum (“Postmodernism and Science Fiction” 147).  Whatever of these speculations does or does not bear out, it is my hope with this text to show the reader that we must pay substantial attention to our future relationships with our media and computing technologies. Such a situation need not be feared, though, if we approach with the optimistic combination of thoughtfulness and adventurousness that a character like Superman can inspire.

Strange Visitor is intended for any who might have the interest and/or resources to help realize the rendering of a densely-developed fiction as an exemplary representative of humanlike, fully-engageable general artificial intelligence, including the developers of the many technologies referenced.  Portions of the work might also be used as an introduction into the complexities of mediated identity construction, while the critical questions presented could be utilized in advanced media studies courses, or as portions of a forward-thinking philosophy curriculum. Other segments may benefit discussions on popular fictions generally, the ethics of narrative, or psychological and identiary dynamics.

Converting natural language text descriptions into images is a challenging problem in computer vision and has many practical applications. Text-image is not different from language translation problems. In the same way similar semantics can be encoded in two different languages, images and text are two different languages to encode related information. None the less, these problems are totally different because text-image or image-text conversions are highly multimodal problems. In this paper, we propose our model for Arabic text description that allows multi-stage, attention-driven for refinement for fine-grained Arabic text-to-image generation. With a modern attentional generative network, the Attentional model enable to synthesize fine-grained details at different sub-regions of the image by paying attentions to the related words in the natural Arabic language description. We train the model from scratch to Modified-Arabic dataset. The important term in our Network is a word level fine-grained image-text matching loss computed by the Deep Attentional Multimodal Similarity Model (DAMSM). The DAMSM learns two main neural networks that map sub-regions of the image and Arabic words of the sentence to a common semantic space. Our model achieves strong performance on Arabic-text encoder and image encoder, it is characterized by ease and accuracy in description the images on the Caltech-UCSD Birds 200-2011 dataset.

In this paper we address the problem of continuous fine-grained action segmentation, in which multiple actions are present in an unsegmented video stream. The challenge for this task lies in the need to represent the hierarchical nature of the actions and to detect the transitions between actions, allowing us to localise the actions within the video effectively. We propose a novel recurrent semi-supervised Generative Adversarial Network (GAN) model for continuous fine-grained human action segmentation. Temporal context information is captured via a novel Gated Context Extractor (GCE) module, composed of gated attention units, that directs the queued context information through the generator model, for enhanced action segmentation. The GAN is made to learn features in a semi-supervised manner, enabling the model to perform action classification jointly with the standard, unsupervised, GAN learning procedure. We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed network architecture, and show that it is capable of outperforming current state-of-the-art on three challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities dataset.

Hyperrealistic replicas of the human face owe their documentary value to the belief that they result from mechanical reproduction. The idea that a picture is automatically produced through a process of imprint taking is often enough to convince the viewer of its truthfulness and reliability, thus contributing to giving images an aura of authenticity and to creating the myth of pure objectivity. But what happens when the link between hyperrealism, mechanicalness, and truthfulness is disentangled? In 2017, French artist Raphaël Fabre successfully applied for an ID card using a computer-generated picture where the real face was, in fact, an artificial, synthetic mask. Starting from this case study, the essay tackles the issue of the increasing overlapping of actual reality and digital (un)reality, particularly focusing on the concerns raised by the confusion between faces and masks caused by the rapid spread of so-called deepfakes in a world that speeds from documentality towards what I propose to call "mockumentality".

We propose a novel conditional GAN (cGAN) model for continuous fine-grained human action segmentation, that utilises multi-modal data and learned scene context information. The proposed approach utilises two GANs: termed Action GAN and Auxiliary GAN, where the Action GAN is trained to operate over the current RGB frame while the Auxiliary GAN utilises supplementary information such as depth or optical flow. The goal of both GANs is to generate similar 'action codes', a vector representation of the current action. To facilitate this process a context extractor that incorporates data and recent outputs from both modes is used to extract context information to aid recognition. The result is a recurrent GAN architecture which learns a task specific loss function from multiple feature modalities. Extensive evaluations on variants of the proposed model to show the importance of utilising different information streams such as context and auxiliary information in the proposed network; and show that our model is capable of outperforming state-of-the-art methods for three widely used datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities, comprising both static and dynamic camera settings. 1

Inspired by human neurological structures for action anticipation , we present an action anticipation model that enables the prediction of plausible future actions by forecasting both the visual and temporal future. In contrast to current state-of-the-art methods which first learn a model to predict future video features and then perform action anticipation using these features, the proposed framework jointly learns to perform the two tasks, future visual and temporal representation synthesis, and early action anticipation. The joint learning framework ensures that the predicted future embeddings are informative to the action anticipation task. Furthermore, through extensive experimental evaluations we demonstrate the utility of using both visual and temporal semantics of the scene, and illustrate how this representation synthesis could be achieved through a recurrent Gen-erative Adversarial Network (GAN) framework. Our model outperforms the current state-of-the-art methods on multiple datasets: UCF101, UCF101-24, UT-Interaction and TV Human Interaction.

Further advances in exoplanet detection and characterisation require sampling a diverse population of extrasolar planets. One technique to detect these distant worlds is through the direct detection of their thermal emission. The so-called direct imaging technique, is suitable for observing young planets far from their star. These are very low signal-to-noise-ratio (SNR) measurements and limited ground truth hinders the use of supervised learning approaches. In this paper, we combine deep generative and discriminative models to bypass the issues arising when directly training on real data. We use a Generative Adversarial Network to obtain a suitable dataset for training Convolutional Neural Network classifiers to detect and locate planets across a wide range of SNRs. Tested on artificial data, our detectors exhibit good predictive performance and robustness across SNRs. To demonstrate the limits of the detectors, we provide maps of the precision and recall of the model per pixel of ...

Image segmentation is the procedure of dividing
a digital image into a multiple set of pixels. The prior goal of
the segmentation is to make things simpler and transform
the representation of medical images into a meaningful
subject. Many variant modalities, such as CT, X-ray, MRI,
microscopy, positron emission tomography, single photon
emission computer tomography, among others, make
segmentation difficult. The challenging problem is for
segmenting the regions with missing edges, absence of
texture contrast, region of interest (ROI), and background.
Most of the current researches only focuses on single-mode
or paired multimodal images, and there are few researches
on single-mode processing of unpaired multimodal images
(Unified multimodal), the latter is more flexible and has good
generalization ability in processing medical images. Based on
the analysis of the existing medical image problems, this
paper focuses on the unified multimodal segmentation of
medical images by leveraging General Adversarial Network
(GAN). GAN provides a simple and effective paradigm for
image generation, which has been increasingly used in
different fields such as migration learning and data
enhancement. In this research, we have proposed a GAN
based framework for addressing the issues concerning
multi-modal medical image segmentation.

This article addresses the need for adaptive ethical analysis within machine learning that accounts for emerging problems concerning social bias and generative adversarial networks (gan s). I use John Dewey's criticisms of the reflex arc concept in psychology as a basis for understanding how these problems stem from human-gan interaction. By combining Dewey's criticisms with Donna Haraway's idea of cyborgs, Luciano Floridi's concept of distributed morality, and Shaowen Bardzell's recommendations for a feminist approach to human-computer interaction, I suggest a dynamic perspective from which to begin analyzing and solving issues of injustice evident in this particular domain of machine learning.

We propose a novel semi supervised, Multi-Level Sequential Generative Adversarial Network (MLS-GAN) architecture for group activity recognition. In contrast to previous works which utilise manually annotated individual human action predictions, we allow the models to learn it's own internal representations to discover pertinent sub-activities that aid the final group activity recognition task. The generator is fed with person-level and scene-level features that are mapped temporally through LSTM networks. Action-based feature fusion is performed through novel gated fusion units that are able to consider long-term dependancies, exploring the relationships among all individual actions, to learn an intermediate representation or 'action code' for the current group activity. The network achieves it's semi-supervised behaviour by allowing it to perform group action classification together with the adver-sarial real/fake validation. We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed architecture. Furthermore, we show that utilising both person-level and scene-level features facilitates the group activity prediction better than using only person-level features. Our proposed architecture outperforms current state-of-the-art results for sports and pedestrian based classification tasks on Volleyball and Collective Activity datasets, showing it's flexible nature for effective learning of group activities.

The rapid development of the Internet of Things (IoT) has prompted a recent interest into realistic IoT network traffic generation. Security practitioners need IoT network traffic data to develop and assess network-based intrusion detection systems (NIDS). Emulating realistic network traffic will avoid the costly physical deployment of thousands of smart devices. From an attacker's perspective, generating network traffic that mimics the legitimate behavior of a device can be useful to evade NIDS. As network traffic data consist of sequences of packets, the problem is similar to the generation of sequences of categorical data, like word by word text generation. Many solutions in the field of natural language processing have been proposed to adapt a Generative Adversarial Network (GAN) to generate sequences of categorical data. In this paper, we propose to combine an autoencoder with a GAN to generate sequences of packet sizes that correspond to bidirectional flows. First, the autoencoder is trained to learn a latent representation of the real sequences of packet sizes. A GAN is then trained on the latent space, to learn to generate latent vectors that can be decoded into realistic sequences. For experimental purposes, bidirectional flows produced by a Google Home Mini are used, and the autoencoder is combined with a Wassertein GAN. Comparison of different network characteristics shows that our proposed approach is able to generate sequences of packet sizes that behave closely to real bidirectional flows. We also show that the synthetic bidirectional flows are close enough to the real ones that they can fool anomaly detectors into labeling them as legitimate.

 Abstract: The extraordinary research in the field of unsupervised machine learning has made the non-technical media to expect to see Robot Lords overthrowing humans in near future. Whatever might be the media exaggeration, but the results of recent advances in the research of Deep Learning applications are so beautiful that it has become very difficult to differentiate between the man-made content and computer-made content. This paper tries to establish a ground for new researchers with different real-time applications of Deep Learning. This paper is not a complete study of all applications of Deep Learning, rather it focuses on some of the highly researched themes and popular applications in domains such Image Processing, Sound/Speech Processing, and Video Processing.

This paper investigates conditional generative adversarial networks (cGANs) to overcome a fundamental limitation of using geotagged media for geographic discovery, namely its sparse and uneven spatial distribution. We train a cGAN to generate ground-level views of a location given overhead imagery. We show the "fake" ground-level images are natural looking and are structurally similar to the real images. More significantly, we show the generated images are representative of the locations and that the representations learned by the cGANs are informative. In particular, we show that dense feature maps generated using our framework are more effective for land-cover classification than approaches which spatially interpolate features extracted from sparse ground-level images. To our knowledge, ours is the first work to use cGANs to generate ground-level views given overhead imagery and to explore the benefits of the learned representations.

Las Generative Adversarial Network (GAN) son una arquitectura de redes neuronales de reciente desarrollo capaces de generar nuevas realidades a partir de un conjunto de datos de entrenamiento. Con el objeto de explorar el potencial de las redes generativas convolucionales en disciplinas que se expresan gráficamente, como el urbanismo, se desarrolla una herramienta de software denominada eMapGAN. Esta herramienta genera a partir de una imagen satélite del terreno, un mapa convencional, un mapa más sostenible e incluso una ortofoto de una realidad no existente, pero más sostenible que la original. Incluye también la posibilidad de editar mapas intermedios generando así nuevas realidades, todo ello mediante una interfaz gráfica de usuario. Se concibe como una herramienta experimental, percibida como útil por los usuarios, que explora las posibilidades de las redes GAN para inferir conocimientos complejos y constituir una herramienta de apoyo en las fases de concepción del diseño de las ciudades.

With the exponentially growing COVID-19 (coronavirus disease 2019) pandemic, clinicians continue to seek accurate and rapid diagnosis methods in addition to virus and antibody testing modalities. Because radiographs such as X-rays and computed tomography (CT) scans are cost-effective and widely available at public health facilities, hospital emergency rooms (ERs), and even at rural clinics, they could be used for rapid detection of possible COVID-19-induced lung infections. Therefore, toward automating the COVID-19 detection, in this paper, we propose a viable and efficient deep learning-based chest radiograph classification (DL-CRC) framework to distinguish the COVID-19 cases with high accuracy from other abnormal (e.g., pneumonia) and normal cases. A unique dataset is prepared from four publicly available sources containing the posteroanterior (PA) chest view of X-ray data for COVID-19, pneumonia, and normal cases. Our proposed DL-CRC framework leverages a data augmentation of radiograph images (DARI) algorithm for the COVID-19 data by adaptively employing the generative adversarial network (GAN) and generic data augmentation methods to generate synthetic COVID-19 infected chest X-ray images to train a robust model. The training data consisting of actual and synthetic chest X-ray images are fed into our customized convolutional neural network (CNN) model in DL-CRC, which achieves COVID-19 detection accuracy of 93.94% compared to 54.55% for the scenario without data augmentation (i.e., when only a few actual COVID-19 chest X-ray image samples are available in the original dataset). Furthermore, we justify our customized CNN model by extensively comparing it with widely adopted CNN architectures in the literature, namely ResNet, Inception-ResNet v2, and DenseNet that represent depth-based, multi-path-based, and hybrid CNN paradigms. The encouragingly high classification accuracy of our proposal implies that it can efficiently automate COVID-19 detection from radiograph images to provide a fast and reliable evidence of COVID-19 infection in the lung that can complement existing COVID-19 diagnostics modalities. INDEX TERMS COVID-19, convolutional neural network (CNN), deep learning, generative adversarial network (GAN), pneumonia.

The rapid growth of the worldwide web and accompanied opportunities of web applications in various aspects of life have attracted the attention of organizations, governments, and individuals. Consequently, web applications have increasingly become the target of cyberattacks. Notably, cross-site scripting (XSS) attacks on web applications are increasing and have become the critical focus of information security experts' reports. Machine learning (ML) technique has significantly advanced and shown impressive results in the area of cybersecurity. However, XSS training datasets are often limited and significantly unbalanced, which does not meet well-developed ML algorithms' requirements and potentially limits the detection system efficiency. Furthermore, XSS attacks have multiple payload vectors that execute in different ways, resulting in many real threats passing through the detection system undetected. In this study, we propose a conditional Wasserstein generative adversarial network with a gradient penalty to enhance the XSS detection system in a low-resource data environment. The proposed method integrates a conditional generative adversarial network and Wasserstein generative adversarial network with a gradient penalty to obtain necessary data from directivity, which improves the strength of the security system over unbalance data. The proposed method generates synthetic samples of minority class that have identical distribution as real XSS attack scenarios. The augmented data were used to train a new boosting model and subsequently evaluated the model using a real test dataset. Experiments on two unbalanced XSS attack datasets demonstrate that the proposed model generates valid and reliable samples. Furthermore, the samples were indistinguishable from real XSS data and significantly enhanced the detection of XSS attacks compared with state-of-the-art methods.

Visual saliency patterns are the result of a variety of factors aside from the image being parsed, however existing approaches have ignored these. To address this limitation, we propose a novel saliency estimation model which leverages the semantic modelling power of conditional generative adversarial networks together with memory architectures which capture the subject's behavioural patterns and task dependent factors. We make contributions aiming to bridge the gap between bottom-up feature learning capabilities in modern deep learning architectures and traditional top-down hand-crafted features based methods for task specific saliency modelling. The conditional nature of the proposed framework enables us to learn contextual semantics and relationships among different tasks together, instead of learning them separately for each task. Our studies not only shed light on a novel application area for generative adversarial networks, but also emphasise the importance of task specific saliency modelling and demonstrate the plausibility of fully capturing this context via an augmented memory architecture.

In a world where seeing is increasingly no longer believing, experts are warning that society must take a multi-pronged approach to combat the potential harms of computer-generated media. As Bill Whitaker reports this week on 60 Minutes, artificial intelligence can manipulate faces and voices to make it look like someone said something they never said. The result is videos of things that never happened, called "deepfakes." Often, they look so real, people watching can't tell. Just this month, Justin Bieber was tricked by a series of deepfake videos on the social media video platform TikTok that appeared to be of Tom Cruise.

In this study, we introduce a novel unsupervised countermeasure for smart grid power systems, based on generative adversarial networks (GANs). Given the pivotal role of smart grid systems (SGSs) in urban life, their security is of particular importance. In recent years, however, advances in the field of machine learning, have raised concerns about cyber attacks on these systems. Power systems, among the most important components of urban infrastructure, have, for example, been widely attacked by adversaries. Attackers disrupt power systems using false data injection attacks (FDIA), resulting in a breach of availability, integrity, or confidential principles of the system. Our model simulates possible attacks on power systems using multiple generators in a step-by-step interaction with a discriminator in the training phase. As a consequence, our system is robust to unseen attacks. Moreover, the proposed model considerably reduces the well-known mode collapse problem of GAN-based mode...

The rapid development of the Internet of Things (IoT) has prompted a recent interest into realistic IoT network traffic generation. Security practitioners need IoT network traffic data to develop and assess network-based intrusion detection systems (NIDS). Emulating realistic network traffic will avoid the costly physical deployment of thousands of smart devices. From an attacker's perspective, generating network traffic that mimics the legitimate behavior of a device can be useful to evade NIDS. As network traffic data consist of sequences of packets, the problem is similar to the generation of sequences of categorical data, like word by word text generation. Many solutions in the field of natural language processing have been proposed to adapt a Generative Adversarial Network (GAN) to generate sequences of categorical data. In this paper, we propose to combine an autoencoder with a GAN to generate sequences of packet sizes that correspond to bidirectional flows. First, the aut...

The primary objective of this paper is to provide a comparative analysis of various generative adversarial networks (GANs). In this paper we present our study of-SinGAN, Conditional Generative Adversarial Networks (CGAN), Star generative adversarial network (StarGAN) and Cycle generative adversarial network (CycleGAN). We also present our results generated from these GANs and provide their comparisons on different metrics such as RMSE, UQI, MS-SSIM, VIF and also comparing the images generated by the FCN-8s architecture in this paper. The main motive behind writing this paper is to provide a one place study for the different variants of GAN which is currently lacking in the literature available. This work will help grasping the concepts of the different GAN architectures and their loss functions which are explained in detail.

In this study, we introduce a novel unsupervised countermeasure for smart grid power systems, based on generative adversarial networks (GANs). Given the pivotal role of smart grid systems (SGSs) in urban life, their security is of particular importance. In recent years, however, advances in the field of machine learning, have raised concerns about cyber attacks on these systems. Power systems, among the most important components of urban infrastructure, have, for example, been widely attacked by adversaries. Attackers disrupt power systems using false data injection attacks (FDIA), resulting in a breach of availability, integrity, or confidential principles of the system. Our model simulates possible attacks on power systems using multiple generators in a step-by-step interaction with a discriminator in the training phase. As a consequence, our system is robust to unseen attacks. Moreover, the proposed model considerably reduces the well-known mode collapse problem of GAN-based mode...

In this study, we introduce a novel unsupervised countermeasure for smart grid power systems, based on generative adversarial networks (GANs). Given the pivotal role of smart grid systems (SGSs) in urban life, their security is of particular importance. In recent years, however, advances in the field of machine learning, have raised concerns about cyber attacks on these systems. Power systems, among the most important components of urban infrastructure, have, for example, been widely attacked by adversaries. Attackers disrupt power systems using false data injection attacks (FDIA), resulting in a breach of availability, integrity, or confidential principles of the system. Our model simulates possible attacks on power systems using multiple generators in a step-by-step interaction with a discriminator in the training phase. As a consequence, our system is robust to unseen attacks. Moreover, the proposed model considerably reduces the well-known mode collapse problem of GAN-based mode...

Every human has a unique face that may vary with a wide variety of details from person to person. But, in twins, these differences are very less to be noticed but much significance. Hence, this raises a problem in creating digital faces of humans with all distinguishable features to that of a real human. Also, identify the faces that are not completely visible in a picture due to the distorted image or wrong angle of the face. To resolve this problem, in this research, we are building a model that can generate human after training of the models. For these face generation, we have used deep convoluted generative adversarial networks (DCGANs). This model is trained using the images available in the form OFA dataset that is being provided to the model. The images in the dataset are passed through a series of modifications called preprocessing. Next, our model is divided into three segments called generator, discriminator, and loss function optimization. These segments are trained individually and together form our complete model. The generator, generating the image of the human face and later, these images are provided to the discriminator. The discriminator that is also trained using the dataset starts discriminating against the image generated by the generator. The results are of generator, and discriminator is improvised using improvement analysis called loss function. Finally, and based on this result analysis, the generator is going to improve in its next epoch using the dataset image. As the loss values are noted for generator and discriminator, the values go as 1.1314, 0.8530, 0.8263, 0.8010 for the generator and 1.0013, 1.1288, 1.3122, 1.3111 for the discriminator. This shows a good loss reduction in generated images by a generator which means the enhancement in features of faces. The loss values increase in discriminator as it fails to discriminate the image. In the final stage of output, the generated face image is so real to be discriminated between fake and realistic face image. This model provides a solution to many existing problems of face recognition. This model can be used to identify people even if their face is partially visible on the screen by generating a clear image by a few modifications.

Inspired by human neurological structures for action anticipation, we present an action anticipation model that enables the prediction of plausible future actions by forecasting both the visual and temporal future. In contrast to current state-of-the-art methods which first learn a model to predict future video features and then perform action anticipation using these features, the proposed framework jointly learns to perform the two tasks, future visual and temporal representation synthesis, and early action anticipation. The joint learning framework ensures that the predicted future embeddings are informative to the action anticipation task. Furthermore, through extensive experimental evaluations we demonstrate the utility of using both visual and temporal semantics of the scene, and illustrate how this representation synthesis could be achieved through a recurrent Generative Adversarial Network (GAN) framework. Our model outperforms the current state-of-the-art methods on multipl...

The recent generative model-driven Generalized Zero-shot Learning (GZSL) techniques overcome the prevailing issue of the model bias towards the seen classes by synthesizing the visual samples of the unseen classes through leveraging the corresponding semantic prototypes. Although such approaches significantly improve the GZSL performance due to data augmentation, they violate the principal assumption of GZSL regarding the unavailability of semantic information of unseen classes during training. In this work, we propose to use a generative model (GAN) for synthesizing the visual proxy samples while strictly adhering to the standard assumptions of the GZSL. The aforementioned proxy samples are generated by exploring the early training regime of the GAN. We hypothesize that such proxy samples can effectively be used to characterize the average entropy of the label distribution of the samples from the unseen classes. Further, we train a classifier on the visual samples from the seen classes and proxy samples using entropy separation criterion such that an average entropy of the label distribution is low and high, respectively, for the visual samples from the seen classes and the proxy samples. Such entropy separation criterion generalizes well during testing where the samples from the unseen classes exhibit higher entropy than the entropy of the samples from the seen classes. Subsequently, low and high entropy samples are classified using supervised learning and ZSL rather than GZSL. We show the superiority of the proposed method by experimenting on AWA1, CUB, HMDB51, and UCF101 datasets.

Image segmentation is the procedure of dividing a digital image into a multiple set of pixels. The prior goal of the segmentation is to make things simpler and transform the representation of medical images into a meaningful subject. Many variant modalities, such as CT, X-ray, MRI, microscopy, positron emission tomography, single photon emission computer tomography, among others, make segmentation difficult. The challenging problem is for segmenting the regions with missing edges, absence of texture contrast, region of interest (ROI), and background. Most of the current researches only focuses on single-mode or paired multimodal images, and there are few researches on single-mode processing of unpaired multimodal images (Unified multimodal), the latter is more flexible and has good generalization ability in processing medical images. Based on the analysis of the existing medical image problems, this paper focuses on the unified multimodal segmentation of medical images by leveraging...

Every human has a unique face that may vary with a wide variety of details from person to person. But, in twins, these differences are very less to be noticed but much significance. Hence, this raises a problem in creating digital faces of humans with all distinguishable features to that of a real human. Also, identify the faces that are not completely visible in a picture due to the distorted image or wrong angle of the face. To resolve this problem, in this research, we are building a model that can generate human after training of the models. For these face generation, we have used deep convoluted generative adversarial networks (DCGANs). This model is trained using the images available in the form OFA dataset that is being provided to the model. The images in the dataset are passed through a series of modifications called preprocessing. Next, our model is divided into three segments called generator, discriminator, and loss function optimization. These segments are trained individually and together form our complete model. The generator, generating the image of the human face and later, these images are provided to the discriminator. The discriminator that is also trained using the dataset starts discriminating against the image generated by the generator. The results are of generator, and discriminator is improvised using improvement analysis called loss function. Finally, and based on this result analysis, the generator is going to improve in its next epoch using the dataset image. As the loss values are noted for generator and discriminator, the values go as 1.1314, 0.8530, 0.8263, 0.8010 for the generator and 1.0013, 1.1288, 1.3122, 1.3111 for the discriminator. This shows a good loss reduction in generated images by a generator which means the enhancement in features of faces. The loss values increase in discriminator as it fails to discriminate the image. In the final stage of output, the generated face image is so real to be discriminated between fake and realistic face image. This model provides a solution to many existing problems of face recognition. This model can be used to identify people even if their face is partially visible on the screen by generating a clear image by a few modifications.

Further advances in exoplanet detection and characterisa-tion require sampling a diverse population of extrasolar planets. One technique to detect these distant worlds is through the direct detection of their thermal emission. The so-called direct imaging technique, is suitable for observing young planets far from their star. These are very low signal-to-noise-ratio (SNR) measurements and limited ground truth hinders the use of supervised learning approaches. In this paper, we combine deep generative and discriminative models to bypass the issues arising when directly training on real data. We use a Generative Adversarial Network to obtain a suitable dataset for training Convolutional Neural Network classifiers to detect and locate planets across a wide range of SNRs. Tested on artificial data, our detectors exhibit good predictive performance and robustness across SNRs. To demonstrate the limits of the detectors, we provide maps of the precision and recall of the model per pixel of...

Radiation budget directly affect on the local and global current atmospheric state and the climate projections through thermal exchange between the atmospheric layers. In this work, we present the capability of Generative Adversarial Networks (GANs) algorithm to model spatial distribution for the upward radiative fluxes at Earth's surface over South America. Input for the GANs algorithm is the spatial distribution for total downward radiative fluxes (long and short wave) at the surface while the output is total upward radiative fluxes. We used ERA-5 monthly dataset for period (2000-2020). The model proof its capability to reproduce the spatial pattern of upward raditative fluxes within max. margin of root mean square error ±10%.

In this study, we introduce a newly developed method called Deep-Performance, to enable automatic environmental performance simulation prediction without the need to perform simulations, by integrating deep learning strategies. The aim is to train neural networks on datasets with thousands of building design samples and their corresponding performance simulation. The trained model would offer performance prediction for design options emerging in generative protocols. The research is a work-in-progress within a broader project aimed at automating buildings' environmental performance evaluations of daylight analysis and energy simulation, using deep learning (DL) models. This paper focuses on the implementation of a supervised DL method for automating the retrieval of daylight analysis metrics, targeting successful daylight design and higher building enclosure efficiency. We have further improved a Pix2Pix model trained on 5 different datasets, each containing 6000 paired images of architectural floor plans and their daylight simulation metrics. In the inference phase, the model was able to accurately predict the daylight simulation for unseen sets of floor plans. For validation, two quantitative assessment metrics were followed to assess the predicted daylight performance against the daylight performance simulation. Both assessment metrics showed high accuracy levels.