research-article

Public Access

Adversarial Variational Embedding for Robust Semi-supervised Learning

Authors:

Feng YuanAuthors Info & Claims

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 139 - 147

https://doi.org/10.1145/3292500.3330966

Published: 25 July 2019 Publication History

Abstract

Semi-supervised learning is sought for leveraging the unlabelled data when labelled data is difficult or expensive to acquire. Deep generative models (e.g., Variational Autoencoder (VAE)) and semi-supervised Generative Adversarial Networks (GANs) have recently shown promising performance in semi-supervised classification for the excellent discriminative representing ability. However, the latent code learned by the traditional VAE is not exclusive (repeatable) for a specific input sample, which prevents it from excellent classification performance. In particular, the learned latent representation depends on a non-exclusive component which is stochastically sampled from the prior distribution. Moreover, the semi-supervised GAN models generate data from pre-defined distribution (e.g., Gaussian noises) which is independent of the input data distribution and may obstruct the convergence and is difficult to control the distribution of the generated data. To address the aforementioned issues, we propose a novel Adversarial Variational Embedding (AVAE) framework for robust and effective semi-supervised learning to leverage both the advantage of GAN as a high quality generative model and VAE as a posterior distribution learner. The proposed approach first produces an exclusive latent code by the model which we call VAE++, and meanwhile, provides a meaningful prior distribution for the generator of GAN. The proposed approach is evaluated over four different real-world applications and we show that our method outperforms the state-of-the-art models, which confirms that the combination of VAE++ and GAN can provide significant improvements in semi-supervised lassification.

References

[1]

M Ehsan Abbasnejad, Anthony Dick, and Anton van den Hengel. 2017. Infinite variational autoencoder for semi-supervised learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 781--790.

[2]

Hojjat Adeli, Samanwoy Ghosh-Dastidar, and Nahid Dadmehr. 2007. A wavelet-chaos methodology for analysis of EEGs and EEG subbands to detect seizure and epilepsy. IEEE Transactions on Biomedical Engineering, Vol. 54, 2 (2007), 205--211.

[3]

Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. 2017. CVAE-GAN: fine-grained image generation through asymmetric training. CoRR, abs/1703.10155, Vol. 5 (2017).

[4]

Jiezhang Cao, Yong Guo, Qingyao Wu, Chunhua Shen, and Mingkui Tan. 2018. Adversarial Learning with Local Coordinate Coding. The International Conference of Machine Learning (ICML) (2018).

[5]

Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In SIGIR. ACM, 335--344.

Digital Library

[6]

Kaixuan Chen, Lina Yao, Xianzhi Wang, Dalin Zhang, Tao Gu, Zhiwen Yu, and Zheng Yang. 2018. Interpretable Parallel Recurrent Neural Networks with Convolutional Attentions for Multi-Modality Activity Modeling. International Joint Conference on Neural Networks (IJCNN) (2018).

[7]

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems. 2172--2180.

Digital Library

[8]

Benish Fida, Daniele Bibbo, Ivan Bernabucci, Antonino Proto, Silvia Conforto, and Maurizio Schmid. 2015. Real time event-based segmentation to classify locomotion activities through a single inertial sensor. In MobiHealth. 104--107.

Digital Library

[9]

Kamran Ghasedi Dizaji, Xiaoqian Wang, and Heng Huang. 2018. Semi-supervised generative adversarial network for gene expression inference. In The 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1435--1444.

Digital Library

[10]

Chen Gong, Dacheng Tao, Stephen J Maybank, Wei Liu, Guoliang Kang, and Jie Yang. 2016. Multi-modal curriculum learning for semi-supervised image classification. IEEE Transactions on Image Processing, Vol. 25, 7 (2016), 3249--3260.

Digital Library

[11]

Travis R Goodwin and Sanda M Harabagiu. 2017. Deep Learning from EEG Reports for Inferring Underspecified Information. AMIA Summits on Translational Science Proceedings, Vol. 2017 (2017), 112--121.

[12]

Haodong Guo, Ling Chen, Liangying Peng, and Gencai Chen. 2016. Wearable sensor based multimodal human activity recognition exploiting the diversity of classifier ensemble. In UbiComp. ACM, 1112--1123.

Digital Library

[13]

Amir Harati, Meysam Golmohammadi, Silvia Lopez, Iyad Obeid, and Joseph Picone. 2015. Improved EEG event classification using differential energy. In Signal Processing in Medicine and Biology Symposium (SPMB). IEEE, 1--4.

[14]

Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 355--364.

Digital Library

[15]

Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. 2014. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems (NIPS). 3581--3589.

Digital Library

[16]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[17]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (NIPS). 1097--1105.

Digital Library

[18]

Oscar D Lara, Alfredo J Pérez, Miguel A Labrador, and José D Posada. 2012. Centinela: A human activity recognition system based on acceleration and vital sign data. Pervasive and mobile computing, Vol. 8, 5 (2012), 717--729.

Digital Library

[19]

Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. 2015. Autoencoding beyond pixels using a learned similarity metric. arXiv preprint arXiv:1512.09300 (2015).

[20]

Lars Maaløe, Casper Kaae Sønderby, Søren Kaae Sønderby, and Ole Winther. 2016. Auxiliary deep generative models. arXiv preprint:1602.05473 (2016).

Digital Library

[21]

Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015).

[22]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).

[23]

Takeru Miyato, Shin-ichi Maeda, Shin Ishii, and Masanori Koyama. 2018. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence (2018).

[24]

Siddharth Narayanaswamy, T Brooks Paige, Jan-Willem Van de Meent, Alban Desmaison, Noah Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. 2017. Learning disentangled representations with semi-supervised deep generative models. In NIPS. 5925--5935.

Digital Library

[25]

Iyad Obeid and Joseph Picone. 2016. The temple university hospital eeg data corpus. Frontiers in neuroscience, Vol. 10 (2016), 196.

[26]

Augustus Odena. 2016. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv:1606.01583 (2016).

[27]

Michael J Pazzani and Daniel Billsus. 2007. Content-based recommendation systems. The adaptive web. Springer, 325--341.

Digital Library

[28]

Lingxi Peng, Wenbin Chen, Wubai Zhou, Fufang Li, Jin Yang, and Jiandong Zhang. 2016. An immune-inspired semi-supervised algorithm for breast cancer diagnosis. Computer methods and programs in biomedicine, Vol. 134 (2016), 259--265.

Digital Library

[29]

Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. The International Conference on Learning Representations (ICLR) (2016).

[30]

Steffen Rendle. 2012. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 3, 3 (2012), 57.

Digital Library

[31]

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems (NIPS). 2234--2242.

Digital Library

[32]

Robin Tibor Schirrmeister, Lukas Gemein, Katharina Eggensperger, Frank Hutter, and Tonio Ball. 2017. Deep learning with convolutional neural networks for decoding and visualization of EEG pathology. arXiv preprint:1708.08012 (2017).

[33]

Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. 2016. Ladder variational autoencoders. In Advances in neural information processing systems (NIPS). 3738--3746.

Digital Library

[34]

Jost Tobias Springenberg. 2016. Unsupervised and semi-supervised learning with categorical generative adversarial networks. The International Conference on Learning Representations (ICLR) (2016).

[35]

Jacob Walker, Carl Doersch, Abhinav Gupta, and Martial Hebert. 2016. An uncertain future: Forecasting from static images using variational autoencoders. In European Conference on Computer Vision. Springer, 835--851.

[36]

Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. 2012. Deep learning via semi-supervised embedding. Neural Networks: Tricks of the Trade. Springer, 639--655.

[37]

Weidi Xu, Haoze Sun, Chao Deng, and Ying Tan. 2017. Variational Autoencoder for Semi-Supervised Text Classification. In AAAI. 3358--3364.

Digital Library

[38]

Carl Yang, Lanxiao Bai, Chao Zhang, Quan Yuan, and Jiawei Han. 2017. Bridging collaborative filtering and semi-supervised learning: a neural approach for poi recommendation. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1245--1254.

Digital Library

[39]

Lina Yao, Feiping Nie, Quan Z Sheng, Tao Gu, Xue Li, and Sen Wang. 2016. Learning from less for better: semi-supervised activity recognition via shared structure discovery. In UbiComp. ACM, 13--24.

Digital Library

[40]

Xiang Zhang, Lina Yao, Chaoran Huang, Sen Wang, Mingkui Tan, Guodong Long, and Can Wang. 2018. Multi-modality Sensor Data Classification with Selective Attention. IJCAI (2018).

Digital Library

[41]

Saeedeh Ziyabari, Vinit Shah, Meysam Golmohammadi, Iyad Obeid, and Joseph Picone. 2017. Objective evaluation metrics for automatic classification of EEG events. arXiv preprint arXiv:1712.10107 (2017).

Cited By

Shakya KAlavi APorteous JK PLaddi AJaiswal M(2024)A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image ClassificationInformation10.3390/info1505024615:5(246)Online publication date: 24-Apr-2024
https://doi.org/10.3390/info15050246
Zhang XCui SZhu TChen LZhou FNing H(2024)CASL: Capturing Activity Semantics Through Location Information for Enhanced Activity RecognitionIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.323806421:4(1051-1059)Online publication date: Jul-2024
https://doi.org/10.1109/TCBB.2023.3238064
Tseng YWen C(2023)Hybrid Learning Models for IMU-Based HAR with Feature Analysis and Data CorrectionSensors10.3390/s2318780223:18(7802)Online publication date: 11-Sep-2023
https://doi.org/10.3390/s23187802
Show More Cited By

Index Terms

Adversarial Variational Embedding for Robust Semi-supervised Learning

Recommendations

Adversarial Variational Knowledge Distillation
Artificial Neural Networks and Machine Learning – ICANN 2021
Abstract
Knowledge Distillation (KD) is one of the most popular and effective techniques for model compression and knowledge transfer. However, most existing KD approaches are heavily relying on the labeled training data, which is usually unavailable due ...
VAEPP: Variational Autoencoder with a Pull-Back Prior
Neural Information Processing
Abstract
Many approaches to training generative models by distinct training objectives have been proposed in the past. Variational Autoencoder (VAE) is an outstanding model of them based on log-likelihood. In this paper, we propose a novel learnable prior, ...
Generative Adversarial Network-Based Semi-supervised Learning for Pathological Speech Classification
Statistical Language and Speech Processing
Abstract
A challenge in applying machine learning algorithms to pathological speech classification is the labelled data shortage problem. Labelled data acquisition often requires significant human effort and time-consuming experimental design. Further, for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2019

3305 pages

ISBN:9781450362016

DOI:10.1145/3292500

General Chairs:
Ankur Teredesai
KenSci
,
Vipin Kumar
University of Minnesota
,
Program Chairs:
Ying Li
EV Analysis Corporation
,
Rómer Rosales
LinkedIn
,
Evimaria Terzi
Boston University
,
George Karypis
University of Minnesota

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

ONRG

Conference

KDD '19

Sponsor:

KDD '19: The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 4 - 8, 2019

AK, Anchorage, USA

Acceptance Rates

KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
1,617
Total Downloads

Downloads (Last 12 months)113
Downloads (Last 6 weeks)13

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shakya KAlavi APorteous JK PLaddi AJaiswal M(2024)A Critical Analysis of Deep Semi-Supervised Learning Approaches for Enhanced Medical Image ClassificationInformation10.3390/info1505024615:5(246)Online publication date: 24-Apr-2024
https://doi.org/10.3390/info15050246
Zhang XCui SZhu TChen LZhou FNing H(2024)CASL: Capturing Activity Semantics Through Location Information for Enhanced Activity RecognitionIEEE/ACM Transactions on Computational Biology and Bioinformatics10.1109/TCBB.2023.323806421:4(1051-1059)Online publication date: Jul-2024
https://doi.org/10.1109/TCBB.2023.3238064
Tseng YWen C(2023)Hybrid Learning Models for IMU-Based HAR with Feature Analysis and Data CorrectionSensors10.3390/s2318780223:18(7802)Online publication date: 11-Sep-2023
https://doi.org/10.3390/s23187802
Fiscal LJennebauffe CBruyneel MRis LLefebvre LSiebert XGosselin B(2023)Explainable AI for EEG Biomarkers Identification in Obstructive Sleep Apnea Severity Scoring Task2023 11th International IEEE/EMBS Conference on Neural Engineering (NER)10.1109/NER52421.2023.10123795(1-6)Online publication date: 24-Apr-2023
https://doi.org/10.1109/NER52421.2023.10123795
Zilelioglu HKhodabandelou GChibani AAmirat Y(2023)Semisupervised Generative Adversarial Networks With Temporal Convolutions for Human Activity RecognitionIEEE Sensors Journal10.1109/JSEN.2023.326724323:11(12355-12369)Online publication date: 1-Jun-2023
https://doi.org/10.1109/JSEN.2023.3267243
La Fisca LJennebauffe CBruyneel MRis LLefebvre LSiebert XGosselin B(2023) Enhancing OSA Assessment with Explainable AI * 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)10.1109/EMBC40787.2023.10341035(1-6)Online publication date: 24-Jul-2023
https://doi.org/10.1109/EMBC40787.2023.10341035
Huang FDeng Y(2023)TCGAN: Convolutional Generative Adversarial Network for time series classification and clusteringNeural Networks10.1016/j.neunet.2023.06.033165(868-883)Online publication date: Aug-2023
https://doi.org/10.1016/j.neunet.2023.06.033
Taha K(2023)Semi-supervised and un-supervised clustering: A review and experimental evaluationInformation Systems10.1016/j.is.2023.102178114(102178)Online publication date: Mar-2023
https://doi.org/10.1016/j.is.2023.102178
Fan YTseng YWen C(2022)A Novel Deep Neural Network Method for HAR-Based Team Training Using Body-Worn Inertial SensorsSensors10.3390/s2221850722:21(8507)Online publication date: 4-Nov-2022
https://doi.org/10.3390/s22218507
An SGazi AInan O(2022)DynaLAP: Human Activity Recognition in Fixed Protocols via Semi-Supervised Variational Recurrent Neural Networks With Dynamic PriorsIEEE Sensors Journal10.1109/JSEN.2022.319467722:18(17963-17976)Online publication date: 15-Sep-2022
https://doi.org/10.1109/JSEN.2022.3194677
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents