Article

Free access

Adversarial ranking for language generation

Authors:

Zhengyou Zhang,

Ming-Ting SunAuthors Info & Claims

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

Pages 3158 - 3168

Published: 04 December 2017 Publication History

PDF eReader Publisher Site

Abstract

Generative adversarial networks (GANs) have great successes on synthesizing data. However, the existing GANs restrict the discriminator to be a binary classifier, and thus limit their learning capacity for tasks that need to synthesize output with rich structures such as natural language descriptions. In this paper, we propose a novel generative adversarial network, RankGAN, for generating high-quality language descriptions. Rather than training the discriminator to learn and assign absolute binary predicate for individual data sample, the proposed RankGAN is able to analyze and rank a collection of human-written and machine-written sentences by giving a reference group. By viewing a set of data samples collectively and evaluating their quality through relative ranking scores, the discriminator is able to make better assessment which in turn helps to learn a better generator. The proposed RankGAN is optimized through the policy gradient technique. Experimental results on multiple public datasets clearly demonstrate the effectiveness of the proposed approach.

References

[1]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.

[2]

Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proc. ACL workshops, volume 29, pages 65-72, 2005.

[3]

Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. Proc. CoNLL, page 10, 2016.

[4]

Bo Dai, Dahua Lin, Raquel Urtasun, and Sanja Fidler. Towards diverse and natural image descriptions via a conditional gan. arXiv preprint arXiv:1703.06029, 2017.

[5]

Emily L Denton, Soumith Chintala, Rob Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. In Proc. NIPS, pages 1486-1494, 2015.

[6]

Hao Fang, Saurabh Gupta, Forrest Iandola, Rupesh K Srivastava, Li Deng, Piotr Dollár, Jianfeng Gao, Xiaodong He, Margaret Mitchell, John C Platt, et al. From captions to visual concepts and back. In Proc. CVPR, pages 1473-1482, 2015.

[7]

Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03122, 2017.

[8]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proc. NIPS, pages 2672-2680, 2014.

Digital Library

[9]

Ian J Goodfellow. On distinguishability criteria for estimating generative models. arXiv preprint arXiv:1412.6515, 2014.

[10]

Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013.

[11]

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.

Digital Library

[12]

Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. Learning deep structured semantic models for web search using clickthrough data. In Proc. CIKM, pages 2333-2338, 2013.

Digital Library

[13]

Ferenc Huszár. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? arXiv preprint arXiv:1511.05101, 2015.

[14]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proc. CVPR, 2017.

[15]

Thorsten Joachims. Optimizing search engines using clickthrough data. In Proc. SIGKDD, pages 133-142, 2002.

Digital Library

[16]

Matt J Kusner and José Miguel Hernández-Lobato. Gans for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint arXiv:1611.04051, 2016.

[17]

Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802, 2016.

[18]

Jiwei Li, Will Monroe, Tianlin Shi, Alan Ritter, and Dan Jurafsky. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547, 2017.

[19]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proc. ECCV, pages 740-755, 2014.

[20]

Siqi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, and Kevin Murphy. Improved image captioning via policy gradient optimization of spider.

[21]

Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3):225-331, 2009.

Digital Library

[22]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proc. ACL, pages 311-318, 2002.

[23]

Devi Parikh and Kristen Grauman. Relative attributes. In Proc. ICCV, pages 503-510, 2011.

Digital Library

[24]

Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.

[25]

Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In Proc. NIPS, 2016.

[26]

Kevin Reschke, Adam Vogel, and Dan Jurafsky. Generating recommendation dialogs by extracting information from user reviews. In ACL, 2013.

[27]

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. arXiv preprint arXiv:1606.03498, 2016.

[28]

William Shakespeare. The complete works of William Shakespeare. Race Point Publishing, 2014.

[29]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Proc. NIPS, pages 3104-3112, 2014.

Digital Library

[30]

Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.

Digital Library

[31]

Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et al. Policy gradient methods for reinforcement learning with function approximation. In NIPS, volume 99, pages 1057-1063, 1999.

Digital Library

[32]

Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. In Proc. CVPR, pages 4566-4575, 2015.

[33]

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016.

[34]

Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. Improving neural machine translation with conditional sequence generative adversarial nets. arXiv preprint arXiv:1703.04887, 2017.

[35]

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. Seqgan: sequence generative adversarial nets with policy gradient. In Proc. AAAI, 2017.

[36]

Xiang Zhang and Yann LeCun. Text understanding from scratch. arXiv preprint arXiv:1502.01710, 2015.

[37]

Xingxing Zhang and Mirella Lapata. Chinese poetry generation with recurrent neural networks. In Proc. EMNLP, 2014.

Cited By

Li YKirchmeyer AMehta AQin YDadachev BPapineni KKumar SRisteski ASalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Promises and pitfalls of generative masked language modelingProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693192(27969-28017)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693192
Yang YLi YHuo YGao ZRui L(2024)Alarm Log Data Augmentation Algorithm Based on a GAN Model and AprioriJournal of Computer Science and Technology10.1007/s11390-024-2408-139:4(951-966)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11390-024-2408-1
Zhang CYu STian ZYu J(2023)Generative Adversarial Networks: A Survey on Attack and Defense PerspectiveACM Computing Surveys10.1145/361533656:4(1-35)Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1145/3615336
Show More Cited By

Recommendations

MetaEx-GAN: Meta Exploration to Improve Natural Language Generation via Generative Adversarial Networks
Generative Adversarial Networks (GANs) have been popularly researched in natural language generation, so-called Language GANs. Existing works adopt reinforcement learning (RL) based methods such as policy gradients for training Language GANs. The previous ...
Stylized Adversarial AutoEncoder for Image Generation
MM '17: Proceedings of the 25th ACM international conference on Multimedia

In this paper, we propose an autoencoder-based generative adversarial network (GAN) for automatic image generation, which is called "stylized adversarial autoencoder". Different from existing generative autoencoders which typically impose a prior ...
Unregularized Auto-Encoder with Generative Adversarial Networks for Image Generation
MM '18: Proceedings of the 26th ACM international conference on Multimedia

With the development of deep neural networks, recent years have witnessed the increasing research interest on generative models. Specificly, Variational Auto-Encoders (VAE) and Generative Adversarial Networks (GAN) have achieved impressive results in ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

NIPS'17: Proceedings of the 31st International Conference on Neural Information Processing Systems

December 2017

7104 pages

ISBN:9781510860964

Publisher

Curran Associates Inc.

Red Hook, NY, United States

Publication History

Published: 04 December 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
127
Total Downloads

Downloads (Last 12 months)65
Downloads (Last 6 weeks)10

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YKirchmeyer AMehta AQin YDadachev BPapineni KKumar SRisteski ASalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Promises and pitfalls of generative masked language modelingProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693192(27969-28017)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693192
Yang YLi YHuo YGao ZRui L(2024)Alarm Log Data Augmentation Algorithm Based on a GAN Model and AprioriJournal of Computer Science and Technology10.1007/s11390-024-2408-139:4(951-966)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11390-024-2408-1
Zhang CYu STian ZYu J(2023)Generative Adversarial Networks: A Survey on Attack and Defense PerspectiveACM Computing Surveys10.1145/361533656:4(1-35)Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1145/3615336
Zhang JLiu YMao JMa WXu JMa STian Q(2023)User Behavior Simulation for Search Result Re-rankingACM Transactions on Information Systems10.1145/351146941:1(1-35)Online publication date: 20-Jan-2023
https://dl.acm.org/doi/10.1145/3511469
Pi XZhong WGao YDuan NLou JKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)LogiGANProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601455(16290-16304)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601455
Haque MTozal M(2022)Negative Insurance Claim Generation Using Distance Pooling on Positive Diagnosis-Procedure Bipartite GraphsJournal of Data and Information Quality10.1145/353134714:3(1-26)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1145/3531347
Dervishaj ECremonesi PHong JBures MPark JCerny T(2022)GAN-based matrix factorization for recommender systemsProceedings of the 37th ACM/SIGAPP Symposium on Applied Computing10.1145/3477314.3507099(1373-1381)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3477314.3507099
Sun TWang CSong XFeng FNie L(2022)Response Generation by Jointly Modeling Personalized Linguistic Styles and EmotionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/347587218:2(1-20)Online publication date: 16-Feb-2022
https://dl.acm.org/doi/10.1145/3475872
Gao NXue HShao WZhao SQin KPrabowo ARahaman MSalim F(2022)Generative Adversarial Networks for Spatio-temporal Data: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/347483813:2(1-25)Online publication date: 6-Feb-2022
https://dl.acm.org/doi/10.1145/3474838
Jabbar ALi XOmar B(2021)A Survey on Generative Adversarial Networks: Variants, Applications, and TrainingACM Computing Surveys10.1145/346347554:8(1-49)Online publication date: 4-Oct-2021
https://dl.acm.org/doi/10.1145/3463475
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents