Data-efficient instance generation from instance discrimination
Article No.: 718, Pages 9378 - 9390
Abstract
Generative Adversarial Networks (GANs) have significantly advanced image synthesis, however, the synthesis quality drops significantly given a limited amount of training data. To improve the data efficiency of GAN training, prior work typically employs data augmentation to mitigate the overfitting of the discriminator yet still learn the discriminator with a bi-classification (i.e., real vs. fake) task. In this work, we propose a data-efficient Instance Generation (InsGen) method based on instance discrimination. Concretely, besides differentiating the real domain from the fake domain, the discriminator is required to distinguish every individual image, no matter it comes from the training set or from the generator. In this way, the discriminator can benefit from the infinite synthesized samples for training, alleviating the overfitting problem caused by insufficient training data. A noise perturbation strategy is further introduced to improve its discriminative power. Meanwhile, the learned instance discrimination capability from the discriminator is in turn exploited to encourage the generator for diverse generation. Extensive experiments demonstrate the effectiveness of our method on a variety of datasets and training settings. Noticeably, on the setting of 2K training images from the FFHQ dataset, we outperform the state-of-the-art approach with 23.5% FID improvement.
Supplementary Material
Supplemental material.
- Download
- 1.79 MB
References
[1]
M. Arjovsky and L. Bottou. Towards principled methods for training generative adversarial networks. Int. Conf. Learn. Represent., 2017. 1
[2]
M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In Int. Conf. Mach. Learn., 2017. 4
[3]
P. Bachman, R. D. Hjelm, and W. Buchwalter. Learning representations by maximizing mutual information across views. In Adv. Neural Inform. Process. Syst., 2019. 2, 4
[4]
A. Brock, J. Donahue, and K. Simonyan. Large scale gan training for high fidelity natural image synthesis. In Int. Conf. Learn. Represent., 2018. 1, 4
[5]
M. Chen, A. Radford, R. Child, J. Wu, H. Jun, P. Dhariwal, D. Luan, and I. Sutskever. Generative pretraining from pixels. In Int. Conf. Mach. Learn., 2020. 2
[6]
T. Chen, X. Zhai, M. Ritter, M. Lucic, and N. Houlsby. Self-supervised gans via auxiliary rotation loss. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. 2, 6, 7
[7]
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In Int. Conf. Mach. Learn., 2020. 2, 4
[8]
X. Chen, H. Fan, R. Girshick, and K. He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020. 5, 9
[9]
Y. Choi, Y. Uh, J. Yoo, and J.-W. Ha. Stargan v2: Diverse image synthesis for multiple domains. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. 6, 7, 8
[10]
E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. Autoaugment: Learning augmentation policies from data. In IEEE Conf. Comput. Vis. Pattern Recog., 2018. 1, 2
[11]
E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. Randaugment: Practical automated data augmentation with a reduced search space. In IEEE Conf. Comput. Vis. Pattern Recog. Worksh., 2020. 1, 2
[12]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conf. Comput. Vis. Pattern Recog., 2009. 3
[13]
C. Doersch, A. Gupta, and A. A. Efros. Unsupervised visual representation learning by context prediction. In Int. Conf. Comput. Vis., 2015. 2
[14]
J. Donahue and K. Simonyan. Large scale adversarial representation learning. In Adv. Neural Inform. Process. Syst., 2019. 2
[15]
S. Gidaris, P. Singh, and N. Komodakis. Unsupervised representation learning by predicting image rotations. In Int. Conf. Learn. Represent., 2018. 2
[16]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. In Adv. Neural Inform. Process. Syst., 2014. 1, 3
[17]
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. 2, 3, 4
[18]
O. Henaff. Data-efficient image recognition with contrastive predictive coding. In Int. Conf. Mach. Learn., 2020. 2, 4
[19]
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., 2017. 6
[20]
J. Jeong and J. Shin. Training gans with stronger augmentations via contrastive discriminator. In Int. Conf. Learn. Represent., 2021. 2, 7, 8
[21]
M. Kang and J. Park. Contragan: Contrastive learning for conditional image generation. In Adv. Neural Inform. Process. Syst., 2020. 2
[22]
T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In Int. Conf. Learn. Represent., 2018. 1, 4
[23]
T. Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2019. 1, 2, 4, 6, 7, 8
[24]
T. Karras, M. Aittala, J. Hellsten, S. Laine, J. Lehtinen, and T. Aila. Training generative adversarial networks with limited data. In Adv. Neural Inform. Process. Syst., 2020. 1, 2, 3, 5, 6, 7, 8, 9, 10
[25]
T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of StyleGAN. In IEEE Conf. Comput. Vis. Pattern Recog., 2020. 1, 4, 5, 7
[26]
V. Kazemi and J. Sullivan. One millisecond face alignment with an ensemble of regression trees. In IEEE Conf. Comput. Vis. Pattern Recog., 2014. 6
[27]
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan. Supervised contrastive learning. In Adv. Neural Inform. Process. Syst., 2020. 2
[28]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Int. Conf. Learn. Represent., 2014. 6
[29]
R. Liu, Y. Ge, C. L. Choi, X. Wang, and H. Li. Divco: Diverse conditional image synthesis via contrastive generative adversarial network. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 2
[30]
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. In Int. Conf. Learn. Represent., 2018. 4
[31]
M. Noroozi and P. Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Eur. Conf. Comput. Vis., 2016. 2
[32]
A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 2, 4
[33]
T. Park, A. A. Efros, R. Zhang, and J.-Y. Zhu. Contrastive learning for unpaired image-to-image translation. In Eur. Conf. Comput. Vis., 2020. 2
[34]
D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In IEEE Conf. Comput. Vis. Pattern Recog., 2016. 2
[35]
D. Pathak, R. Girshick, P. Dollár, T. Darrell, and B. Hariharan. Learning features by watching objects move. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. 2
[36]
A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In Int. Conf. Learn. Represent., 2016. 5
[37]
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014. 7
[38]
N.-T. Tran, V.-H. Tran, N.-B. Nguyen, L. Yang, and N.-M. Cheung. Self-supervised gan: Analysis and improvement with multi-class minimax game. In Adv. Neural Inform. Process. Syst., 2019. 2
[39]
N.-T. Tran, V.-H. Tran, N.-B. Nguyen, T.-K. Nguyen, and N.-M. Cheung. On data augmentation for gan training. IEEE Trans. Image Process., 2021. 1, 2, 3
[40]
Z. Wu, Y. Xiong, S. X. Yu, and D. Lin. Unsupervised feature learning via non-parametric instance discrimination. In IEEE Conf. Comput. Vis. Pattern Recog., 2018. 2, 3, 4, 8
[41]
Y. Xu, Y. Shen, J. Zhu, C. Yang, and B. Zhou. Generative hierarchical features from synthesizing images. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 2
[42]
C. Yang, Z. Wu, B. Zhou, and S. Lin. Instance localization for self-supervised detection pretraining. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 2
[43]
N. Yu, G. Liu, A. Dundar, A. Tao, B. Catanzaro, L. Davis, and M. Fritz. Dual contrastive loss and attention for gans. arXiv preprint arXiv:2103.16748, 2021. 2
[44]
D. Zhang and A. Khoreva. Pa-gan: Improving gan training by progressive augmentation. In Adv. Neural Inform. Process. Syst., 2019. 1, 6, 7
[45]
H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. In Int. Conf. Learn. Represent., 2017. 1, 2
[46]
H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena. Self-attention generative adversarial networks. In Int. Conf. Mach. Learn., 2019. 4
[47]
H. Zhang, Z. Zhang, A. Odena, and H. Lee. Consistency regularization for generative adversarial networks. In Int. Conf. Learn. Represent., 2020. 2
[48]
R. Zhang, P. Isola, and A. A. Efros. Colorful image colorization. In Eur. Conf. Comput. Vis., 2016. 2
[49]
S. Zhao, Z. Liu, J. Lin, J.-Y. Zhu, and S. Han. Differentiable augmentation for data-efficient gan training. In Adv. Neural Inform. Process. Syst., 2020. 1, 2, 3, 6
[50]
Z. Zhao, S. Singh, H. Lee, Z. Zhang, A. Odena, and H. Zhang. Improved consistency regularization for gans. In Assoc. Adv. Artif. Intell., 2020. 2, 6, 7
[51]
Z. Zhao, Z. Zhang, T. Chen, S. Singh, and H. Zhang. Image augmentations for gan training. arXiv preprint arXiv:2006.02595, 2020. 1, 2, 3
Recommendations
Revisiting multiple instance neural networks
We revisit the problem of solving MIL using neural networks (MINNs), which are ignored in current MIL research community. Our experiments show that MINNs are very effective and efficient.We proposed a novel MI-Net which is centered on learning bag ...
Context-aware MIML instance annotation: exploiting label correlations with classifier chains
In multi-instance multi-label (MIML) instance annotation, the goal is to learn an instance classifier while training on a MIML dataset, which consists of bags of instances paired with label sets; instance labels are not provided in the training data. ...
Multi-instance causal representation learning for instance label prediction and out-of-distribution generalization
NIPS '22: Proceedings of the 36th International Conference on Neural Information Processing SystemsMulti-instance learning (MIL) deals with objects represented as bags of instances and can predict instance labels from bag-level supervision. However, significant performance gaps exist between instance-level MIL algorithms and supervised learners since ...
Comments
Information & Contributors
Information
Published In
December 2021
30517 pages
ISBN:9781713845393
Copyright © 2021 Neural Information Processing Systems Foundation, Inc.
Publisher
Curran Associates Inc.
Red Hook, NY, United States
Publication History
Published: 10 June 2024
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Nov 2024