Abstract
Augmentation strategies for image recognition based on local image patches have gained widespread popularity. Their main idea is to replace or remove some local regions of the image. The advantage of these methods is that they change part of the image and force the network to pay attention to the less significant parts, which leads to a greater generalization capacity of the network. While these methods work good for image recognition, they do not perform as well for face recognition tasks. The purpose of this work is to create augmentation specialized for face recognition and devoid of the shortcomings of previous works. We present FaceMix: a flexible face-specific data augmentation technique that transfers a local area of an image to another image. The method has two operating modes: it can generate new images within a class, and it can generate images for a class, using face data from other classes, and these two modes also could be combined. FaceMix is helping to solve the problems of class imbalance and insufficient number of images per identity. A feature of this method is that the number of possible artificial images grows quadratically with the growth of real images. Experiments on face recognition benchmarks, such as CFP-FP, AgeDB, CALFW, CPLFW, XQLFW, SLLFW, RFW and MegaFace, demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An, X., et al.: Partial FC: training 10 million identities on a single machine. arXiv preprint arXiv:2010.05222 (2020)
Buslaev, A., Iglovikov, V.I., Khvedchenya, E., Parinov, A., Druzhinin, M., Kalinin, A.A.: Albumentations: fast and flexible image augmentations. Information 11(2), 125 (2020)
Conway, D., Simon, L., Lechervy, A., Jurie, F.: Training face verification models from generated face identity data. arXiv preprint arXiv:2108.00800 (2021)
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. In: CVPR, pp. 4690–4699 (2019)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017)
Goodfellow, I.J., et al.: Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014)
Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 87–102. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_6
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Hu, G., Peng, X., Yang, Y., Hospedales, T.M., Verbeek, J.: Frankenstein: learning deep face representations using small data. IEEE Trans. Image Process. 27(1), 293–303 (2017)
Inoue, H.: Data augmentation by pairing samples for images classification. arXiv preprint arXiv:1801.02929 (2018)
Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The MegaFace benchmark: 1 million faces for recognition at scale. In: CVPR, pp. 4873–4882 (2016)
Knoche, M., Hormann, S., Rigoll, G.: Cross-quality LFW: a database for analyzing cross-resolution image face recognition in unconstrained environments. In: FG (2021)
Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., Zafeiriou, S.: AgeDB: the first manually collected, in-the-wild age database. In: CVPR Workshops (2017)
Park, S., Hong, Y., Heo, B., Yun, S., Choi, J.Y.: The majority can help the minority: context-rich minority oversampling for long-tailed classification. In: CVPR, pp. 6887–6896 (2022)
Ruiz, N., Chong, E., Rehg, J.M.: Fine-grained head pose estimation without keypoints. In: CVPR Workshops, pp. 2074–2083 (2018)
Sandfort, V., Yan, K., Pickhardt, P.J., Summers, R.M.: Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks. Sci. Rep. 9(1), 1–9 (2019)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: CVPR, pp. 815–823 (2015)
Sengupta, S., Chen, J.C., Castillo, C., Patel, V.M., Chellappa, R., Jacobs, D.W.: Frontal to profile face verification in the wild. In: WACV (2016)
Smirnov, E., Garaev, N., Galyuk, V., Lukyanets, E.: Prototype memory for large-scale face representation learning. IEEE Access 10, 12031–12046 (2022)
Smirnov, E., Melnikov, A., Novoselov, S., Luckyanets, E., Lavrentyeva, G.: Doppelganger mining for face representation learning. In: ICCV Workshops, pp. 1916–1923 (2017)
Smirnov, E., Melnikov, A., Oleinik, A., Ivanova, E., Kalinovskiy, I., Luckyanets, E.: Hard example mining with auxiliary embeddings. In: CVPR Workshops, pp. 37–46 (2018)
Smirnov, E., et al.: Face representation learning using composite mini-batches. In: ICCV Workshops, pp. 551–559 (2019)
Summers, C., Dinneen, M.J.: Improved mixed-example data augmentation. In: WACV, pp. 1262–1270. IEEE (2019)
Sun, R., Masson, C., Hénaff, G., Thome, N., Cord, M.: Swapping semantic contents for mixing images. arXiv preprint arXiv:2205.10158 (2022)
Szegedy, C., Toshev, A., Erhan, D.: Deep neural networks for object detection. In: NeurIPS, vol. 26 (2013)
Takahashi, R., Matsubara, T., Uehara, K.: Data augmentation using random image cropping and patching for deep CNNs. IEEE Trans. Circuits Syst. Video Technol. 30(9), 2917–2931 (2019)
Wang, H., et al.: CosFace: large margin cosine loss for deep face recognition. In: CVPR (2018)
Wang, M., Deng, W., Hu, J., Tao, X., Huang, Y.: Racial faces in the wild: reducing racial bias by information maximization adaptation network. In: ICCV (2019)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: regularization strategy to train strong classifiers with localizable features. In: ICCV, pp. 6023–6032 (2019)
Zhang, H., et al.: Context encoding for semantic segmentation. In: CVPR, pp. 7151–7160 (2018)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: Mixup: beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
Zhang, L., Huang, S., Liu, W.: Intra-class part swapping for fine-grained image classification. In: WACV, pp. 3208–3217 (2021)
Zhang, N., Deng, W.: Fine-grained LFW database. In: ICB (2016)
Zheng, T., Deng, W.: Cross-pose LFW: a database for studying cross-pose face recognition in unconstrained environments. Beijing University of Posts and Telecommunications, Technical report (2018)
Zheng, T., Deng, W., Hu, J.: Cross-age LFW: a database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197 (2017)
Zheng, Y., Chen, Q., Zhang, Y.: Deep learning and its new progress in object and behavior recognition. J. Image Graph. 19(2), 175–184 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Garaev, N., Smirnov, E., Galyuk, V., Lukyanets, E. (2023). FaceMix: Transferring Local Regions for Data Augmentation in Face Recognition. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_56
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)