Face Clustering via Adaptive Aggregation of Clean Neighbors
Authors: 
Shiyong Hong, Yaobin Zhang, Xu Ling, Weihong Deng, Yunfeng Yin, Yingjie Zhang, Hongzhi Shi, Dongchao WenAuthors Info & Claims
Shiyong Hong, Yaobin Zhang, Xu Ling, Weihong Deng, Yunfeng Yin, Yingjie Zhang, Hongzhi Shi, Dongchao WenAuthors Info & ClaimsPages 83 - 92
Abstract
Face clustering has been widely studied to solve the problem of data annotation in large-scale unlabeled face images. In recent years, state-of-the-art performance has been updated every year based on the application of Graph Convolutional Networks(GCN) in face clustering tasks. The existing GCN-based methods make each node accept the feature information from its neighbors, and then aggregate the neighbors' information with equal weights to learn enhanced feature embedding. However, rare attention has been paid to improving the quality of aggregated information. In this paper, we aim to make each node aggregate the feature information that is more conducive to clustering. The proposed novel method named Adaptive Aggregation of Clean Neighbors(AACN) has two stages of preparation before inputting the graph into GCN. Specifically, we first design a noise edge cleaner to remove the wrong neighbors of each node to ensure that they receive more accurate neighbor information. Then, we carefully allocate adaptive weights to the clean neighbors of each node, and make all nodes aggregate the received information via adaptive aggregation instead of mean aggregation. The two-stage preparation enables nodes to learn more robust features through the GCN module. Experiments on standard face clustering benchmark MS1M show that AACN has achieved state-of-the-art performance, significantly boosting the pairwise F-score from 92.79% to 93.72% on 584K unlabeled face images and from 83.99% to 86.41% on 5.21M unlabeled face images.
Supplementary Material
This presentation video introduces our paper, starting with the research objectives of face clustering. Then we show the problems of existing methods and briefly introduce the motivation and framework of our approach. Finally, we show the sota performance of our method on the MS-celeb-1M dataset.
- Download
- 78.48 MB
References
[1]
James Atwood and Don Towsley. 2016. Diffusion-convolutional neural networks. Advances in neural information processing systems, Vol. 29 (2016).
[2]
Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The first international conference on language resources and evaluation workshop on linguistics coreference, Vol. 1. Citeseer, 563--566.
[3]
Yizong Cheng. 1995. Mean shift, mode seeking, and clustering. IEEE transactions on pattern analysis and machine intelligence, Vol. 17, 8 (1995), 790--799.
[4]
Francc ois Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In CVPR.
[5]
Thomas Cover and Peter Hart. 1967. Nearest neighbor pattern classification. TIT (1967).
[6]
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, Vol. 29 (2016).
[7]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In Kdd, Vol. 96. 226--231.
[8]
Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. 2019. Hypergraph neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 3558--3565.
[9]
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 315--323.
[10]
Senhui Guo, Jing Xu, Dapeng Chen, Chao Zhang, Xiaogang Wang, and Rui Zhao. 2020. Density-Aware Feature Embedding for Face Clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6698--6706.
[11]
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. 2016. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In European conference on computer vision. Springer, 87--102.
[12]
William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. arXiv preprint arXiv:1706.02216 (2017).
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026--1034.
[14]
Jeffrey Ho, Ming-Husang Yang, Jongwoo Lim, Kuang-Chih Lee, and David Kriegman. 2003. Clustering appearances of objects under varying illumination conditions. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., Vol. 1. IEEE, I--I.
[15]
Anil K Jain, Jianchang Mao, and K Moidin Mohiuddin. 1996. Artificial neural networks: A tutorial. Computer, Vol. 29, 3 (1996), 31--44.
[16]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
[17]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012).
[18]
John Boaz Lee, Ryan A Rossi, Xiangnan Kong, Sungchul Kim, Eunyee Koh, and Anup Rao. 2018. Higher-order graph convolutional networks. arXiv preprint arXiv:1809.07697 (2018).
[19]
Wanhua Li, Yueqi Duan, Jiwen Lu, Jianjiang Feng, and Jie Zhou. 2020. Graph-Based Social Relation Reasoning. In European Conference on Computer Vision. Springer, 18--34.
[20]
Wei-An Lin, Jun-Cheng Chen, Carlos D Castillo, and Rama Chellappa. 2018. Deep density clustering of unconstrained faces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8128--8137.
[21]
Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE transactions on information theory, Vol. 28, 2 (1982), 129--137.
[22]
Linyuan Lü and Tao Zhou. 2011. Link prediction in complex networks: A survey. Physica A: statistical mechanics and its applications, Vol. 390, 6 (2011), 1150--1170.
[23]
Federico Monti, Oleksandr Shchur, Aleksandar Bojchevski, Or Litany, Stephan Günnemann, and Michael M Bronstein. 2018. Dual-primal graph convolutional networks. arXiv preprint arXiv:1806.00770 (2018).
[24]
Xuan-Bac Nguyen, Duc Toan Bui, Chi Nhan Duong, Tien D Bui, and Khoa Luu. 2021. Clusformer: A Transformer Based Clustering Approach to Unsupervised Large-Scale Face and Visual Landmark Recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10847--10856.
[25]
Yifan Niu and Weihong Deng. 2022. Federated learning for face recognition with gradient correction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 1999--2007.
[26]
Charles Otto, Dayong Wang, and Anil K Jain. 2017. Clustering millions of faces by identity. IEEE transactions on pattern analysis and machine intelligence, Vol. 40, 2 (2017), 289--303.
[27]
Karl Pearson. 1901. LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, Vol. 2, 11 (1901), 559--572.
[28]
Lorenzo Rosasco, Ernesto De Vito, Andrea Caponnetto, Michele Piana, and Alessandro Verri. 2004. Are loss functions all the same? Neural computation, Vol. 16, 5 (2004), 1063--1076.
[29]
Aravind Sankar, Xinyang Zhang, and Kevin Chen-Chuan Chang. 2017. Motif-based convolutional neural network on graphs. arXiv preprint arXiv:1711.05697 (2017).
[30]
Shuai Shen, Wanhua Li, Zheng Zhu, Guan Huang, Dalong Du, Jiwen Lu, and Jie Zhou. 2021. Structure-Aware Face Clustering on a Large-Scale Graph with 107 Nodes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
[31]
Yichun Shi, Charles Otto, and Anil K Jain. 2018. Face clustering: representation and pairwise constraints. IEEE Transactions on Information Forensics and Security, Vol. 13, 7 (2018), 1626--1640.
[32]
Robin Sibson. 1973. SLINK: an optimally efficient algorithm for the single-link cluster method. The computer journal, Vol. 16, 1 (1973), 30--34.
[33]
Yi Sun, Yuheng Chen, Xiaogang Wang, and Xiaoou Tang. 2014. Deep learning face representation by joint identification-verification. Advances in neural information processing systems, Vol. 27 (2014).
[34]
Makarand Tapaswi, Marc T Law, and Sanja Fidler. 2019. Video face clustering with unknown number of clusters. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5027--5036.
[35]
Shikhar Vashishth, Prateek Yadav, Manik Bhandari, and Partha Talukdar. 2019. Confidence-based graph convolutional networks for semi-supervised learning. In The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 1792--1801.
[36]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rJXMpikCZ
[37]
Fei Wang, Liren Chen, Cheng Li, Shiyao Huang, Yanjie Chen, Chen Qian, and Chen Change Loy. 2018. The devil of face recognition is in the noise. In Proceedings of the European Conference on Computer Vision (ECCV). 765--780.
[38]
Mei Wang and Weihong Deng. 2020. Deep face recognition with clustering based domain adaptation. Neurocomputing, Vol. 393 (2020), 1--14.
[39]
Mei Wang, Yaobin Zhang, and Weihong Deng. 2021. Meta Balanced Network for Fair Face Recognition. IEEE transactions on pattern analysis and machine intelligence (2021).
[40]
Yaohua Wang, Yaobin Zhang, Fangyi Zhang, Senzhang Wang, Ming Lin, YuQi Zhang, and Xiuyu Sun. 2022. Ada-nets: Face clustering via adaptive neighbour discovery in the structure space. arXiv preprint arXiv:2202.03800 (2022).
[41]
Zhongdao Wang, Liang Zheng, Yali Li, and Shengjin Wang. 2019. Linkage based face clustering via graph convolution network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1117--1125.
[42]
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, Vol. 32, 1 (2020), 4--24.
[43]
Yiqing Xie, Sha Li, Carl Yang, Raymond Chi Wing Wong, and Jiawei Han. 2020. When do gnns work: Understanding and improving neighborhood aggregation. In IJCAI International Joint Conference on Artificial Intelligence.
[44]
Yifan Xing, Tong He, Tianjun Xiao, Yongxin Wang, Yuanjun Xiong, Wei Xia, David Wipf, Zheng Zhang, and Stefano Soatto. 2021. Learning hierarchical graph neural networks for image clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3467--3477.
[45]
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation learning on graphs with jumping knowledge networks. In International conference on machine learning. PMLR, 5453--5462.
[46]
Huafeng Yang, Xingjian Chen, Fangyi Zhang, Guangyue Hei, Yunjie Wang, and Rong Du. 2021. GCN-Based Linkage Prediction for Face Clustering on Imbalanced Datasets: An Empirical Study. arXiv preprint arXiv:2107.02477 (2021).
[47]
Lei Yang, Dapeng Chen, Xiaohang Zhan, Rui Zhao, Chen Change Loy, and Dahua Lin. 2020. Learning to cluster faces via confidence and connectivity estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13369--13378.
[48]
Lei Yang, Xiaohang Zhan, Dapeng Chen, Junjie Yan, Chen Change Loy, and Dahua Lin. 2019. Learning to cluster faces on an affinity graph. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2298--2306.
[49]
Xiaohang Zhan, Ziwei Liu, Junjie Yan, Dahua Lin, and Chen Change Loy. 2018. Consensus-driven propagation in massive unlabeled data for face recognition. In Proceedings of the European Conference on Computer Vision (ECCV). 568--583.
[50]
Yaobin Zhang, Weihong Deng, Mei Wang, Jiani Hu, Xian Li, Dongyue Zhao, and Dongchao Wen. 2020b. Global-local gcn: Large-scale label noise cleansing for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7731--7740.
[51]
Yaobin Zhang, Weihong Deng, Yaoyao Zhong, Jiani Hu, Xian Li, Dongyue Zhao, and Dongchao Wen. 2021. Adaptive label noise cleaning with meta-supervision for deep face recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 15065--15075.
[52]
Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2020a. Deep learning on graphs: A survey. IEEE Transactions on Knowledge and Data Engineering (2020).
[53]
Yaoyao Zhong, Weihong Deng, Han Fang, Jiani Hu, Dongyue Zhao, Xian Li, and Dongchao Wen. 2021a. Dynamic Training Data Dropout for Robust Deep Face Recognition. IEEE Transactions on Multimedia, Vol. 24 (2021), 1186--1197.
[54]
Yaoyao Zhong, Weihong Deng, Jiani Hu, Dongyue Zhao, Xian Li, and Dongchao Wen. 2021b. SFace: Sigmoid-constrained hypersphere loss for robust face recognition. IEEE Transactions on Image Processing, Vol. 30 (2021), 2587--2598.
[55]
Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Dalong Du, et al. 2021. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10492--10502.
Index Terms
- Face Clustering via Adaptive Aggregation of Clean Neighbors
Recommendations
Integrated System for Face Detection, Clustering and Recognition
ICMLT '18: Proceedings of the 2018 International Conference on Machine Learning TechnologiesRecent years, many approaches have achieved remarkable performances in face detection, clustering and recognition. However, real life can barely see systems designed for their combination, and thousands of videos and images need to be handled in a ...
Improving face clustering using social context
MM '10: Proceedings of the 18th ACM international conference on MultimediaIn this paper we describe an algorithm to improve the performance of face clustering using the social relationship of people. One common challenge in face clustering techniques is that very often the faces of the same person are clustered into different ...
Multi-cue Augmented Face Clustering
MM '15: Proceedings of the 23rd ACM international conference on MultimediaFace clustering is an important but challenging task since facial images always have huge variation due to change in facial expressions, head poses and partial occlusions, etc. Moreover, face clustering is actually an unsupervised problem which makes it ...
Comments
Information & Contributors
Information
Published In
October 2022
106 pages
ISBN:9781450394925
DOI:10.1145/3552458
- Program Chairs:
- Dingwen Zhang,
- Chaowei Fang,
- Wu Liu,
- Xinchen Liu,
- Jingkuan Song,
- Hongyuan Zhu,
- Wenbing Huang,
- John Smith
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '22
Sponsor:
Acceptance Rates
HCMA '22 Paper Acceptance Rate 12 of 21 submissions, 57%;
Overall Acceptance Rate 12 of 21 submissions, 57%
Upcoming Conference
MM '24
- Sponsor:
- sigmm
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 106Total Downloads
- Downloads (Last 12 months)38
- Downloads (Last 6 weeks)1
Reflects downloads up to 26 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in