research-article

Learning Disentangled Features for Person Re-identification under Clothes Changing

Authors:

Patrick P. K. Chan,

Keke ChenAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications and Applications, Volume 19, Issue 6

Article No.: 180, Pages 1 - 21

https://doi.org/10.1145/3584359

Published: 31 May 2023 Publication History

Abstract

Clothes changing is one of the challenges in person re-identification (ReID), since clothes provide remarkable and reliable information for decision, especially when the resolution of an image is low. Variation of clothes significantly downgrades standard ReID models, since the clothes information dominates the decisions. The performance of the existing methods considering clothes changing is still not satisfying, since they fail to extract sufficient identity information that excludes clothes information. This study aims to disentangle identity, clothes, and unrelated features with a Generative Adversarial Network (GAN). A GAN model with three encoders, one generator, and three discriminators, and its training procedure are proposed to learn these kinds of features separately and exclusively. Experimental results indicate that our model generally achieves the best performance among state-of-the-art methods in both ReID tasks with and without clothes changing, which confirms that the identity, clothes, and unrelated features are extracted by our model more precisely and effectively.

References

[1]

S. Bak, P. Carr, and J. F. Lalonde. 2018. Domain adaptation through synthesis for unsupervised person re-identification. In Proceedings of the European Conference on Computer Vision (ECCV’18).

[2]

Igor Barros Barbosa, Marco Cristani, Del A. Bue, Loris Bazzani, and Vittorio Murino. 2012. Re-identification with RGB-D sensors. In Proceedings of the 1st International Workshop on Re-identification in Conjunction with ECCV.

Digital Library

[3]

Patrick P. K. Chan, YuanChao Liang, Fei Zhang, and Daniel S. Yeung. 2021. Distribution-based adversarial filter feature selection against evasion attack. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’21). 1–8. DOI:

[4]

Patrick P. K. Chan, Yaxuan Wang, Natasha Kees, and Daniel S. Yeung. 2021. Multiple-model based defense for deep reinforcement learning against adversarial attack. In Proceedings of the Conference on Artificial Neural Networks and Machine Learning (ICANN’21). 42–53.

Digital Library

[5]

Jiaxing Chen, Xinyang Jiang, Fudong Wang, Jun Zhang, Feng Zheng, Xing Sun, and Wei-Shi Zheng. 2021. Learning 3D shape feature for texture-insensitive person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’21). 8142–8151. DOI:

[6]

Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. 886–893.

Digital Library

[7]

Chanho Eom and Bumsub Ham. 2019. Learning disentangled representation for robust person re-identification. In Proceedings of the 33rd International Conference on Neural Information Processing Systems.

Digital Library

[8]

Xing Fan, Hao Luo, Xuan Zhang, Lingxiao He, Chi Zhang, and Wei Jiang. 2018. SCPNet: Spatial-channel parallelism network for joint holistic and partial person re-identification. In Proceedings of Asian Conference on Computer Vision. 19–34.

[9]

Yixiao Ge, Zhuowan Li, Haiyu Zhao, Guojun Yin, Shuai Yi, Xiaogang Wang, and Hongsheng Li. 2018. FD-GAN: Pose-guided feature distilling GAN for robust person re-identification. In Proceedings of the Conference and Workshop on Neural Information Processing Systems. 1222–1233.

[10]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Y. Bengio. 2014. Generative adversarial nets. Advances in Neural Information Processing Systems, Vol. 27.

Digital Library

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’16). 770–778. DOI:https://doi.org/10.1109/CVPR.2016.90

[12]

L. He, Y. Wang, W. Liu, X. Liao, H. Zhao, Z. Sun, and J. Feng. 2019. Foreground-aware pyramid reconstruction for alignment-free occluded person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’19).

[13]

Peixian Hong, Tao Wu, Ancong Wu, Xintong Han, and Wei-Shi Zheng. 2021. Fine-grained shape-appearance mutual learning for cloth-changing person re-identification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’21). 10508–10517. DOI:

[14]

Yan Huang, Qiang Wu, Jingsong Xu, and Yi Zhong. 2019. Celebrities-ReID: A benchmark for clothes variation in long-term person re-identification. In Proceedings of International Joint Conference on Neural Networks. 1–8.

[15]

Yan Huang, Jingsong Xu, Qiang Wu, Yi Zhong, Peng Zhang, and Zhaoxiang Zhang. 2019. Beyond scalar neuron: Adopting vector-neuron capsules for long-term person re-identification. IEEE Trans. Circ. Syst. Video Technol. (2019), 1–1.

[16]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the IEEE International Conference on Machine Learning.

[17]

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei Efros. 2016. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[18]

D. Jia, D. Wei, R. Socher, L. J. Li, L. Kai, and F. F. Li. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 248–255.

[19]

Martin Kostinger, Martin Hirzer, Paul Wohlhart, Peter M. Roth, and Horst Bischof. 2012. Large scale metric learning from equivalence constraints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2288–2295.

[20]

Van Der Maaten Laurens and Geoffrey Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2605 (2008), 2579–2605.

[21]

Yaoyu Li, Tianzhu Zhang, Lingyu Duan, and Changsheng Xu. 2018. A unified generative adversarial framework for image generation and person re-identification. In Proceedings of the ACM International Conference on Multimedia. 163–172. DOI:

Digital Library

[22]

Yu-Jhe Li, Yun-Chun Chen, Yen-Yu Lin, Xiaofei Du, and Yu-Chiang Frank Wang. 2019. Recover and identify: A generative dual model for cross-resolution person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’19). 8089–8098. DOI:

[23]

Yu-Jhe Li, Xinshuo Weng, and Kris M. Kitani. 2021. Learning shape representations for person re-identification under clothing change. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV’21). 2431–2440.

[24]

Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z. Li. 2015. Person re-identification by local maximal occurrence representation and metric learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2197–2206.

[25]

Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. DeepFashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1096–1104. DOI:

[26]

Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019. Bag of tricks and a strong baseline for deep person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19).

[27]

Andrew L. Maas. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proceedings of the IEEE International Conference on Machine Learning.

[28]

Dai Pingyang, Rongrong Ji, Haibin Wang, Qiong Wu, and Yuyu Huang. 2018. Cross-modality person re-identification with generative adversarial training. In Proceedings of the International Joint Conference on Artificial Intelligence. 677–683. DOI:

[29]

Xuelin Qian, Yanwei Fu, Wenxuan Wang, Tao Xiang, Yang Wu, Yu-Gang Jiang, and Xiangyang Xue. 2017. Pose-normalized image generation for person re-identification. In Proceedings of the European Conference on Computer Vision (ECCV’17).

[30]

Xuelin Qian, Wenxuan Wang, Li Zhang, Fangrui Zhu, Yanwei Fu, Tao Xiang, Yu Gang Jiang, and Xiangyang Xue. 2020. Long-term cloth-changing person re-identification. In Retrieved from https://arXiv:2005.07862.

[31]

Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. 2017. Dynamic routing between capsules. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates.

Digital Library

[32]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15 (June 2014), 1929–1958.

Digital Library

[33]

Yumin Suh, Jingdong Wang, Siyu Tang, Tao Mei, and Kyoung Lee. 2018. Part-aligned bilinear representations for person re-identification. In Proceedings of the European Conference on Computer Vision (ECCV’18).

Digital Library

[34]

Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. 2018. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Proceedings of the European Conference on Computer Vision (ECCV’18). 501–518.

Digital Library

[35]

Fangbin Wan, Yang Wu, Xuelin Qian, and Yanwei Fu. 2020. When person re-identification meets changing clothes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[36]

Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. 2018. Learning discriminative features with multiple granularities for person re-identification. In Proceedings of ACM International Conference on Multimedia.

Digital Library

[37]

Longhui Wei, Shiliang Zhang, Hantao Yao, Wen Gao, and Qi Tian. 2017. GLAD: Global-local-alignment descriptor for pedestrian retrieval. In Proceedings of the ACM International Conference on Multimedia.

Digital Library

[38]

Wanlu Xu, Hong Liu, Wei Shi, Ziling Miao, Zhisheng Lu, and Feihu Chen. 2021. Adversarial feature disentanglement for long-term person re-identification. In Proceedings of the 13th International Joint Conference on Artificial Intelligence (IJCAI’21), Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 1201–1207.

[39]

Qize Yang, Ancong Wu, and Weishi Zheng. 2020. Person re-identification by contour sketch under moderate clothing change. IEEE Trans. Pattern Anal. Mach. Intell. 43, 6 (2020), 2029–2046. DOI:https://doi.org/10.1109/TPAMI.2019.2960509

[40]

Hantao Yao, Shiliang Zhang, Yongdong Zhang, Jintao Li, and Qi Tian. 2019. Deep representation learning with part loss for person re-identification. IEEE Transactions on Image Processing 28, 6 (2019), 2860–2871. DOI:https://doi.org/10.1109/TIP.2019.2891888

[41]

Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li. 2019. Adversarial examples: Attacks and defenses for deep learning. IEEE Trans. Neural Netw. Learn. Syst. 30, 9 (2019), 2805–2824. DOI:

[42]

Chongzhi Zhang, Aishan Liu, Xianglong Liu, Yitao Xu, Hang Yu, Yuqing Ma, and Tianlin Li. 2021. Interpreting and improving adversarial robustness of deep neural networks with neuron sensitivity. IEEE Trans. Image Process. 30 (2021), 1291–1304. DOI:

Digital Library

[43]

Junhao Zheng, Patrick P. K. Chan, Huiyang Chi, and Zhimin He. 2022. A concealed poisoning attack to reduce deep neural networks robustness against adversarial samples. Info. Sci. 615 (2022), 758–773. DOI:

Digital Library

[44]

Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015. Scalable person re-identification: A benchmark. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15). 1116–1124.

[45]

Zhedong Zheng, Xiaodong Yang, Zhiding Yu, Liang Zheng, Yi Yang, and Jan Kautz. 2019. Joint discriminative and generative learning for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2133–2142.

[46]

Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. 2017. Random erasing data augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34.

[47]

Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. 2019. Omni-scale feature learning for person re-identification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’19). 3701–3711.

Cited By

Nguyen VMantini PShah S(2024)Contrastive Clothing and Pose Generation for Cloth-Changing Person Re-Identification2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00749(7541-7549)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00749
Nguyen VMantini PShah S(2024)Occluded Cloth-Changing Person Re-Identification via Occlusion-aware Appearance and Shape Reasoning2024 IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)10.1109/AVSS61716.2024.10672564(1-8)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AVSS61716.2024.10672564
Bi XWang H(2024)Appearance-Pose Joint Coordinates Information Collaboration Model for clothes-changing person re-identificationExpert Systems with Applications10.1016/j.eswa.2023.122473241(122473)Online publication date: May-2024
https://doi.org/10.1016/j.eswa.2023.122473
Show More Cited By

Index Terms

Learning Disentangled Features for Person Re-identification under Clothes Changing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
      2. Computer vision tasks
        Visual content-based indexing and retrieval
2. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Adversarial retrieval

Recommendations

Deep multi-instance learning for end-to-end person re-identification

In this paper, we introduce a deep multi-instance learning framework to boost the instance-level person re-identification performance. Motivated by the observation of considerably dramatic and complex varieties of visual appearances in many current ...
A loss combination based deep model for person re-identification

The Convolutional Neural Network (CNN) has significantly improved the state-of-the-art in person re-identification (re-ID). In the existing available identification CNN model, the softmax loss function is employed as the supervision signal to train the ...
Disentangled Feature Learning for Semi-supervised Person Re-identification
Pattern Recognition and Computer Vision
Abstract
Semi-supervised learning has become more and more popular in person re-identification because acquiring annotations is time-consuming and cumbersome. In this paper, we design a Disentangled Feature Learning (DFL) model based on an encoder-decoder ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 19, Issue 6

November 2023

858 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3599695

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2023

Online AM: 16 February 2023

Accepted: 28 January 2023

Revised: 19 November 2022

Received: 29 November 2021

Published in TOMM Volume 19, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Guangdong Province, China
Fundamental Research Funds for the Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
506
Total Downloads

Downloads (Last 12 months)229
Downloads (Last 6 weeks)17

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nguyen VMantini PShah S(2024)Contrastive Clothing and Pose Generation for Cloth-Changing Person Re-Identification2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00749(7541-7549)Online publication date: 17-Jun-2024
https://doi.org/10.1109/CVPRW63382.2024.00749
Nguyen VMantini PShah S(2024)Occluded Cloth-Changing Person Re-Identification via Occlusion-aware Appearance and Shape Reasoning2024 IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)10.1109/AVSS61716.2024.10672564(1-8)Online publication date: 15-Jul-2024
https://doi.org/10.1109/AVSS61716.2024.10672564
Bi XWang H(2024)Appearance-Pose Joint Coordinates Information Collaboration Model for clothes-changing person re-identificationExpert Systems with Applications10.1016/j.eswa.2023.122473241(122473)Online publication date: May-2024
https://doi.org/10.1016/j.eswa.2023.122473
Wang YYu HYan YSong SLiu BLu YEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Exploring Shape Embedding for Cloth-Changing Person Re-Identification via 2D-3D CorrespondencesProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3611715(7121-7130)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3611715

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents