Abstract
Person re-identification (re-id) aims at matching the same individual in videos captured by multiple cameras, and much progress has been made in recent years due to large scale pedestrian data sets and deep learning-based techniques. In this paper, we propose deep feature embedding learning for person re-id based on lifted structured loss. Triplet loss is commonly used in deep neural networks for person re-id. However, the triplet loss-based framework is not able to make full use of the batch information, and thus needs to choose hard negative samples manually that is time-consuming. To address this problem, we adopt lifted structured loss for deep neural networks that makes the network learn better feature embedding by minimizing intra-class variation and maximizing inter-class variation. Extensive experiments on Market-1501, CUHK03, CUHK01 and VIPeR data sets demonstrate the superior performance of the proposed method over state-of-the-arts in terms of the cumulative match curve (CMC) metric.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Barbosa IG, Cristani M, Caputo B, Rognhaugena A, Theoharis T (2017) Looking beyond appearances: Synthetic training data for deep cnns in re-identification. Comput Vis Image Underst 167:50–62
Chen S-Z, Guo C-C, Lai J-H (2016) Deep ranking for person re-identification via joint representation learning. IEEE Trans Image Process 25(5):2353–2367
Chen D, Yuan Z, Chen B, Zheng N (2016) Similarity learning with spatial constraints for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1268–1277
Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: A deep quadruplet network for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)
Cheng D, Gong Y, Zhou S, Wang J, Zheng N (2016) Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1335–1344
Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn 48(10):2993–3003
Felzenszwalb PF, Girshick R, McAllester D, Ramanan D (2010) Object detection with discriminatively trained partbased models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645
Geng M, Wang Y, Xiang T, Tian Y (2016) Deep transfer learning for person re-identification. arXiv:1611.05244zz
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 580–587
Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proc. IEEE international workshop on performance evaluation for tracking and surveillance (PETS), vol 3, pp 1–7
Guo C-C, Chen S-Z, Lai J-H, Hu X-J, Shi S-C (2014) Multi-shot person re-identification with automatic ambiguity inference and removal. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 3540–3545
He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385
He Z, Zhang Z, Jung C (2018) Deep feature embedding learning for person re-identification using lifted structured loss. In: Proceedings of IEEE conference on acoustics, speech and signal processing (ICASSP), pp 1957–1961
Hermans A, Beyer L, Leibe B (2017) In defense of the triplet loss for person re-identification. arXiv:1703.07737
Jin H, Wang X, Liao S, Li SZ (2017) Deep person re-identification with improved embedding. arXiv:1705.03332
Jones M, Marks T (2015) An improved deep learning architecture for person re-identification. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR)
Karen S, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Khamis S, Kuo C-H, Singh VK, Shet V D, Davis LS (2014) Joint learning for attribute-consistent person re-identification. In: Proc. ECCV Workshops, vol 3, pp 134–146
Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: Proc. IEEE conference on computer vision and pattern recognition (CVPR), pp 2288–2295
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems (NIPS), pp 1097–1105
Li W, Wang X (2013) Locally aligned feature transforms across views. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3594-3601
Li W, Zhao R, Xiao T, Wang X (2014) Deepreid, Deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2197–2206
Liu H, Feng J, Qi M, Jiang J, Yan S (2017) End-to-end comparative attention networks for person re-identification. IEEE Trans Image Process 26(7):3492–3506
Liu X, Song M, Zhao Q, Tao D, Chen C, Bu J (2012) Attribute-restricted latent topic model for person reidentification. Pattern Recogn 45(12):4204–4213
Nanda A, Chauhan DS, Sa PK, Bakshi S (2017) Illumination and scale invariant relevant visual features with hypergraph-based learning for multi-shot person re-identification. Multimedia Tools and Applications, pp 1–26
Nanda A, Sa PK, Choudhury SK, Bakshi S, Majhi B (2017) A Neuromorphic Person Re-Identification Framework for Video Surveillance. IEEE Access 5:6471–6482
Paisitkriangkrai S, Shen C, van den Hengel A (2015) Learning to rank in person re-identification with metric ensembles. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1846–1855
Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 815–823
Shi H, Yang Y, Zhu X, Liao S, Lei Z, Zheng W, Li S Z (2016) Embedding deep metric for person re-identification: A study against large variations. In: Proc. European conference on computer vision (ECCV), pp 732–748, Springer
Song H, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4004–4012
Subramaniam A, Chatterjee M, Mittal A (2016) Deep neural networks with inexact matching for person re-identification. In: Advances in neural information processing systems (NIPS), pp 2667–2675
Varior RR, Haloi M, Wang G (2016) Gated siamese convolutional neural network architecture for human re-identification. In: Proc. European conference on computer vision (ECCV), pp 791–808, Springer
Varior RR, Shuai B, Lu J, Xu D, Wang G (2016) A siamese long short-term memory architecture for human re-identification. In: Proc. European Conference on Computer Vision, pp 135–153, Springer
Wang F, Zuo W, Lin L, Zhang D, Zhang L (2016) Joint learning of single-image and cross-image representations for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1288–1296
Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244
Xiao T, Li H, Ouyang W, Wang X (2016) Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1249–1258
Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: Proc. European conference on computer vision (ECCV), pp 1–16, Springer
Yi D, Lei Z, Liao S, Li SZ (2014) Deep metric learning for person re-identification. In: Proc. 22nd international conference on pattern recognition (ICPR), pp 34–39
Zhang L, Xiang T, Gong S (2016) Learning a discriminative null space for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1239–1248
Zhang Y, Li B, Lu H, Irie A, Ruan X (2016) Sample specific svm learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1278–1287
Zhao H, Tian M, Sun S, Shao J, Yan J, Yi S, Wang X, Tang X (2017) Spindle net: Person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 907–915
Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2528–2535
Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: A benchmark. In: Proc. IEEE international conference on computer vision (ICCV)
Zheng L, Yang Y, Hauptmann AG (2016) Person reidentification: Past, present and future. arXiv:1610.02984
Zheng Z, Zheng L, Yang Y (2016) A discriminatively learned cnn embedding for person re-identification. arXiv:1611.05666
Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE international conference on computer vision (ICCV)
Acknowledgements
The authors are grateful to Mr. Jun Sun in Xidian University for his contributions to various experiments and collected data.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
An earlier version of this paper was presented in the 2018 IEEE Conference on Acoustics, Speech, and Signal Processing (ICASSP), Calgary, Alberta, Canada, April 15-20, 2018 [13]. This work was supported by the National Natural Science Foundation of China (No. 61271298) and the International S, T Cooperation Program of China (No. 2014DFG12780).
Rights and permissions
About this article
Cite this article
He, Z., Jung, C., Fu, Q. et al. Deep feature embedding learning for person re-identification based on lifted structured loss. Multimed Tools Appl 78, 5863–5880 (2019). https://doi.org/10.1007/s11042-018-6408-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-6408-4