Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

GaitCTCG: cross-view gait recognition via cascaded residual temporal shift and comprehensive multi-granularity learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Gait is one of the most popular biometrics for identity authentication today due to its noninvasive perception. Diverse spatial representations and temporal modeling are crucial information for gait recognition, especially under covariation conditions. However, most existing algorithms only focus on the specific temporal-scale modeling (i.e., short-term or long-term) and single-level or single-granularity (i.e., global or local) spatial representation; these algorithms lack flexibility and diversity for the extraction of features. To address this issue, we propose a cascaded residual temporal shift and comprehensive multi-granularity learning (GaitCTCG) network for gait recognition. Specifically, a cascaded residual temporal shift (CRTS) module was proposed to capture multiple receptive fields in the temporal dimension without any additional parameters or computational cost, thereby flexibly integrating features of different temporal scales. A comprehensive multi-granularity learning (CMGL) module was designed with a multi-layer multi-granularity scheme to extract and fuse comprehensive spatial representations at different scales, exploiting various visual details of the input. Furthermore, a micro gait energy generator (MGEG) was also developed to distill sequence representation, which refined the local temporal segments while preserving richer spatial information. Extensive experiments on two of the most popular public datasets demonstrated the state-of-the-art performance of our proposed method, which achieved rank-1 accuracies of 98.0%, 95.3%, and 84.4% in the normal walking (NM), bag carrying (BG), and coat-wearing (CL) scenarios on CASIA-B, and 91.2% on OUMVLP. The source code will be published at https://github.com/HUAFOR/GaitCTCG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The two publicly available gait datasets used to evaluate our approach, CASIA-B and OUMVLP, are available at the following two links, respectively: http://www.cbsr.ia.ac.cn/china/Gait%20Databases%20CH.asp; http://www.am.sanken.osaka-u.ac.jp/BiometricDB/GaitMVLP.html

References

  1. Premalatha G, Chandramani VP (2020) Improved gait recognition through gait energy image partitioning. Comput Intell 36(3):1261–1274

  2. Macoveciuc I, Rando CJ, Borrion H (2019) Forensic gait analysis and recognition: standards of evidence admissibility. J Forensic Sci 64(5):1294–1303

    Article  PubMed  Google Scholar 

  3. Zhang Z, Tran L, Yin X, Atoum Y, Liu X, Wan J, Wang N (2019) Gait recognition via disentangled representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4710–4719

  4. Fan C, Peng Y, Cao C, Liu X, Hou S, Chi J, Huang Y, Li Q, He Z (2020) Gaitpart: temporal part–based model for gait recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14225–14233

  5. Lin B, Zhang S, Bao F (2020) Gait recognition with multipletemporal- scale 3d convolutional neural network. In: Proceedings of the 28th ACM international conference on multimedia, pp 3054–3062

  6. Chao H, He Y, Zhang J, Feng J (2019) Gaitset: regarding gait as a set for cross-view gait recognition. Proceedings of the AAAI Conference on Artificial Intelligence 33:8126–8133

  7. Hou S, Cao C, Liu X, Huang Y (2020) In: European conference on computer vision, pp 382–398

  8. Hou S, Liu X, Cao C, Huang Y (2021) Set residual network for silhouette-based gait recognition. IEEE Transactions on Biometrics, Behavior, and Identity Science 3(3):384–393

    Article  Google Scholar 

  9. Sepas-Moghaddam A, Etemad A (2020) View-invariant gait recognition with attentive recurrent learning of partial representations. IEEE Transactions on Biometrics, Behavior, and Identity Science 3(1):124–137

    Article  Google Scholar 

  10. Zhang Y, Huang Y, Yu S, Wang L (2019) Cross-view gait recognition by discriminative feature learning. IEEE Trans Image Process 29:1001–1015

    Article  ADS  MathSciNet  Google Scholar 

  11. Lin B, Zhang S, Yu X (2021) Gait recognition via effective global–local feature representation and local temporal aggregation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 14648–14656

  12. Li X, Makihara Y, Xu C, Yagi Y, Ren M (2020) Gait recognition via semi–supervised disentangled representation learning to identity and covariate features. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13309–13319

  13. He Y, Zhang J, Shan H, Wang L (2018) Multi-task gans for viewspecific feature learning in gait recognition. IEEE Trans Inf Forensics Secur 14(1):102–113

    Article  Google Scholar 

  14. Han J, Bhanu B (2005) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(2):316–322

    Article  Google Scholar 

  15. Lin J, Gan C, Han S (2019) Tsm: temporal shift module for efficient video understanding. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7083–7093

  16. Wu Z, Huang Y, Wang L, Wang X, Tan T (2016) A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans Pattern Anal Mach Intell 39(2):209–226

    Article  PubMed  Google Scholar 

  17. Shiraga K, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2016) Geinet: View–invariant gait recognition using a convolutional neural network. In: 2016 International conference on biometrics (ICB), pp 1–8

  18. Xiao J, Yang H, Xie K, Zhu J, Zhang J (2021) Learning discriminative representation with global and fine-grained features for cross–view gait recognition. CAAI Transactions on Intelligence Technology

  19. Yu S, Chen H, Wang Q, Shen L, Huang Y (2017) Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing 239:81–93

    Article  Google Scholar 

  20. Hu B, Gao Y, Guan Y, Long Y, Lane N, Ploetz T (2018) Robust cross-view gait identification with evidence: a discriminant gait gan (diggan) approach on 10000 people

  21. Xu W (2021) Graph-optimized coupled discriminant projections for cross-view gait recognition. Appl Intell 51(11):8149–8161

    Article  Google Scholar 

  22. Han J, Bhanu B (2005) Individual recognition using gait energy image. IEEE Trans Pattern Anal Mach Intell 28(2):316–322

    Article  Google Scholar 

  23. Zhao L, Guo L, Zhang R, Xie X, Ye X (2022) Mmgaitset: multimodal based gait recognition for countering carrying and clothing changes. Appl Intell 52(2):2023–2036

    Article  Google Scholar 

  24. Wolf T, Babaee M, Rigoll G (2016) Multi–view gait recognition using 3d convolutional neural networks. In: 2016 IEEE International conference on image processing (ICIP), pp 4165–4169

  25. Liu Y, Zeng Y, Pu J, Shan H, He P, Zhang J (2021) Selfgait: A spatiotemporal representation learning method for self-supervised gait recognition. In: ICASSP 2021–2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2570–2574

  26. Wu H, Tian J, Fu Y, Li B, Li X (2020) Condition-aware comparison scheme for gait recognition. IEEE Trans Image Process 30:2734–2744

    Article  ADS  Google Scholar 

  27. Wang Z, She Q, Smolic A (2021) Action-net: multipath excitation for action recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13214–13223

  28. Sudhakaran S, Escalera S, Lanz O (2020) Gate–shift networks for video action recognition. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 1102–1111

  29. Gao S-H, Cheng M-M, Zhao K, Zhang X-Y, Yang M-H, Torr P (2019) Res2net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell 43(2):652–662

  30. Zhao T, Wu X (2019) Pyramid feature attention network for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3085–3094

  31. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: 18th International conference on pattern recognition (ICPR’06), vol 4, pp 441–444

  32. Takemura N, Makihara Y, Muramatsu D, Echigo T, Yagi Y (2018) Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Transactions on Computer Vision and Applications 10(1):1–14

    Article  Google Scholar 

  33. Ye M, Shen J, Lin G, Xiang T, Shao L, Hoi SCH (2021) Deep learning for person re-identification: a survey and outlook. IEEE transactions on pattern analysis and machine intelligence 44(6):2872–2893

  34. Han Y, Huang G, Song S, Yang L, Wang H, Wang Y (2021) Dynamic neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(11):7436–7456

Download references

Acknowledgements

This work was partially supported by the STI 2030-Major Projects under grant 2022ZD0208900, the National Natural Science Foundation of China under grant 62076103, and the Special Innovation Projects of Colleges, Universities in Guangdong Province under grant 2022KTSCX035, and the Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation (“Climbing Program" Special Funds) under grant pdjh2022a0125.

Author information

Author notes

  1. Binyuan Huang and Chengju Zhou contributed equally to this work

  2. Binyuan Huang and Chengju Zhou are co-first authors.

Authors and Affiliations

  1. School of Software, South China Normal University, Guangzhou, 510631, China

    Binyuan Huang, Chengju Zhou, Lewei He & Jiahui Pan

  2. The Institute of Scientific and Industrial Research, Osaka University, Ibaraki, 5670047, Japan

    Chi Xu

  3. Pazhou Lab, Guangzhou, 510330, China

    Jiahui Pan

Authors
  1. Binyuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengju Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lewei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiahui Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chengju Zhou or Jiahui Pan.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, B., Zhou, C., He, L. et al. GaitCTCG: cross-view gait recognition via cascaded residual temporal shift and comprehensive multi-granularity learning. Appl Intell 54, 2428–2444 (2024). https://doi.org/10.1007/s10489-023-05241-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05241-9

Keywords

Search

Navigation