Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Spatial-Temporal Graph Attention Network for Video-Based Gait Recognition

  • Conference paper
  • First Online:
Pattern Recognition (ACPR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12047))

Included in the following conference series:

  • 1622 Accesses

Abstract

Gait is a kind of attractive feature for human identification at a distance. It can be regarded as a kind of temporal signal. At the same time the human body shape can be regarded as the signal in the spatial domain. In the proposed method, we try to extract discriminative feature from video sequences in the spatial and temporal domains by only one network, Spatial-Temporal Graph Attention Network (STGAN). In spatial domain, we designed one branch to select some distinguished regions and enhance their contribution. It can make the network focus on these distinguished regions. We also constructed another branch, a Spatial-Temporal Graph (STG), to discover the relationship between frames and the variation of a region in temporal domain. The proposed method can extract gait feature in the two domains, and the two branches in the model can be trained end to end. The experimental results on two popular datasets, CASIA-B and OU-ISIR Treadmill-B, show the proposed method can improve gait recognition obviously.

The work is partly supported by the Science Foundation of Shenzhen (Grant No. 20170504160426188).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. An, W., Liao, R., Yu, S., Huang, Y., Yuen, P.C.: Improving gait recognition with 3D pose estimation. In: Zhou, J., et al. (eds.) CCBR 2018. LNCS, vol. 10996, pp. 137–147. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97909-0_15

    Chapter  Google Scholar 

  2. Ariyanto, G., Nixon, M.S.: Model-based 3D gait biometrics. In: International Joint Conference on Biometrics, pp. 1–7 (2011)

    Google Scholar 

  3. Bashir, K., Xiang, T., Gong, S.: Gait recognition using gait entropy image. In: International Conference on Crime Detection and Prevention, pp. 1–6 (2010)

    Google Scholar 

  4. Bruna, J., Zaremba, W., Szlam, A., Lecun, Y.: Spectral networks and locally connected networks on graphs. CoRR abs\(/\)1312.6203 (2013)

    Google Scholar 

  5. Chao, H., He, Y., Zhang, J., Feng, J.: GaitSet: regarding gait as a set for cross-view gait recognition. In: AAAI (2019)

    Google Scholar 

  6. Du, T., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning spatiotemporal features with 3D convolutional networks. In: IEEE International Conference on Computer Vision, pp. 4489–4497 (2015)

    Google Scholar 

  7. Duvenaud, D., et al.: Convolutional networks on graphs for learning molecular fingerprints. In: International Conference on Neural Information Processing Systems, pp. 2224–2232 (2015)

    Google Scholar 

  8. Feng, Y., Li, Y., Luo, J.: Learning effective gait features using LSTM. In: International Conference on Pattern Recognition, pp. 325–330 (2017)

    Google Scholar 

  9. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61 (2009)

    Article  Google Scholar 

  10. He, Y., Zhang, J., Shan, H., Wang, L.: Multi-task gans for view-specific feature learning in gait recognition. IEEE Trans. Inform. Forensics Secur. 14(1), 102–113 (2018)

    Article  Google Scholar 

  11. Henaff, M., Bruna, J., Lecun, Y.: Deep convolutional networks on graph-structured data. arXiv:abs/1506.05163 (2015)

  12. Hu, M., Wang, Y., Zhang, Z., Little, J.J., Huang, D.: View-invariant discriminative projection for multi-view gait-based human identification. IEEE Trans. Inform. Forensics Secur. 8(12), 2034–2045 (2013)

    Article  Google Scholar 

  13. Han, J., Bir, B.: Individual recognition using gait energy image. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 316 (2006)

    Article  Google Scholar 

  14. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (2016)

    Google Scholar 

  15. Kusakunniran, W., Wu, Q., Zhang, J., Li, H.: Support vector regression for multi-view gait recognition based on local motion feature selection. In: Computer Vision and Pattern Recognition (2010)

    Google Scholar 

  16. Kusakunniran, W., Qiang, W., Zhang, J., Li, H., Wang, L.: Recognizing gaits across views through correlated motion co-clustering. IEEE Trans. Image Process. 23(2), 696–709 (2014)

    Article  MathSciNet  Google Scholar 

  17. Li, R., Tapaswi, M., Liao, R., Jia, J., Urtasun, R., Fidler, S.: Situation recognition with graph neural networks. In: IEEE International Conference on Computer Vision (ICCV), pp. 4183–4192 (2017)

    Google Scholar 

  18. Li, X., Makihara, Y., Xu, C., Muramatsu, D., Yagi, Y., Ren, M.: Gait energy response function for clothing-invariant gait recognition. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10112, pp. 257–272. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54184-6_16

    Chapter  Google Scholar 

  19. Liao, R., Cao, C., Garcia, E.B., Yu, S., Huang, Y.: Pose-based temporal-spatial network (PTSN) for gait recognition with carrying and clothing variations. In: Zhou, J., et al. (eds.) CCBR 2017. LNCS, vol. 10568, pp. 474–483. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69923-3_51

    Chapter  Google Scholar 

  20. Makihara, Y., et al.: The OU-ISIR gait database comprising the treadmill dataset. IPSJ Trans. Comput. Vis. Appl. 4, 53–62 (2012)

    Article  Google Scholar 

  21. Makihara, Y., Sagawa, R., Mukaigawa, Y., Echigo, T., Yagi, Y.: Gait recognition using a view transformation model in the frequency domain. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 151–163. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_12

    Chapter  Google Scholar 

  22. Niepert, M., Ahmed, M., Kutzkov, K.: Learning convolutional neural networks for graphs. In: the 33rd International Conference on Machine Learning, pp. 2014–2023 (2016)

    Google Scholar 

  23. Shiraga, K., Makihara, Y., Muramatsu, D., Echigo, T., Yagi, Y.: GEINet: view-invariant gait recognition using a convolutional neural network. In: International Conference on Biometrics, pp. 1–8 (2016)

    Google Scholar 

  24. Sun, Y., Liang, Z., Yi, Y., Qi, T., Wang, S.: Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline). In: 15th European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  25. Urtasun, R., Fua, P.: 3D tracking for gait characterization and recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 17–22 (2004)

    Google Scholar 

  26. Vaswani, A., et al.: Attention is all you need. In: The 31st Conference on Neural Information Processing Systems (2017)

    Google Scholar 

  27. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018)

    Google Scholar 

  28. Wang, C., Zhang, J., Pu, J., Yuan, X., Wang, L.: Chrono-gait image: a novel temporal template for gait recognition. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 257–270. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_19

    Chapter  Google Scholar 

  29. Wang, X. Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR (2018)

    Google Scholar 

  30. Wu, Z., Huang, Y., Wang, L., Wang, X., Tan, T.: A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Trans. Pattern Anal. Mach. Intell. 39(2), 209–226 (2017)

    Article  Google Scholar 

  31. Xing, X., Wang, K., Yan, T., Lv, Z.: Complete canonical correlation analysis with application to multi-view gait recognition. Pattern Recogn. 50(C), 107–117 (2016)

    Article  Google Scholar 

  32. Yan, S., Xiong, Y., Lin, D.: Spatial temporal graph convolutional networks for skeleton-based action recognition. In: AAAI (2018)

    Google Scholar 

  33. Yu, S., Chen, H., Wang, Q., Shen, L., Huang, Y.: Invariant feature extraction for gait recognition using only one uniform model. Neurocomputing 239(C), 81–93 (2017)

    Article  Google Scholar 

  34. Yu, S., et al.: GaitGANv2: Invariant gait feature extraction using generative adversarial networks. Pattern Recogn. 87, 179–189 (2019)

    Article  Google Scholar 

  35. Yu, S., Tan, D., Tan, T.: A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: International Conference on Pattern Recognition, pp. 441–444 (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

    Xinhui Wu & Weizhi An

  2. Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

    Shiqi Yu

  3. Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China

    Shiqi Yu

  4. Watrix Technology Limited Co., Ltd., Beijing, China

    Weiyu Guo

  5. Advanced Technologies Application Center (CENATAV), Havana, Cuba

    Edel B. García

Authors
  1. Xinhui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weizhi An
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiqi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiyu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Edel B. García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xinhui Wu , Weizhi An , Shiqi Yu , Weiyu Guo or Edel B. García .

Editor information

Editors and Affiliations

  1. University of Malaya, Kuala Lumpur, Malaysia

    Shivakumara Palaiahnakote

  2. Consiglio Nazionale delle Ricerche, ICAR, Naples, Italy

    Gabriella Sanniti di Baja

  3. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  4. Auckland University of Technology, Auckland, New Zealand

    Wei Qi Yan

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, X., An, W., Yu, S., Guo, W., García, E.B. (2020). Spatial-Temporal Graph Attention Network for Video-Based Gait Recognition. In: Palaiahnakote, S., Sanniti di Baja, G., Wang, L., Yan, W. (eds) Pattern Recognition. ACPR 2019. Lecture Notes in Computer Science(), vol 12047. Springer, Cham. https://doi.org/10.1007/978-3-030-41299-9_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-41299-9_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-41298-2

  • Online ISBN: 978-3-030-41299-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation