Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Recognizing Gaits Across Walking and Running Speeds

Published: 04 March 2022 Publication History
  • Get Citation Alerts
  • Abstract

    For decades, very few methods were proposed for cross-mode (i.e., walking vs. running) gait recognition. Thus, it remains largely unexplored regarding how to recognize persons by the way they walk and run. Existing cross-mode methods handle the walking-versus-running problem in two ways, either by exploring the generic mapping relation between walking and running modes or by extracting gait features which are non-/less vulnerable to the changes across these two modes. However, for the first approach, a mapping relation fit for one person may not be applicable to another person. There is no generic mapping relation given that walking and running are two highly self-related motions. The second approach does not give more attention to the disparity between walking and running modes, since mode labels are not involved in their feature learning processes. Distinct from these existing cross-mode methods, in our method, mode labels are used in the feature learning process, and a mode-invariant gait descriptor is hybridized for cross-mode gait recognition to handle this walking-versus-running problem. Further research is organized in this article to investigate the disparity between walking and running. Running is different from walking not only in the speed variances but also, more significantly, in prominent gesture/motion changes. According to these rationales, in our proposed method, we give more attention to the differences between walking and running modes, and a robust gait descriptor is developed to hybridize the mode-invariant spatial and temporal features. Two multi-task learning-based networks are proposed in this method to explore these mode-invariant features. Spatial features describe the body parts non-/less affected by mode changes, and temporal features depict the instinct motion relation of each person. Mode labels are also adopted in the training phase to guide the network to give more attention to the disparity across walking and running modes. In addition, relevant experiments on OU-ISIR Treadmill Dataset A have affirmed the effectiveness and feasibility of the proposed method. A state-of-the-art result can be achieved by our proposed method on this dataset.

    References

    [1]
    Nick Carey. 2005. Establishing Pedestrian Walking Speeds. https://www.westernite.org/datacollectionfund/2005/psu_ped_summary.pdf.
    [2]
    TranSafety, Inc. 1997. Study Compares Older and Younger Pedestrian Walking Speeds. https://en.wikipedia.org/wiki/Walking.
    [3]
    Hanqing Chao, Yiwei He, Junping Zhang, and Jianfeng Feng. 2018. Gaitset: Regarding gait as a set for cross-view gait recognition. arXiv:1811.06186.
    [4]
    Xu Chi, Daisuke Adachi, Yasushi Makihara, Yasushi Yagi, and Jianfeng Lu. [n.d.]. Deformgait: Gait recognition under posture changes using deformation patterns between gait feature pairs. In IEEE International Joint Conference on Biometrics (IJCB’20), Houston. IEEE, 1–10.
    [5]
    Chao Fan, Yunjie Peng, Chunshui Cao, Xu Liu, Saihui Hou, Jiannan Chi, Yongzhen Huang, Qing Li, and Zhiqiang He. 2020. Gaitpart: Temporal part-based model for gait recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle. IEEE, 14225–14233.
    [6]
    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul. IEEE, 6202–6211.
    [7]
    Yang Feng, Yuncheng Li, and Jiebo Luo. 2016. Learning effective gait features using LSTM. In 23rd International Conference on Pattern Recognition (ICPR’16), Cancun. IEEE, 325–330.
    [8]
    Yu Guan and Chang-Tsun Li. 2013. A robust speed-invariant gait recognition system for walker and runner identification. In International Conference on Biometrics (ICB’13), Madrid. IEEE, 1–8.
    [9]
    Ju Han and Bir Bhanu. 2005. Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 2 (2005), 316–322.
    [10]
    Alexander Hermans, Lucas Beyer, and Bastian Leibe. 2017. In defense of the triplet loss for person re-identification. arXiv:1703.07737.
    [11]
    Huaxi Huang, Junjie Zhang, Jian Zhang, Qiang Wu, and Jingsong Xu. 2019. Compare more nuanced: Pairwise alignment bilinear network for few-shot fine-grained learning. arXiv:1904.03580.
    [12]
    Yan Huang, Qiang Wu, Jingsong Xu, and Yi Zhong. 2019. SBSGAN: Suppression of inter-domain background shift for person re-identification. In IEEE International Conference on Computer Vision (ICCV’19), Seoul. IEEE, 9526–9535.
    [13]
    Yan Huang, Jingsong Xu, Qiang Wu, Zhedong Zheng, Zhaoxiang Zhang, and Jian Zhang. 2018. Multi-pseudo regularized label for generated data in person re-identification. IEEE Transactions on Image Processing 28, 3 (2018), 1391–1403.
    [14]
    Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980.
    [15]
    Xiang Li, Yasushi Makihara, Chi Xu, Yasushi Yagi, and Mingwu Ren. 2020. Gait recognition via semi-supervised disentangled representation learning to identity and covariate features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle. IEEE, 13309–13319.
    [16]
    Zongyi Liu and Sudeep Sarkar. 2006. Improved gait recognition by gait dynamics normalization. IEEE Transactions on Pattern Analysis and Machine Intelligence6 (2006), 863–876.
    [17]
    Yasushi Makihara, Yusuke Fujihara, and Yasushi Yagi. 2012. Can gait fluctuations improve gait recognition?. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR’12), Tsukuba. IEEE, 3276–3279.
    [18]
    Yasushi Makihara, Hidetoshi Mannami, Akira Tsuji, Md Altab Hossain, Kazushige Sugiura, Atsushi Mori, and Yasushi Yagi. 2012. The OU-ISIR gait database comprising the treadmill dataset. IPSJ Transactions on Computer Vision and Applications 4 (2012), 53–62.
    [19]
    Yasushi Makihara, Darko S. Matovski, Mark S. Nixon, John N. Carter, and Yasushi Yagi. 1999. Gait recognition: Databases, representations, and applications. Wiley Encyclopedia of Electrical and Electronics Engineering (1999), 1–15.
    [20]
    Al Mansur, Yasushi Makihara, Daigo Muramatsu, and Yasushi Yagi. 2014. Cross-view gait recognition using view-dependent discriminative analysis. In IEEE International Joint Conference on Biometrics, Clearwater. IEEE, 1–8.
    [21]
    Kohei Shiraga, Yasushi Makihara, Daigo Muramatsu, Tomio Echigo, and Yasushi Yagi. 2016. Geinet: View-invariant gait recognition using a convolutional neural network. In International Conference on Biometrics (ICB’16), Halmstad. IEEE, 1–8.
    [22]
    Karen Simonyan and Andrew Zisserman. 2014. Two-stream convolutional networks for action recognition in videos. arXiv:1406.2199.
    [23]
    Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. 2019. Deep high-resolution representation learning for human pose estimation. arXiv:1902.09212.
    [24]
    Aravind Sundaresan, Amit Roy Chowdhury, and Rama Chellappa. 2003. A hidden Markov model based framework for recognition of humans from gait sequences. In Proceedings of the 2003 International Conference on Image Processing (Cat. No. 03CH37429), Vol. 2, Barcelona. IEEE, II–93.
    [25]
    Noriko Takemura, Yasushi Makihara, Daigo Muramatsu, Tomio Echigo, and Yasushi Yagi. 2017. On input/output architectures for convolutional neural network-based cross-view gait recognition. IEEE Transactions on Circuits and Systems for Video Technology 29 (2019), 2708–2719.
    [26]
    Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. 2018. A closer look at spatiotemporal convolutions for action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6450–6459.
    [27]
    Heng Wang, Alexander Kläser, Cordelia Schmid, and Cheng-Lin Liu. 2011. Action recognition by dense trajectories. In CVPR’11, Colorado Springs. IEEE, 3169–3176.
    [28]
    Yitong Wang, Dihong Gong, Zheng Zhou, Xing Ji, Hao Wang, Zhifeng Li, Wei Liu, and Tong Zhang. 2018. Orthogonal deep features decomposition for age-invariant face recognition. In Proceedings of the European Conference on Computer Vision (ECCV’18), Munich. 738–753.
    [29]
    David A. Winter. 2009. Biomechanics and Motor Control of Human Movement. John Wiley & Sons.
    [30]
    Thomas Wolf, Mohammadreza Babaee, and Gerhard Rigoll. 2016. Multi-view gait recognition using 3D convolutional neural networks. In IEEE International Conference on Image Processing (ICIP’16), Phoenix. IEEE, 4165–4169.
    [31]
    Zifeng Wu, Yongzhen Huang, and Liang Wang. 2015. Learning representative deep features for image set analysis. IEEE Transactions on Multimedia 17, 11 (2015), 1960–1968.
    [32]
    Zifeng Wu, Yongzhen Huang, Liang Wang, Xiaogang Wang, and Tieniu Tan. 2016. A comprehensive study on cross-view gait based human identification with deep cnns. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 2 (2016), 209–226.
    [33]
    Chi Xu, Yasushi Makihara, Xiang Li, Yasushi Yagi, and Jianfeng Lu. 2019. Speed-invariant gait recognition using single-support gait energy image. Multimedia Tools and Applications (2019), 1–28.
    [34]
    Chi Xu, Yasushi Makihara, Xiang Li, Yasushi Yagi, and Jianfeng Lu. 2020. Cross-view gait recognition using pairwise spatial transformer networks. IEEE Transactions on Circuits and Systems for Video Technology 31 (2021), 260–274.
    [35]
    ChewYean Yam, Mark S. Nixon, and John N. Carter. 2001. Extended model-based automatic gait recognition of walking and running. In International Conference on Audio-and Video-based Biometric Person Authentication, Halmstad. Springer, 278–283.
    [36]
    ChewYean Yam, Mark S Nixon, and John N Carter. 2002. Gait recognition by walking and running: A model-based approach.
    [37]
    ChewYean Yam, Mark S. Nixon, and John N. Carter. 2002. On the relationship of human walking and running: Automatic person identification by gait. In Object Recognition Supported by User Interaction for Service Robots, Vol. 1. IEEE, 287–290.
    [38]
    Ke Yang, Yong Dou, Shaohe Lv, Fei Zhang, and Qi Lv. 2016. Relative distance features for gait recognition with Kinect. Journal of Visual Communication and Image Representation 39 (2016), 209–217.
    [39]
    Lingxiang Yao, Worapan Kusakunniran, Qiang Wu, Jian Zhang, and Zhenmin Tang. 2017. Robust gait recognition under unconstrained environments using hybrid descriptions. In International Conference on Digital Image Computing: Techniques and Applications (DICTA’17), Sydney. IEEE, 1–7.
    [40]
    Lingxiang Yao, Worapan Kusakunniran, Qiang Wu, Jian Zhang, and Zhenmin Tang. 2018. Robust CNN-based gait verification and identification using skeleton gait energy image. In Digital Image Computing: Techniques and Applications (DICTA’18), Canberra. IEEE, 1–7.
    [41]
    Lingxiang Yao, Worapan Kusakunniran, Qiang Wu, Jian Zhang, Zhenmin Tang, and Wankou Yang. 2021. Robust gait recognition using hybrid descriptors based on skeleton gait energy image. Pattern Recognition Letters 150 (2021), 289–296.
    [42]
    Cheng Zhang, Wu Liu, Huadong Ma, and Huiyuan Fu. 2016. Siamese neural network based gait recognition for human identification. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’16), Shanghai. IEEE, 2832–2836.
    [43]
    Peng Zhang, Qiang Wu, and Jingsong Xu. 2019. VT-GAN: View transformation GAN for gait recognition across views. In International Joint Conference on Neural Networks (IJCNN’1), Budapest. IEEE, 1–8.
    [44]
    Yuqi Zhang, Yongzhen Huang, Shiqi Yu, and Liang Wang. 2019. Cross-view gait recognition by discriminative feature learning. IEEE Transactions on Image Processing 29 (2019), 1001–1015.

    Cited By

    View all
    • (2024)Self-Supervised Monocular Depth Estimation via Binocular Geometric Correlation LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366357020:8(1-19)Online publication date: 13-Jun-2024

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 18, Issue 3
    August 2022
    478 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/3505208
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 March 2022
    Accepted: 01 September 2021
    Revised: 01 August 2021
    Received: 01 February 2021
    Published in TOMM Volume 18, Issue 3

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Cross-mode gait recognition
    2. walking mode
    3. running mode

    Qualifiers

    • Research-article
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)83
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 11 Aug 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Self-Supervised Monocular Depth Estimation via Binocular Geometric Correlation LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366357020:8(1-19)Online publication date: 13-Jun-2024

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media