Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3664647.3681166acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article
Open access

Gait Recognition in Large-scale Free Environment via Single LiDAR

Published: 28 October 2024 Publication History

Abstract

Human gait recognition is crucial in multimedia, enabling identification through walking patterns without direct interaction, enhancing the integration across various media forms in real-world applications like smart homes, healthcare and non-intrusive security. LiDAR's ability to capture depth makes it pivotal for robotic perception and holds promise for real-world gait recognition. In this paper, based on a single LiDAR, we present the Hierarchical Multi-representation Feature Interaction Network (HMRNet) for robust gait recognition. Prevailing LiDAR-based gait datasets primarily derive from controlled settings with predefined trajectory, remaining a gap with real-world scenarios. To facilitate LiDAR-based gait recognition research, we introduce FreeGait, a comprehensive gait dataset from large-scale, unconstrained settings, enriched with multi-modal and varied 2D/3D data. Notably, our approach achieves state-of-the-art performance on prior dataset (SUSTech1K) and on FreeGait. https://4dvlab.github.io/project_page/FreeGait.html

References

[1]
Jeongho Ahn, Kazuto Nakashima, Koki Yoshino, Yumi Iwashita, and Ryo Kurazume. 2022. 2V-Gait: Gait Recognition using 3D LiDAR Robust to Changes in Walking Direction and Measurement Distance. In SII. IEEE, 602--607.
[2]
Csaba Benedek and etc. 2016. Lidar-based gait analysis and activity recognition in a 4d surveillance system. TCSVT, Vol. 28, 1 (2016), 101--113.
[3]
Francisco M Castro, Manuel J Marin-Jimenez, Nicolás Guil, and Nicolás Pérez de la Blanca. 2020. Multimodal feature fusion for CNN-based gait recognition: an empirical comparison. Neural Computing and Applications, Vol. 32 (2020), 14173--14193.
[4]
Hanqing Chao and etc. 2019. Gaitset: Regarding gait as a set for cross-view gait recognition. In AAAI, Vol. 33. 8126--8133.
[5]
Runnan Chen, Xinge Zhu, Nenglun Chen, Wei Li, Yuexin Ma, Ruigang Yang, and Wenping Wang. 2023. Bridging Language and Geometric Primitives for Zero-shot Point Cloud Segmentation. In Proceedings of the 31st ACM International Conference on Multimedia. 5380--5388.
[6]
Peishan Cong, Ziyi WangZhiyang Dou, Yiming Ren, Wei Yin, Kai Cheng, Yujing Sun, Xiaoxiao Long, Xinge Zhu, and Yuexin Ma. 2024. LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment. arXiv preprint arXiv:2403.13307 (2024).
[7]
Peishan Cong and etc. 2022. STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes. In CVPR. 19608--19617.
[8]
Yudi Dai, Yi Lin, and etc. 2022. HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR. CVPR (2022), 6782--6792.
[9]
Chao Fan and etc. 2023. OpenGait: Revisiting Gait Recognition Towards Better Practicality. In CVPR. 9707--9716.
[10]
Chao Fan, Yunjie Peng, and etc. 2020. Gaitpart: Temporal part-based model for gait recognition. In CVPR. 14225--14233.
[11]
Zheng Ge, Songtao Liu, Feng Wang, Zeming Li, and Jian Sun. 2021. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 (2021).
[12]
Martin Hofmann, Jürgen Geiger, and etc. 2014. The TUM Gait from Audio, Image and Depth (GAID) database: Multimodal recognition of subjects and traits. JVCIR, Vol. 25, 1 (2014), 195--206.
[13]
Saihui Hou, Chunshui Cao, and etc. 2020. Gait lateral network: Learning discriminative and compact representations for gait recognition. In ECCV. Springer, 382--398.
[14]
Jie Hu, Li Shen, and etc. 2018. Squeeze-and-excitation networks. In CVPR. 7132--7141.
[15]
Xiaohu Huang and etc. 2021. Context-sensitive temporal feature learning for gait recognition. In ICCV. 12909--12918.
[16]
Haruyuki Iwama, Mayu Okumura, Yasushi Makihara, and Yasushi Yagi. 2012. The ou-isir gait database comprising the large population dataset and performance evaluation of gait recognition. TIFS, Vol. 7, 5 (2012), 1511--1521.
[17]
Lingdong Kong and etc. 2023. Rethinking range view representation for lidar segmentation. arXiv preprint arXiv:2303.05367 (2023).
[18]
Kumar and etc. 2021. Gait recognition based on vision systems: A systematic survey. J. Vis. Commun. Image Represent., Vol. 75 (2021), 103052.
[19]
Weijia Li and etc. 2023. An In-Depth Exploration of Person Re-Identification and Gait Recognition in Cloth-Changing Conditions. In CVPR. 13824--13833.
[20]
Junhao Liang and etc. 2022. Gaitedge: Beyond plain end-to-end gait recognition for better practicality. In ECCV 2022. Springer, 375--390.
[21]
Rijun Liao and etc. 2020. A model-based gait recognition method with body pose and human prior knowledge. PR, Vol. 98 (2020), 107069.
[22]
Beibei Lin, Shunli Zhang, and Feng Bao. 2020. Gait recognition with multiple-temporal-scale 3d convolutional neural network. In Proceedings of the 28th ACM international conference on multimedia. 3054--3062.
[23]
Beibei Lin, Shunli Zhang, and etc. 2021. Gait recognition via effective global-local feature representation and local temporal aggregation. In ICCV. 14648--14656.
[24]
Guanze Liu, Yu Rong, and Lu Sheng. 2021. Votehmr: Occlusion-aware voting network for robust 3d human mesh recovery from partial point clouds. In Proceedings of the 29th ACM International Conference on Multimedia. 955--964.
[25]
Jiaming Liu, Yue Wu, Maoguo Gong, Qiguang Miao, Wenping Ma, and Cai Xu. 2023. Exploring Dual Representations in Large-Scale Point Clouds: A Simple Weakly Supervised Semantic Segmentation Framework. In Proceedings of the 31st ACM International Conference on Multimedia. 2371--2380.
[26]
Yuhang Lu, Qi Jiang, Runnan Chen, Yuenan Hou, Xinge Zhu, and Yuexin Ma. 2023. See more and know more: Zero-shot point cloud segmentation via multi-modal visual data. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 21674--21684.
[27]
Xu Ma, Can Qin, and etc. 2022. Rethinking network design and local geometry in point cloud: A simple residual mlp framework. arXiv preprint arXiv:2202.07123 (2022).
[28]
Yasushi Makihara. 2010. Silhouette transformation based on walking speed for gait identification. In CVPR.
[29]
Manuel J Marín-Jiménez, Francisco M Castro, Rubén Delgado-Esca no, Vicky Kalogeiton, and Nicolas Guil. 2021. UGaitNet: multimodal gait recognition with missing input modalities. IEEE Transactions on Information Forensics and Security, Vol. 16 (2021), 5452--5462.
[30]
Zhenhua Ning, Zhuotao Tian, Guangming Lu, and Wenjie Pei. 2023. Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancement. In Proceedings of the 31st ACM International Conference on Multimedia. 1895--1904.
[31]
Xidong Peng, Runnan Chen, Feng Qiao, Lingdong Kong, Youquan Liu, Tai Wang, Xinge Zhu, and Yuexin Ma. 2023. Sam-guided unsupervised domain adaptation for 3d segmentation. arXiv preprint arXiv:2310.08820 (2023).
[32]
Xidong Peng, Xinge Zhu, and Yuexin Ma. 2023. Cl3d: Unsupervised domain adaptation for cross-lidar 3d detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 2047--2055.
[33]
Yunjie Peng, Saihui Hou, Kang Ma, Yang Zhang, Yongzhen Huang, and Zhiqiang He. 2021. Learning Rich Features for Gait Recognition by Integrating Skeletons and Silhouettes. arXiv preprint arXiv:2110.13408 (2021).
[34]
Charles R Qi and etc. 2017. Pointnet: Deep learning on point sets for 3d classification and segmentation. In CVPR. 652--660.
[35]
Yiming Ren, Xiao Han, Yichen Yao, Xiaoxiao Long, Yujing Sun, and Yuexin Ma. 2024. LiveHPS: Robust and Coherent Motion Capture in Dynamic Free Environment. arXiv preprint arXiv:2407.09833 (2024).
[36]
Chuanfu Shen, Chao Fan, Wei Wu, Rui Wang, George Q Huang, and Shiqi Yu. 2023. Lidargait: Benchmarking 3d gait recognition with point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1054--1063.
[37]
Chunfeng Song and etc. 2022. CASIA-E: a large comprehensive dataset for gait recognition. TPAMI, Vol. 45, 3 (2022), 2801--2815.
[38]
Noriko Takemura, Yasushi Makihara, and etc. 2018. Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ TCVA, Vol. 10, 1 (2018), 1--14.
[39]
Daoliang Tan and etc. 2006. Efficient night gait recognition based on template matching. In ICPR, Vol. 3. IEEE, 1000--1003.
[40]
Torben Teepe and etc. 2021. Gaitgraph: Graph convolutional network for skeleton-based gait recognition. In ICIP. IEEE, 2314--2318.
[41]
Rahul Thour and etc. 2021. Gait Recognition System: A Survey. Journal of University of Shanghai for Science and Technology (2021).
[42]
Ashish Vaswani, Noam M. Shazeer, and etc. 2017. Attention is All you Need. ArXiv, Vol. abs/1706.03762 (2017).
[43]
Jingdong Wang and etc. 2020. Deep high-resolution representation learning for visual recognition. TPAMI, Vol. 43, 10 (2020), 3349--3364.
[44]
Jilong Wang, Saihui Hou, Yan Huang, Chunshui Cao, Xu Liu, Yongzhen Huang, and Liang Wang. 2023. Causal Intervention for Sparse-View Gait Recognition. In Proceedings of the 31st ACM International Conference on Multimedia. 77--85.
[45]
Liang Wang, Tieniu Tan, Huazhong Ning, and Weiming Hu. 2003. Silhouette analysis-based gait recognition for human identification. TPAMI, Vol. 25, 12 (2003), 1505--1518.
[46]
Yuanbin Wang, Shaofei Huang, Yulu Gao, Zhen Wang, Rui Wang, Kehua Sheng, Bo Zhang, and Si Liu. 2023. Transferring CLIP's Knowledge into Zero-Shot Point Cloud Semantic Segmentation. In Proceedings of the 31st ACM International Conference on Multimedia. 3745--3754.
[47]
Zengbin Wang, Saihui Hou, Man Zhang, Xu Liu, Chunshui Cao, Yongzhen Huang, and Shibiao Xu. 2023. LandmarkGait: Intrinsic Human Parsing for Gait Recognition. In Proceedings of the 31st ACM International Conference on Multimedia. 2305--2314.
[48]
Yiteng Xu, Peishan Cong, Yichen Yao, Runnan Chen, Yuenan Hou, Xinge Zhu, Xuming He, Jingyi Yu, and Yuexin Ma. 2023. Human-centric scene understanding for 3d large-scale scenarios. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 20349--20359.
[49]
Yiteng Xu, Kecheng Ye, Xiao Han, Yiming Ren, Xinge Zhu, and Yuexin Ma. 2024. A Unified Framework for Human-centric Point Cloud Video Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1155--1164.
[50]
Yichen Yao, Zimo Jiang, Yujing Sun, Zhencai Zhu, Xinge Zhu, Runnan Chen, and Yuexin Ma. 2024. HUNTER: Unsupervised Human-centric 3D Detection via Transferring Knowledge from Synthetic Instances to Real Scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 28120--28129.
[51]
Shiqi Yu and etc. 2013. A large RGB-D gait dataset and the baseline algorithm. In Biometric Recognition. Springer, 417--424.
[52]
Shiqi Yu, Daoliang Tan, and etc. 2006. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In ICPR, Vol. 4. IEEE, 441--444.
[53]
Weichen Yu, Hongyuan Yu, Yan Huang, and Liang Wang. 2022. Generalized inter-class loss for gait recognition. In Proceedings of the 30th ACM International Conference on Multimedia. 141--150.
[54]
Yifu Zhang, Peize Sun, and etc. 2022. Bytetrack: Multi-object tracking by associating every detection box. In ECCV. Springer.
[55]
Chengfeng Zhao, Yiming Ren, and etc. 2022. LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors. arXiv preprint arXiv:2205.15410 (2022).
[56]
Jinkai Zheng and etc. 2022. Gait Recognition in the Wild with Dense 3D Representations and A Benchmark. In CVPR. 20228--20237.
[57]
Jinkai Zheng, Xinchen Liu, Xiaoyan Gu, Yaoqi Sun, Chuang Gan, Jiyong Zhang, Wu Liu, and Chenggang Yan. 2022. Gait recognition in the wild with multi-hop temporal switch. In Proceedings of the 30th ACM International Conference on Multimedia. 6136--6145.
[58]
Zheng Zhu, Xianda Guo, and etc. 2021. Gait recognition in the wild: A benchmark. In ICCV. 14789--14799.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '24: Proceedings of the 32nd ACM International Conference on Multimedia
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2024

Check for updates

Author Tags

  1. gait recogntion
  2. human-centric multimedia understanding
  3. lidar point cloud

Qualifiers

  • Research-article

Funding Sources

Conference

MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia

Acceptance Rates

MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 194
    Total Downloads
  • Downloads (Last 12 months)194
  • Downloads (Last 6 weeks)107
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media