research-article

We Hear Your PACE: Passive Acoustic Localization of Multiple Walking Persons

Authors:

Jun LuoAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 2

Article No.: 55, Pages 1 - 24

https://doi.org/10.1145/3463510

Published: 24 June 2021 Publication History

Abstract

Indoor localization is crucial to enable context-aware applications, but existing solutions mostly require a user to carry a device, so as to actively sense location-discriminating signals. However, many applications do not prefer user involvement due to, e.g., the cumbersome of carrying a device. Therefore, solutions that track user locations passively can be desirable, yet lack of active user involvement has made passive indoor localization very challenging even for a single person. To this end, we propose Passive Acoustic loCalization of multiple walking pErsons (PACE) as a solution for small-scale indoor scenarios: it passively locates users by pinpointing the positions of their footsteps. In particular, PACE leverages both structure-borne and air-borne footstep impact sounds (FIS); it uses structure-borne FIS for range estimations exploiting their acoustic dispersion nature, and it employs air-borne FIS for Angle-of-Arrival (AoA) estimations and person identifications. To combat the low-SNR nature of FIS, PACE innovatively employs domain adversarial adaptation and spectral weighting to ranging/identification and AoA estimations, respectively. We implement a PACE prototype and extensively evaluate its performance in representative environments. The results demonstrate a promising sub-meter localization accuracy with a median error of 30 cm.

Supplementary Material

cai (cai.zip)

Supplemental movie, appendix, image and software files for, We Hear Your PACE: Passive Acoustic Localization of Multiple Walking Persons

Download
118.72 KB

References

[1]

Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C. Miller. 2014. 3D Tracking via Body Radio Reflections. In Proc. of the 11st USENIX NSDI. 317--329.

[2]

X. Anguera, C. Wooters, and J. Hernando. 2007. Acoustic Beamforming for Speaker Diarization of Meetings. IEEE Transactions on Audio, Speech, and Language Processing 15, 7 (2007), 2011--2022.

Digital Library

[3]

Arm. 2020. Neon Technology. https://developer.arm.com/architectures/instruction-sets/simd-isas/neon.

[4]

P. Bahl and V.N. Padmanabhan. 2000. RADAR: An In-Building RF-based User Location and Tracking System. In Proc. of the 19th IEEE INFOCOM. 775--784.

[5]

M. Berouti, R. Schwartz, and J. Makhoul. 1979. Enhancement of Speech Corrupted by Acoustic Noise. In ICASSP '79. IEEE International Conference on Acoustics, Speech, and Signal Processing. 208--211.

[6]

Chao Cai, Ruinan Jin, Peng Wang, Liyuan Ye, Hongbo Jiang, and Jun Luo. 2021. PURE: Passive mUlti-peRson idEntification via Deep Footstep Separation and Recognition. https://arxiv.org/abs/2104.07177.

[7]

NDT Research Center. 2020. Temperature and the Speed of Sound. https://www.nde-ed.org/EducationResources/HighSchool/Sound/tempandspeed.htm. Accessed: 2020-07-21.

[8]

Jagmohan Chauhan, Yining Hu, Suranga Seneviratne, Archan Misra, Aruna Seneviratne, and Youngki Lee. 2017. BreathPrint: Breathing Acoustics-based User Authentication. In Proc. of the 15th ACM MobiSys. 278--291.

Digital Library

[9]

J. C. Chen, Kung Yao, and R. E. Hudson. 2002. Source Localization and Beamforming. IEEE Signal Processing Magazine 19, 2 (2002), 30--39.

[10]

Krishna Chintalapudi, Anand Padmanabha Iyer, and Venkata N. Padmanabhan. 2010. Indoor Localization without the Pain. In Proc. of the 16th ACM MobiCom. 173--184.

[11]

P. Connor and A. Ross. 2018. Biometric Recognition by Gait: A Survey of Modalities and Features. Elsevier Computer Vision and Image Understanding 167 (2018), 1--27.

Digital Library

[12]

Y. Ephraim and D. Malah. 1985. Speech Enhancement Using a Minimum Mean-Square Error Log-Spectral Amplitude Estimator. IEEE Transactions on Acoustics, Speech, and Signal Processing 33, 2 (1985), 443--445.

[13]

ESPRESSIF. [n.d.]. ESP32 Wireless SoCs. https://www.espressif.com/en/products/socs/esp32/overview. Accessed: 2020-08-08.

[14]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. The Journal of Machine Learning Research (JMLR) 17, 1 (Jan. 2016), 2096--2030.

[15]

Y. Gao, J. Niu, R. Zhou, and G. Xing. 2013. ZiFind: Exploiting Cross-Technology Interference Signatures for Energy-Efficient Indoor Localization. In Proc. of the 32nd IEEE INFOCOM. 2940--2948.

[16]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.

Digital Library

[17]

Tobias Grosse-Puppendahl, Xavier Dellangnol, Christian Hatzfeld, Biying Fu, Mario Kupnik, Arjan Kuijper, Matthias R. Hastall, James Scott, and Marco Gruteser. 2016. Platypus: Indoor Localization and Identification through Sensing of Electric Potential Changes in Human Bodies. In Proc. of the 14th ACM MobiSys. 17--30.

Digital Library

[18]

A. Hyvarinen. 1999. Fast and Robust Fixed-point Algorithms for Independent Component Analysis. IEEE Transactions on Neural Networks 10, 3 (1999), 626--634.

Digital Library

[19]

Kiran Joshi, Dinesh Bharadia, Manikanta Kotaru, and Sachin Katti. 2015. WiDeo: Fine-grained Device-free Motion Tracing using RF Backscatter. In Proc. of the 12th USENIX NSDI. 189--204.

[20]

C. R. Karanam, B. Korany, and Y. Mostofi. 2019. Tracking from One Side: Multi-person Passive Tracking with WiFi Magnitude Measurements. In Proc. of the 18th ACM/IEEE IPSN. 181--192.

[21]

Hyosu Kim, Anish Byanjankar, Yunxin Liu, Yuanchao Shu, and Insik Shin. 2018. UbiTap: Leveraging Acoustic Dispersion for Ubiquitous Touch Interface on Solid Surfaces. In Proc. of the 16th ACM SenSys. 211--223.

Digital Library

[22]

C. Knapp and G. Carter. 1976. The Generalized Correlation Method for Estimation of Time Delay. IEEE Transactions on Acoustics, Speech, and Signal Processing 24, 4 (1976), 320--327.

[23]

Manikanta Kotaru, Kiran Joshi, Dinesh Bharadia, and Sachin Katti. 2015. SpotFi: Decimeter Level Localization Using WiFi. In Proc. of the 43th ACM SIGCOMM. 269--282.

Digital Library

[24]

Manikanta Kotaru, Pengyu Zhang, and Sachin Katti. 2017. Localizing Low-Power Backscatter Tags Using Commodity WiFi. In Proc. of the 13th ACM CoNEXT. 251--262.

Digital Library

[25]

Patrick Lazik, Niranjini Rajagopal, Oliver Shih, Bruno Sinopoli, and Anthony Rowe. 2015. ALPS: A Bluetooth and Ultrasound Platform for Mapping and Localization. In Proc. of the 13th ACM SenSys. 73--84.

Digital Library

[26]

Patrick Lazik and Anthony Rowe. 2012. Indoor Pseudo-ranging of Mobile Devices Using Ultrasonic Chirps. In Proc. of the 10th ACM SenSys. 391--404.

[27]

Xiang Li, Shengjie Li, Daqing Zhang, Jie Xiong, Yasha Wang, and Hong Mei. 2016. Dynamic-MUSIC: Accurate Device-Free Indoor Localization. In Proc. of ACM UbiComp. 196--207.

Digital Library

[28]

Hongbo Liu, Yu Gan, Jie Yang, Simon Sidhom, Yan Wang, Yingying Chen, and Fan Ye. 2012. Push the Limit of WiFi Based Localization for Smartphones. In Proc. of the 19th ACM MobiCom. 305--316.

Digital Library

[29]

Kaikai Liu, Xinxin Liu, and Xiaolin Li. 2013. Guoguo: Enabling Fine-grained Indoor Localization via Smartphone. In Proc. of the 11st ACM MobiSys. 235--248.

Digital Library

[30]

Wenguang Mao, Mei Wang, Wei Sun, Lili Qiu, Swadhin Pradhan, and Yi-Chao Chen. 2019. RNN-Based Room Scale Hand Motion Tracking. In Proc. of the 25th ACM MobiCom. 1--16.

Digital Library

[31]

MarketsandMarkets. [n.d.]. Indoor Location Market - Global Forecast to 2025. https://www.marketsandmarkets.com/Market-Reports/indoor-location-market-989.html. Accessed: 2020-08-08.

[32]

S. Nakagawa, L. Wang, and S. Ohtsuka. 2012. Speaker Identification and Verification by Combining MFCC and Phase Information. IEEE Transactions on Audio, Speech, and Language Processing 20, 4 (2012), 1085--1095.

Digital Library

[33]

Rajalakshmi Nandakumar, Alex Takakuwa, Tadayoshi Kohno, and Shyamnath Gollakota. 2017. CovertBand: Activity Information Leakage Using Music. Proc. of the 17th ACM UbiComp (2017), 1--24.

Digital Library

[34]

S. Narayana, V. Rao, R. V. Prasad, A. K. Kanthila, K. Managundi, L. Mottola, and T. V. Prabhakar. 2020. LOCI: Privacy-aware, Device-free, Low-power Localization of Multiple Persons using IR Sensors. In Proc. of the 19th ACM/IEEE IPSN. 121--132.

[35]

Shijia Pan, Tong Yu, Mostafa Mirshekari, Jonathon Fagert, Amelie Bonde, Ole J. Mengshoel, Hae Young Noh, and Pei Zhang. 2017. FootprintID: Indoor Pedestrian Identification through Ambient Structural Vibration Sensing. In Proc. of ACM UbiComp. 89:1-31.

[36]

Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. 2018. Multi-Adversarial Domain Adaptation. In Proc. of AAAI. 3934--3941.

[37]

J. Portelo, M. Bugalho, I. Trancoso, J. Neto, A. Abad, and A. Serralheiro. 2009. Non-speech Audio Event Detection. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing. 1973--1976.

[38]

Kun Qian, Chenshu Wu, Zheng Yang, Yunhao Liu, and Kyle Jamieson. 2017. Widar: Decimeter-Level Passive Tracking via Velocity Monitoring with Commodity Wi-Fi. In Proc. of the 18th ACM MobiCom.

Digital Library

[39]

Kun Qian, Chenshu Wu, Yi Zhang, Guidong Zhang, Zheng Yang, and Yunhao Liu. 2018. Widar2.0: Passive Human Tracking with a Single Wi-Fi Link. In Proc. of the 16th ACM MobiSys. 350--361.

Digital Library

[40]

B. Rafaely. 2005. Analysis and Design of Spherical Microphone Arrays. IEEE Transactions on Speech and Audio Processing 13, 1 (2005), 135--143.

[41]

RaspberryPi. [n.d.]. ReSpeaker 6-Mic Circular Array Kit for Raspberry Pi. https://wiki.seeedstudio.com/ReSpeaker_6-Mic_Circular_Array_kit_for_Raspberry_Pi/. Accessed: 2020-08-08.

[42]

Annie Ross and Germain Ostiguy. 2007. Propagation of the Initial Transient Noise From an Impacted Plate. Journal of Sound and Vibration - J SOUND VIB 301 (03 2007), 28--42.

[43]

Mikkel Schmidt and Rasmus Olsson. 2006. Single-Channel Speech Separation using Sparse Non-negative Matrix Factorization. In Proc. of the 9th ISCA INTERSPEECH. 1652--1658.

[44]

Rijurekha Sen, Youngki Lee, Kasthuri Jayarajah, Archan Misra, and Rajesh Krishna Balan. 2014. Grumon: Fast and Accurate Group Monitoring for Heterogeneous Urban Spaces. In Proc. of the 12th ACM SenSys. 46--60.

Digital Library

[45]

Sheng Shen, Daguan Chen, Yu-Lin Wei, Zhijian Yang, and Romit Roy Choudhury. 2020. Voice Localization Using Nearby Wall Reflections. In Proc. of the 26th ACM MobiCom. 1--14.

Digital Library

[46]

Y. Shu, C. Bo, G. Shen, C. Zhao, L. Li, and F. Zhao. 2015. Magicol: Indoor Localization Using Pervasive Magnetic Field and Opportunistic WiFi Sensing. IEEE Journal on Selected Areas in Communications 33, 7 (2015), 1443--1457.

Digital Library

[47]

Yuanchao Shu, Kang G. Shin, Tian He, and Jiming Chen. 2015. Last-Mile Navigation Using Smartphones. In Proc. of the 21st ACM MobiCom. 512--524.

Digital Library

[48]

Ke Sun, Ting Zhang, Wei Wang, and Lei Xie. 2018. VSkin: Sensing Touch Gestures on Surfaces of Mobile Devices Using Acoustic Signals. In Proc. of the 24th ACM MobiCom. 591--605.

Digital Library

[49]

TensorFlow. [n.d.]. An End-to-End Open Source Machine Learning Platform. https://www.tensorflow.org/. Accessed: 2020-08-08.

[50]

Laurens van der Maaten and Geoffrey Hinton. 2008. Viualizing Data Using t-SNE. Journal of Machine Learning Research 9 (11 2008), 2579--2605.

[51]

Deepak Vasisht, Swarun Kumar, and Dina Katabi. 2016. Decimeter-level Localization with a Single WiFi Access Point. In Proc. of the 13th USENIX NSDI. 165--178.

[52]

Anran Wang, Jacob E. Sunshine, and Shyamnath Gollakota. 2019. Contactless Infant Monitoring Using White Noise. In Proc. of the 25th ACM MobiCom. 1--16.

Digital Library

[53]

Weiguo Wang, Jinming Li, Yuan He, and Yunhao Liu. 2020. Symphony: Localizing Multiple Acoustic Sources with a Single Microphone Array. In Proc. of the 18th ACM SenSys. 82--94.

Digital Library

[54]

R. Want, A. Hopper, V. Falcao, and J. Gibbons. 1992. The Active Badge Location System. ACM Trans. on Information Systems 40, 1 (Jan. 1992), 91--102.

[55]

M. Wax, Tie-Jun Shan, and T. Kailath. 1984. Spatio-Temporal Apectral Analysis by Eigenstructure Methods. IEEE Transactions on Acoustics, Speech, and Signal Processing 32, 4 (1984), 817--827.

[56]

Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A Discriminative Feature Learning Approach for Deep Face Recognition. In Proc. of ECCV. 499--515.

[57]

Bo Xie, Guang Tan, and Tian He. 2015. SpinLight: A High Accuracy and Robust Light Positioning System for Indoor Applications. In Proc. of the 13th ACM Sensys. 211--223.

Digital Library

[58]

Moustafa Youssef and Ashok Agrawala. 2005. The Horus WLAN Location Determination System. In Proc. of the 3rd ACM MobiSys. 205--218.

Digital Library

[59]

Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proc. of the 15th ACM MobiSys. 15--28.

Digital Library

[60]

Chi Zhang, Feng Li, Jun Luo, and Ying He. 2014. iLocScan: Harnessing Multipath for Simultaneous Indoor Source Localization and Space Scanning. In Proc. of the 12th ACM SenSys. 91---104.

Digital Library

[61]

Chi Zhang, Kalyan P. Subbu, Jun Luo, and Jianxin Wu. 2015. GROPING: Geomagnetism and cROwdsensing Powered Indoor NaviGation. IEEE Transactions on Mobile Computing 14, 2 (2015), 387--400.

[62]

Shilin Zhu and Xinyu Zhang. 2017. Enabling High-Precision Visible Light Localization in Today's Buildings. In Proc. of the 15th ACM MobiSys. 96--108.

Digital Library

Cited By

Murakami HSasatani TSugimoto MSukeda IMita YKawahara YShu YLiu JTan RHe YChen J(2024)SyncEcho: Echo-Based Single Speaker Time Offset Estimation for Time-of-Flight LocalizationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699369(718-729)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699369
Li ZLuo WZhang YChen JShu YZhang YShu YLiu JTan RHe YChen J(2024)ASLiquid: Non-Intrusive Liquid Counterfeit Identification with Your EarphonesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699321(41-53)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699321
Wang PHu JLiu CLuo JLuo BLiao XXu JKirda ELie D(2024)RefleXnoop: Passwords Snooping on NLoS Laptops Leveraging Screen-Induced Sound ReflectionProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670341(3361-3375)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670341
Show More Cited By

Index Terms

We Hear Your PACE: Passive Acoustic Localization of Multiple Walking Persons
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

Suppression of NLOS errors in TDOA-AOA hybrid localization
Abstract
This article handles locating a non-cooperative emitter in cluttered outdoor environments, where many sensors are positioned to locate the emitter. For outdoor localization, this article utilizes Time Difference of Arrival (TDOA) and Angle of ...
Magnitude-based angle-of-arrival estimation, localization, and target tracking
IPSN '18: Proceedings of the 17th ACM/IEEE International Conference on Information Processing in Sensor Networks

In this paper, we are interested in estimating the angle of arrival (AoA) of all the signal paths arriving at a receiver array using only the corresponding received signal magnitude measurements (or, equivalently, the received power measurements). ...
Dynamic-MUSIC: accurate device-free indoor localization
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Device-free passive indoor localization is playing a critical role in many applications such as elderly care, intrusion detection, smart home, etc. However, existing device-free localization systems either suffer from labor-intensive offline training or ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 2

June 2021

932 pages

EISSN:2474-9567

DOI:10.1145/3472726

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 June 2021

Published in IMWUT Volume 5, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

AcRF Tier 1

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
681
Total Downloads

Downloads (Last 12 months)138
Downloads (Last 6 weeks)16

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Murakami HSasatani TSugimoto MSukeda IMita YKawahara YShu YLiu JTan RHe YChen J(2024)SyncEcho: Echo-Based Single Speaker Time Offset Estimation for Time-of-Flight LocalizationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699369(718-729)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699369
Li ZLuo WZhang YChen JShu YZhang YShu YLiu JTan RHe YChen J(2024)ASLiquid: Non-Intrusive Liquid Counterfeit Identification with Your EarphonesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699321(41-53)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699321
Wang PHu JLiu CLuo JLuo BLiao XXu JKirda ELie D(2024)RefleXnoop: Passwords Snooping on NLoS Laptops Leveraging Screen-Induced Sound ReflectionProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security10.1145/3658644.3670341(3361-3375)Online publication date: 2-Dec-2024
https://dl.acm.org/doi/10.1145/3658644.3670341
Zhang JWang J(2024)FusionTrack: Towards Accurate Device-free Acoustic Motion Tracking with Signal FusionACM Transactions on Sensor Networks10.1145/365466620:3(1-30)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.1145/3654666
Tian YWang YWang YTong XLiu XQu W(2024)Device-Free Human Tracking and Gait Recognition Based on the Smart SpeakerIEEE Transactions on Mobile Computing10.1109/TMC.2024.337964723:11(10610-10627)Online publication date: Nov-2024
https://doi.org/10.1109/TMC.2024.3379647
Sun YWang WMottola LZhang JWang RHe Y(2024)Indoor Drone Localization and Tracking Based on Acoustic Inertial MeasurementIEEE Transactions on Mobile Computing10.1109/TMC.2023.333586023:6(7537-7551)Online publication date: Jun-2024
https://doi.org/10.1109/TMC.2023.3335860
Xu XDeng RZhao GZhang BLiu C(2024)Deep Domain Generalization-Based Indoor Pedestrian Identification Using Footstep-Induced VibrationsIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.335122773(1-8)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3351227
Yang QCui KZheng Y(2024)Room-scale Voice Liveness Detection for Smart DevicesIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.3367269(1-14)Online publication date: 2024
https://doi.org/10.1109/TDSC.2024.3367269
Sun WWang MQiu L(2024)Spatial-Aware Multi-Task Learning Based Speech Separation2024 IEEE 21st International Conference on Mobile Ad-Hoc and Smart Systems (MASS)10.1109/MASS62177.2024.00024(100-108)Online publication date: 23-Sep-2024
https://doi.org/10.1109/MASS62177.2024.00024
Qu YShi LWang XWang Z(2024)Indoor Multiperson Detection and Recognition Through Footsteps: A Deep Learning Approach With Acoustic Signal AnalysisIEEE Sensors Journal10.1109/JSEN.2024.339421224:12(19482-19496)Online publication date: 15-Jun-2024
https://doi.org/10.1109/JSEN.2024.3394212
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents