research-article

BreathListener: Fine-grained Breathing Monitoring in Driving Environments Utilizing Acoustic Signals

Authors:

Minglu LiAuthors Info & Claims

MobiSys '19: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services

Pages 54 - 66

https://doi.org/10.1145/3307334.3326074

Published: 12 June 2019 Publication History

Abstract

Given the increasing amount of time people spent on driving, the physical and mental health of drivers is essential to road safety. Breathing patterns are critical indicators of the well-being of drivers on the road. Existing studies on breathing monitoring require active user participation of wearing special sensors or relatively quiet environments during sleep, which are hardly applicable to noisy driving environments. In this work, we propose a fine-grained breathing monitoring system, BreathListener, which leverages audio devices on smartphones to estimate the fine-grained breathing waveform in driving environments. By investigating the data collected from real driving environments, we find that Energy Spectrum Density (ESD) of acoustic signals can be utilized to capture breathing procedures in driving environments. To extract breathing pattern in ESD signals, BreathListener eliminates interference from driving environments in ESD signals utilizing background subtraction and Ensemble Empirical Mode Decomposition (EEMD). After that, the extracted breathing pattern is transformed into Hilbert spectrum, and we further design a deep learning architecture based on Generative Adversarial Network (GAN) to generate fine-grained breathing waveform from the Hilbert spectrum of extracted breathing patterns in ESD signals. Experiments with 10 drivers in real driving environments show that BreathListener can accurately capture breathing patterns of drivers in driving environments.

References

[1]

Heba Abdelnasser, Khaled A Harras, and Moustafa Youssef. 2015. UbiBreathe: A ubiquitous non-invasive WiFi-based breathing estimator. In Proc. ACM MobiHoc'15, Hangzhou, China .

Digital Library

[2]

Fadel Adib, Hongzi Mao, et almbox. 2015. Smart homes that monitor breathing and heart rate. In Proc. ACM CHI'15, Seoul, Korea .

Digital Library

[3]

Heba Aly and Moustafa Youssef. 2016. Zephyr: Ubiquitous accurate multi-sensor fusion-based respiratory rate estimation using smartphones. In Proc. IEEE INFOCOM'16, San Francisco, CA, USA .

[4]

Yifan Chen and Predrag Rapajic. 2008. Human respiration rate estimation using ultra-wideband distributed cognitive radar system. International Journal of Automation and Computing, Vol. 5, 4 (2008), 325--333.

[5]

M Elliott et almbox. 2012. Critical care: the eight vital signs of patient monitoring. British Journal of Nursing, Vol. 21, 10 (2012), 621--625.

[6]

Jin Fei and Ioannis Pavlidis. 2006. Analysis of breathing air flow patterns in thermal imaging. In International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 946--952.

[7]

Patrick Flandrin, Gabriel Rilling, and Paulo Goncalves. 2004. Empirical mode decomposition as a filter bank. IEEE signal processing letters, Vol. 11, 2 (2004), 112--114.

[8]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, et almbox. 2014. Generative adversarial nets. In NIPS'14, Montreal, Canada .

Digital Library

[9]

John E Hall. 2010. Guyton and Hall textbook of medical physiology e-Book .Elsevier Health Sciences.

[10]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. IEEE CVPR'16, Las Vegas, USA, .

[11]

Jennifer Healey and Rosalind Picard. 2000. SmartCar: detecting driver stress. In Proc. IEEE ICPR'00, Barcelona, Spain .

Digital Library

[12]

David Hilbert. 1989. Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen. In Integralgleichungen und Gleichungen mit unendlich vielen Unbekannten. Springer, 8--171.

[13]

Norden E Huang et almbox. 1998. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of The Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 454, 1971 (1998), 903--995.

[14]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).

[15]

E Jonathan and Martin Leahy. 2010. Investigating a smartphone imaging unit for photoplethysmography. Physiological measurement, Vol. 31, 11 (2010), 70--79.

[16]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In NIPS'12, Lake Tahoe, USA .

Digital Library

[17]

Christian Ledig, Lucas Theis, et almbox. 2016. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 (2016).

[18]

Wei Li, Qi-chang He, Xiu-min Fan, and Zhi-min Fei. 2012. Evaluation of driver fatigue on two channels of EEG data. Neuroscience letters, Vol. 506, 2 (2012), 235--239.

[19]

Jian Liu, Yan Wang, et almbox. 2015. Tracking vital signs during sleep leveraging off-the-shelf wifi. In Proc. ACM MobiHoc'15, Hangzhou, China .

Digital Library

[20]

Andrew L Maas, Awni Y Hannun, et almbox. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proc. ICML'13, Atlanta, USA .

[21]

Wenguang Mao, Jian He, and Lili Qiu. 2016. CAT: high-precision acoustic motion tracking. In Proc. ACM MOBICOM'16, New York City, USA .

Digital Library

[22]

Wenguang Mao, Mei Wang, and Lili Qiu. 2018. AIM: Acoustic Imaging on a Mobile. In Proc. ACM Mobisys'18, Munich, Germany . 468--481.

Digital Library

[23]

Toshiyuki Matsuda et almbox. 2008. ECG monitoring of a car driver using capacitively-coupled electrodes. In Proc. IEEE EMBC'08, Vancouver, Canada .

[24]

Brian McKenzie and Melanie Rapino. 2011. Commuting in the united states, 2009 .US Department of Commerce, Economics and Statistics Administration, US Census Bureau Washington, DC.

[25]

J. P. Mortola. 2004. Breathing around the clock: an overview of the circadian pattern of respiration. European Journal of Applied Physiology, Vol. 91, 2--3 (2004), 119--129.

[26]

Rajalakshmi Nandakumar, Shyamnath Gollakota, et almbox. 2015. Contactless sleep apnea detection on smartphones. In Proc. ACM Mobisys'15, Florence, Italy .

Digital Library

[27]

Rajalakshmi Nandakumar, Shyamnath Gollakota, and Jacob E Sunshine. 2019. Opioid overdose detection using smartphones. Science translational medicine, Vol. 11, 474 (2019), eaau8914.

[28]

Rajalakshmi Nandakumar, Vikram Iyer, et almbox. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proc. ACM CHI'16, San Jose, USA .

Digital Library

[29]

G Pastor et almbox. 2006. Rear-view mirror use, driver alertness and road type: an empirical study using EEG measures. Transportation research part F: traffic psychology and behaviour, Vol. 9, 4 (2006), 286--297.

[30]

Neal Patwari, Lara Brewer, et almbox. 2014. Breathfinding: A wireless network that monitors and locates breathing in a home. IEEE Journal of Selected Topics in Signal Processing, Vol. 8, 1 (2014), 30--42.

[31]

Massimo Piccardi. 2004. Background subtraction techniques: a review. In Proc. IEEE SMC'04, The Hague, Netherlands .

[32]

Tauhidur Rahman, Alexander T Adams, Ruth Vinisha Ravichandran, Mi Zhang, Shwetak N Patel, Julie A Kientz, and Tanzeem Choudhury. 2015. Dopplesleep: A contactless unobtrusive sleep sensing system using short-range doppler radar. In Proc ACM Ubicomp'15, Osaka, Japan .

Digital Library

[33]

Theodore S Rappaport et almbox. 1996. Wireless communications: principles and practice. Vol. 2. prentice hall PTR New Jersey.

Digital Library

[34]

Yanzhi Ren, Chen Wang, Jie Yang, and Yingying Chen. 2015. Fine-grained sleep monitoring: Hearing your breathing with smartphones. In Proc. IEEE INFOCOM'15, Kowloon, Hong Kong .

[35]

Christopher G Scully, Jinseok Lee, Joseph Meyer, Alexander M Gorbach, Domhnull Granquist-Fraser, Yitzhak Mendelson, and Ki H Chon. 2012. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Transactions on Biomedical Engineering, Vol. 59, 2 (2012), 303--306.

[36]

NeuLog Logger Sensors. {n. d.}. NEULOG Respiration Monitor Logger Sensor. {Online}. Avaliable: http://www.neulog.com/.

[37]

Hugo Silva, André Lourencc o, and Ana Fred. 2012. In-vehicle driver recognition based on hand ECG signals. In Proc. IUI'12. ACM, 25--28.

Digital Library

[38]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[39]

Wei Wang, Alex X Liu, et almbox. 2016. Device-free gesture tracking using acoustic signals. In Proc. ACM MOBICOM'16, New York City, USA .

Digital Library

[40]

Yifan Wang, Federico Perazzi, Brian McWilliams, Alexander Sorkine-Hornung, Olga Sorkine-Hornung, and Christopher Schroers. 2018. A Fully Progressive Approach to Single-Image Super-Resolution. arXiv preprint arXiv:1804.02900 (2018).

[41]

Zhaohua Wu and Norden E Huang. 2004. A study of the characteristics of white noise using the empirical mode decomposition method. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, Vol. 460, 2046 (2004), 1597--1611.

[42]

Zhaohua Wu and Norden E. Huang. 2005. Ensemble Empirical Mode Decomposition a Noise-Assisted Data Analysis Method. Advances in Adaptive Data Analysis, Vol. 1, 01 (2005), 1--41.

[43]

Guosheng Yang, Yingzi Lin, and Prabir Bhattacharya. 2010. A driver fatigue recognition model based on information fusion and dynamic Bayesian network. Information Sciences, Vol. 180, 10 (2010), 1942--1954.

Digital Library

[44]

Sangki Yun et almbox. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proc. ACM Mobisys'17, Niagara Falls, USA .

Digital Library

[45]

Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a mobile device into a mouse in the air. In Proc. ACM Mobisys'15, Florence, Italy .

Digital Library

Cited By

Wang JZhang DZhang BChen JHu YChen Y(2024)RF-GymCare: Introducing Respiratory Prior for RF Sensing in Gym EnvironmentsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785688:3(1-28)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678568
Xie WXu CGong YWang YLiu YZhang JZhang QZheng ZYang S(2024)DeepBreath: Breathing Exercise Assessment with a Depth CameraProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785198:3(1-26)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678519
Yuan KIbrahim MSong YDeng GNerone RVijayan SGadre AKumar S(2024)ToMoBrush: Exploring Dental Health Sensing Using a Sonic ToothbrushProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785058:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678505
Show More Cited By

Index Terms

BreathListener: Fine-grained Breathing Monitoring in Driving Environments Utilizing Acoustic Signals
1. Networks
  1. Network types
    1. Cyber-physical networks

Recommendations

Wi-Tracker: Monitoring Breathing Airflow with Acoustic Signals
Wireless Algorithms, Systems, and Applications
Abstract
Continuous and accurate breathing monitoring during sleep plays a critical role in early warning and health diagnosis of diseases. To extract breathing patterns, there exist three categories of solutions, i.e., using camera to collect image data, ...
The Effects of Collision Avoidance Warning Systems on Driver’s Visual Behaviors
HCI in Mobility, Transport, and Automotive Systems. Automated Driving and In-Vehicle Experience Design
Abstract
This study aimed to test collision avoidance systems in on-road environments and analyze drivers’ visual behaviors caused by the warnings from such systems through an on-road field operational test. Most previous driving studies related to ...
A motion-based waveform for the detection of breathing difficulties during sleep

Individuals who suffer from different sleep breathing disorders suffer from a wide range of serious health problems. Unfortunately, the rate of diagnosis is very low, and the existing breathing monitoring techniques are expensive, uncomfortable and time- ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '19: Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services

June 2019

736 pages

ISBN:9781450366618

DOI:10.1145/3307334

General Chairs:
Junehwa Song
KAIST, South Korea
,
Minkyong Kim
Samsung Electronics
,
Program Chairs:
Nicholas D. Lane
University of Oxford & Samsung AI
,
Rajesh K. Balan
Singapore Management University

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Nature Science Foundation of China

Conference

MobiSys '19

Sponsor:

SIGMOBILE

MobiSys '19: The 17th Annual International Conference on Mobile Systems, Applications, and Services

June 17 - 21, 2019

Seoul, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

92
Total Citations
View Citations
1,726
Total Downloads

Downloads (Last 12 months)140
Downloads (Last 6 weeks)20

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang JZhang DZhang BChen JHu YChen Y(2024)RF-GymCare: Introducing Respiratory Prior for RF Sensing in Gym EnvironmentsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785688:3(1-28)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678568
Xie WXu CGong YWang YLiu YZhang JZhang QZheng ZYang S(2024)DeepBreath: Breathing Exercise Assessment with a Depth CameraProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785198:3(1-26)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678519
Yuan KIbrahim MSong YDeng GNerone RVijayan SGadre AKumar S(2024)ToMoBrush: Exploring Dental Health Sensing Using a Sonic ToothbrushProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785058:3(1-27)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678505
Chen YYu JChen YKong LZhu YChen YOkoshi TKo JLiKamWa R(2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661887
Xu CZheng XRen ZLiu LMa H(2024)UHeadProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435518:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643551
Wang SZhong LFu YChen LRen JZhang Y(2024)UFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435468:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643546
Chang ZZhang FXiong JChen WZhang DGanesan DShi W(2024)MSense: Boosting Wireless Sensing Capability Under Motion InterferenceProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649350(108-123)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3636534.3649350
Bai YShahid ITakawale HRoy N(2024)ScribeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314117:4(1-31)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631411
Bi HZhang WLi SChen YZhou CZhou T(2024)SmartSit: Sitting Posture Recognition Through Acoustic Sensing on SmartphonesIEEE Transactions on Multimedia10.1109/TMM.2024.337576126(8119-8130)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3375761
Cao HLiu DJiang HLuo J(2024)MagSign: Harnessing Dynamic Magnetism for User Authentication on IoT DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2022.321685123:1(597-611)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3216851
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents