research-article

SymListener: Detecting Respiratory Symptoms via Acoustic Sensing in Driving Environments

Authors:

Zheng YangAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 19, Issue 1

Article No.: 3, Pages 1 - 21

https://doi.org/10.1145/3517014

Published: 14 January 2023 Publication History

Abstract

Sound-related respiratory symptoms are commonly observed in our daily lives. They are closely related to illnesses, infections, or allergies but ignored by the majority. Existing detection methods either depend on specific devices, which are inconvenient to wear, or are sensitive to noises and only work for indoor environment. Considering the lack of monitoring method for in-car environment, where there is high risk of spreading infectious diseases, we propose a smartphone-based system, named SymListener, to detect respiratory symptoms in driving environment. By continuously recording acoustic data through a built-in microphone, SymListener can detect the sounds of cough, sneeze, and sniffle. We design a modified ABSE-based method to remove the strong and changeable driving noises while saving energy of the smartphone. An LSTM network is adopted to classify the three types of symptoms according to the carefully designed acoustic features. We implement SymListener on different Android devices and evaluate its performance in real driving environment. The evaluation results show that SymListener can reliably detect target respiratory symptoms with an average accuracy of 92.19% and an average precision of 90.91%.

References

[1]

S. Sairam Akhil, N. Arun Vignesh, Sudharsan Jayabalan, E. Karthikeyan, Ayyem Pillai V., Ch. Usha Kumari, and Asisa Kumar Panigrahy. 2019. A novel approach for detection of the symptomatic patterns in the acoustic biological signal using truncation multiplier. In International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT’19). IEEE, 49–53.

[2]

Ali Azarbarzin and Zahra Moussavi. 2010. Unsupervised classification of respiratory sound signal into snore/no-snore classes. In International Conference of the IEEE Engineering in Medicine and Biology (EMBC’10). IEEE, 3666–3669.

[3]

Samantha J. Barry, Adrie D. Dane, Alyn H. Morice, and Anthony D. Walmsley. 2006. The automatic recognition and counting of cough. Cough 2, 8 (Sept.2006), 8–8. DOI:DOI:

[4]

Antony Barton, Patrick Gaydecki, Kimberley Holt, and Jaclyn A. Smith. 2012. Data reduction for cough studies using distribution of audio frequency content. Cough 8, 1 (Dec.2012), 12–12. DOI:DOI:

[5]

Soujanya Chatterjee, Md Mahbubur Rahman, Ebrahim Nemanti, and Jilong Kuang. 2019. WheezeD: Respiration phase based wheeze detection using acoustic data from pulmonary patients under attack. In EAI International Conference on Pervasive Computing Technologies for Healthcare - Demos and Posters (PervasiveHealth’19). EAI, 1–4.

[6]

Thomas Drugman, Jerome Urbain, Nathalie Bauwens, Ricardo Chessini, Carlos Valderrama, Patrick Lebecque, and Thierry Dutoit. 2013. Objective study of sensor relevance for automatic cough detection. IEEE J. Biomed. Health Inform. 17, 3 (May2013), 699–707. DOI:DOI:

[7]

W. T. Goldsmith, A. M. Mahmoud, J. S. Reynolds, W. G. McKinney, A. A. Afshari, A. A. Abaza, and D. G. Frazer. 2010. A system for recording high fidelity cough sound and airflow characteristics. Ann. Biom. Eng. 38, 2 (Feb.2010), 469–477. DOI:DOI:

[8]

Tian Hao, Guoliang Xing, and Gang Zhou. 2013. iSleep: Unobtrusive sleep quality monitoring using smartphones. In ACM Conference on Embedded Network Sensor Systems (SenSys’13). ACM, 1–14.

Digital Library

[9]

Healthgrades. 2020. Respiratory Symptoms. Retrieved from https://www.healthgrades.com/right-care/lungs-breathing-and-respiration/respiratory-symptoms.

[10]

Healthline. 2020. What causes sniffling and how to stop. Retrieved from https://www.healthline.com/health/sniffles.

[11]

J. Korpas, J. Sadlonova, and M. Vrabec. 1996. Analysis of the cough sound: An overview. Pulmon. Pharmacol. 9, 5 (Oct.1996), 261–268. DOI:DOI:

[12]

Steve S. Kraman, George R. Wodicka, Gary A. Pressler, and Hans Pasterkamp. 2006. Comparison of lung sound transducers using a bioacoustic transducer testing system. J. Appl. Physiol. 101, 2 (Aug.2006), 469–76. DOI:DOI:

[13]

Stefan T. Kulnik, Natalie M. Williams, Lalit Kalra, John Moxham, and Surinder S. Birring. 2016. Cough frequency monitors: Can they discriminate patient from environmental coughs? J. Thorac. Dis. 8, 11 (Nov.2016), 3152–3159. DOI:DOI:

[14]

Eric Larson, Tien Jui Lee, Sean Liu, Margaret Rosenfeld, and Shwetak Patel. 2011. Accurate and privacy preserving cough sensing using a low-cost microphone. In International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’11). ACM, New York, NY, 375–384.

Digital Library

[15]

Hanshang Li, Ting Li, Fan Li, Yue Wu, and Yu Wang. 2018. Cumulative participant selection with switch costs in large-scale mobile crowd sensing. In International Conference on Computer Communication and Networks (ICCCN’18). IEEE, 1095–2055.

[16]

Xin Li, Huaping Liu, Yu Zheng, and Bolin Xu. 2007. Robust speech endpoint detection based on improved adaptive band-partitioning spectral entropy. In International Conference on Life System Modeling and Simulation (LSMS’07). Springer, 36–45.

[17]

Chi Harold Liu, Zipeng Dai, Haoming Yang, and Jian Tang. 2020. Multi-task-oriented vehicular crowdsensing: A deep learning approach. In IEEE Conference on Computer Communications (INFOCOM’20). IEEE, 1123–1132.

Digital Library

[18]

Hong Lu, Wei Pan, Nicholas D. Lane, Tanzeem Choudhury, and Andrew T. Campbell. 2009. SoundSense: Scalable sound sensing for people-centric applications on mobile phones. In International Conference on Mobile Systems, Applications, and Services (MobiSys’09). ACM, 165–178.

Digital Library

[19]

Lyft. 2018. Safety policies. Retrieved from https://help.lyft.com/hc/en-us/articles/115012923127-Safety-policies#nosmoking.

[20]

Jesus Monge-Alvarez, Carlos Hoyos-Barcelo, Paul Lesso, and Pablo Casaseca de-la Higuera. 2019. Robust detection of audio-cough events using local Hu moments. IEEE J. Biomed. Health Inform. 23, 1 (Jan.2019), 184–196. DOI:DOI:

[21]

Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015. Contactless sleep apnea detection on smartphones. In International Conference on Mobile Systems, Applications, and Services (MobiSys’15). ACM, New York, NY, 45–57.

Digital Library

[22]

Ebrahim Nemati, Md Mahbubur Rahman, Viswam Nathan, and Jilong Kuang. 2018. Private audio-based cough sensing for in-home pulmonary assessment using mobile devices. In EAI International Conference on Body Area Networks (BODYNETS’20). EAI, 221–232.

[23]

USD of Transportation. 2019. Ratings. Retrieved from https://www.nhtsa.gov/ratings.

[24]

Jun Qi, Dong Wang, Yi Jiang, and Runsheng Liu. 2013. Auditory features based on Gammatone filters for robust speech recognition. In IEEE International Symposium on Circuits and Systems (ISCAS’13). IEEE, 305–308.

[25]

Kun Qian, Chenshu Wu, Fu Xiao, Yue Zheng, Yi Zhang, Zheng Yang, and Yunhao Liu. 2018. Acousticcardiogram: Monitoring heartbeats using acoustic signals on smart devices. In IEEE Conference on Computer Communications (INFOCOM’18). IEEE, 1574–1582.

Digital Library

[26]

Md Juber Rahman, Ebrahim Nemati, Mahbubur Rahman, and Korosh Vatanparvar. 2019. Efficient online cough detection with a minimal feature set using smartphones for automated assessment of pulmonary patients. In International Conference on Ambient Computing, Applications, Services and Technologies (AMBIENT’19). IARIA XPS Press, 1–7.

[27]

Shuai Ren, Yan Shi, Maolin Cai, and Weiqing Xu. 2018. Influence of airway secretion on airflow dynamics of mechanical ventilated respiratory system. IEEE/ACM Trans. Computat. Biol. Bioinform. 15, 5 (Sept.2018), 1660–1668. DOI:DOI:

Digital Library

[28]

Xiao Sun, Zongqing Lu, Wenjie Hu, and Guohong Cao. 2015. SymDetector: Detecting sound-related respiratory symptoms using smartphones. In International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’15). ACM, 97–108.

Digital Library

[29]

Zheng Sun, Aveek Purohit, Kathleen Yang, Neha Pattan, Dan Siewiorek, Asim Smailagic, Ian Lane, and Pei Zhang. 2011. CoughLoc: Location-aware indoor acoustic sensing for non-intrusive cough detection. In International Workshop on Emerging Mobile Sensing Technologies, Systems, and Applications (MobiSense’11). 1–7.

[30]

Keiichi Tokuda, Takao Kobayashi, Takashi Masuko, and Satoshi Imai. 1994. Mel generalized cepstral analysis—Unified approach to speech spectral estimation. In International Conference on Spoken Language Processing (ICSLP’94). INTERSPEECH, 1043–1046.

[31]

Korosh Vatanparvar, Viswam Nathan, Ebrahim Nemati, Md Mahbubur Rahman, and Jilong Kuang. 2019. A generative model for speech segmentation and obfuscation for remote health monitoring. In IEEE International Conference on Wearable and Implantable Body Sensor Networks (BSN’19). IEEE, 1–4.

[32]

Eldad Vizel, Mordechai Yigla, Yulia Goryachev, Eyal Dekel, Vered Felis, Hanna Levi, Isaac Kroin, Simon Godfrey, and Noam Gavriely. 2010. Validation of an ambulatory cough detection and counting application using voluntary cough under different conditions. Cough 6, 3 (May2010), 3–3. DOI:DOI:

[33]

Can Wang, Jianxin Peng, Lijuan Song, and Xiaowen Zhang. 2017. Automatic snoring sounds detection from sleep sounds via multi-features analysis. Australas. Phys. Eng. Sci. Med. 40, 1 (Mar.2017), 127–135. DOI:DOI:

[34]

Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, and Xingshe Zhou. 2017. C-FMCW based contactless respiration detection using acoustic signal. ACM Interact., Mob., Wear. Ubiq. Technol. 1, 4 (Dec.2017), 1–20. DOI:DOI:

Digital Library

[35]

Bing-Fei Wu and Kun-Ching Wang. 2005. Robust endpoint detection algorithm based on the adaptive band-partitioning spectral entropy in adverse environments. IEEE Trans. Speech Audio Process. 13, 5 (Sept.2005), 762–775. DOI:DOI:

[36]

Yue Wu, Fan Li, Liran Ma, Yadong Xie, Ting Li, and Yu Wang. 2019. A context-aware multi-armed bandit incentive mechanism for mobile crowd sensing systems. IEEE Internet Things J. 6, 5 (Oct.2019), 7648–7658. DOI:DOI:

[37]

Yadong Xie, Fan Li, Yue Wu, and Yu Wang. 2021. HearFit: Fitness monitoring on smart speakers via active acoustic sensing. In IEEE Conference on Computer Communications (INFOCOM’21). IEEE, 1–1.

Digital Library

[38]

Yadong Xie, Fan Li, Yue Wu, Song Yang, and Yu Wang. 2019. \(D^3\)-Guard: Acoustic-based drowsy driving detection using smartphones. In IEEE Conference on Computer Communications (INFOCOM’19). IEEE, 1225–1233.

Digital Library

[39]

Xiangyu Xu, Jiadi Yu, Yingying Chen, Yanmin Zhu, Linghe Kong, and Minglu Li. 2019. BreathListener: Fine-grained breathing monitoring in driving environments utilizing acoustic signals. In International Conference on Mobile Systems, Applications, and Services (MobiSys’19). ACM, New York, NY, 54–66.

Digital Library

Cited By

Zhang MLu LWu YYan ZSun JLin FRen K(2025) DroneAudioID: A Lightweight Acoustic Fingerprint-Based Drone Authentication System for Secure Drone Delivery IEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.352781420(1447-1461)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3527814
Li JTan YYang JLi ZYe HXia CLi Y(2024)Unsupervised Adversarial Example Detection of Vision Transformers for Trustworthy Edge ComputingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3674981Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1145/3674981
Wu HWang ZLi YLiu XLee T(2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3652518
Show More Cited By

Index Terms

SymListener: Detecting Respiratory Symptoms via Acoustic Sensing in Driving Environments
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing design and evaluation methods

Recommendations

Out-Clinic Pulmonary Disease Evaluation via Acoustic Sensing and Multi-Task Learning on Commodity Smartphones
SenSys '22: Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems

Pulmonary diseases, such as asthma and Chronic Obstructive Pulmonary Disease (COPD), constitute a major public health challenge. The disease symptoms, including airway obstruction and inflammation, usually result in changes in airway mechanical ...
Cross-Platform Support for Rapid Development of Mobile Acoustic Sensing Applications
MobiSys '18: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services

LibAS is a cross-platform framework to facilitate the rapid development of mobile acoustic sensing apps. It helps developers quickly realize their ideas by using a high-level Matlab script, and test them on various OS platforms, such as Android, iOS, ...
Poster Abstract: Acoustic Analysis System for Monitoring Respiratory Disease Symptoms
SenSys '24: Proceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems

The aim of this work was to develop a proprietary device for real-time cough detection in audio recordings. Using models such as CNN, ResNet-50, and MobileNet, the system classifies cough sounds, enabling the early detection of potential infection cases, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 19, Issue 1

February 2023

565 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3561987

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 14 January 2023

Online AM: 15 July 2022

Accepted: 02 February 2022

Revised: 11 December 2021

Received: 01 September 2021

Published in TOSN Volume 19, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China (NSFC)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
415
Total Downloads

Downloads (Last 12 months)118
Downloads (Last 6 weeks)11

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang MLu LWu YYan ZSun JLin FRen K(2025) DroneAudioID: A Lightweight Acoustic Fingerprint-Based Drone Authentication System for Secure Drone Delivery IEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.352781420(1447-1461)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3527814
Li JTan YYang JLi ZYe HXia CLi Y(2024)Unsupervised Adversarial Example Detection of Vision Transformers for Trustworthy Edge ComputingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3674981Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1145/3674981
Wu HWang ZLi YLiu XLee T(2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3652518
Suo YZheng ZWang XZhang BYang Y(2024)Jointly Harnessing Prior Structures and Temporal Consistency for Sign Language Video GenerationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364836820:6(1-18)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3648368
Su YZhang FNiu KWang TJin BWang ZJiang YZhang DQiu LXiong J(2024)Embracing Distributed Acoustic Sensing in Car Cabin for Children Presence DetectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435488:1(1-28)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643548
Shi PHu MShi XRen F(2024)Deep Modular Co-Attention Shifting Network for Multimodal Sentiment AnalysisACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363470620:4(1-23)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3634706
Ma ZWang SLuo XGu ZChen CLi JHua XLu G(2024)HARR: Learning Discriminative and High-Quality Hash Codes for Image RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/362716220:5(1-23)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3627162
Wu XFeng X(2024)Size Invariant Visual Cryptography Schemes With Evolving Threshold Access StructuresIEEE Transactions on Multimedia10.1109/TMM.2023.328257326(1488-1503)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3282573
Li ZHong FXue YLi QGuo Z(2024)SymRecorder: Detecting Respiratory Symptoms in Multiple Indoor Environments Using Earphone-MicrophonesWireless Sensor Networks10.1007/978-981-97-1010-2_22(301-315)Online publication date: 1-Mar-2024
https://doi.org/10.1007/978-981-97-1010-2_22
Wang HWang HGuo YLi Z(2023)A Novel Graph-level Anomaly Detection ModelProceedings of the 2023 7th International Conference on Electronic Information Technology and Computer Engineering10.1145/3650400.3650474(451-455)Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1145/3650400.3650474
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents