research-article

HeartSense: Ubiquitous Accurate Multi-Modal Fusion-based Heart Rate Estimation Using Smartphones

Authors:

Moustafa YoussefAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 3

Article No.: 97, Pages 1 - 18

https://doi.org/10.1145/3132028

Published: 11 September 2017 Publication History

Abstract

Heart rate is one of the most important vital signals for personal health tracking. A number of smartphone-based heart rate estimation systems have been proposed over the years. However, they either depend on special hardware sensors or suffer from the high noise due to the weakness of the heart signals, affecting their accuracy in many practical scenarios.

Inspired by medical studies about the heart motion mechanics, we propose the HeartSense heart rate estimation system. Specifically, we show that the gyroscope sensor is the most sensitive sensor for measuring the heart rate. To further counter noise and handle different practical scenarios, we introduce a novel quality metric that allows us to fuse the different gyroscope axes in a probabilistic framework to achieve a robust and accurate estimate.

We have implemented and evaluated our system on different Android phones. Results using 836 experiments on different subjects in practical scenarios with a side-by-side comparison with other systems show that HeartSense can achieve 1.03 bpm median absolute error for heart rate estimation. This is better than the state-of-the-art by more than 147% in median error, highlighting HeartSense promise as a ubiquitous system for medical and personal well-being applications.

References

[1]

Ahmed H Abdelhafiz. 2002. Heart failure in older people: causes, diagnosis and treatment. Age and ageing 31, 1 (2002), 29--36.

[2]

Heba Abdelnasser, Khaled A Harras, and Moustafa Youssef. 2015. UbiBreathe: A ubiquitous non-invasive WiFi-based breathing estimator. In Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 277--286.

Digital Library

[3]

Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C Miller. 2015. Smart homes that monitor breathing and heart rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 837--846.

Digital Library

[4]

Heba Aly and Moustafa Youssef. 2016. Zephyr: Ubiquitous Accurate multi-Sensor Fusion-based Respiratory Rate Estimation Using Smartphones. In Proceedings of IEEE Conference on Computer Communications (INFOCOM).

[5]

J Bednar and T Watt. 1984. Alpha-trimmed means and their relationship to median filters. IEEE Transactions on acoustics, speech, and signal processing 32, 1 (1984), 145--153.

[6]

Gerald D Buckberg. 2002. Basic science review: the helix and the heart. The Journal of Thoracic and Cardiovascular Surgery 124, 5 (2002), 863--883.

[7]

Daniel G Carey. 2009. Quantifying differences in the fat burning zone and the aerobic zone: implications for training. The Journal of Strength 8 Conditioning Research 23, 7 (2009), 2090--2095.

[8]

The Helical Heart Company. 2005. The helical heart. http://www.helicalheart.com/video_center.htm. (2005). {Online. Last accessed 14-Feb-2017}.

[9]

Lilian De Greef, Mayank Goel, Min Joon Seo, Eric C Larson, James W Stout, James A Taylor, and Shwetak N Patel. 2014. Bilicam: using mobile phones to monitor newborn jaundice. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 331--342.

Digital Library

[10]

Lawrence T DeCarlo. 1997. On the meaning and use of kurtosis. Psychological methods 2, 3 (1997), 292.

[11]

M Di Rienzo, E Vaini, P Castiglioni, G Merati, P Meriggi, G Parati, A Faini, and F Rizzo. 2013. Wearable seismocardiography: Towards a beat-by-beat assessment of cardiac mechanics in ambulant subjects. Autonomic Neuroscience 178, 1 (2013), 50--59.

[12]

Malcolm Elliott and Alysia Coventry. 2012. Critical care: the eight vital signs of patient monitoring. Br J Nurs 21, 10(2012), 621--625.

[13]

Matthew W Gillman, William B Kannel, Albert Belanger, and Ralph B D’Agostino. 1993. Influence of heart rate on mortality among persons with hypertension: the Framingham Study. American heart journal 125, 4 (1993), 1148--1154.

[14]

Javier Hernandez, Daniel J McDuff, and Rosalind W Picard. 2015. Biophone: Physiology monitoring from peripheral smartphone motions. In Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. IEEE, 7180--7183.

[15]

Azumio Inc. 2010. Instant Heart Rate {Mobile application software}. (2010). Retrieved from https://play.google.com.

[16]

E Jonathan and Martin Leahy. 2010. Investigating a smartphone imaging unit for photoplethysmography. Physiological measurement 31, 11 (2010), N79.

[17]

C Omar F Kamlin, Colm PF O’Donnell, Neil J Everest, Peter G Davis, and Colin J Morley. 2006. Accuracy of clinical assessment of infant heart rate in the delivery room. Resuscitation 71, 3 (2006), 319--321.

[18]

William B Kannel, Craig Kannel, Ralph S Paffenbarger, and L Adrienne Cupples. 1987. Heart rate and cardiovascular mortality: the Framingham Study. American heart journal 113, 6 (1987), 1489--1494.

[19]

Federica Landreani, Alba Martin-Yebra, Claudia Casellato, Esteban Pavan, Carlo Frigo, Pierre-François Migeotte, and Enrico G Caiani. 2015. Feasibility study for beat-to-beat heart rate detection by smartphone’s accelerometers. In E-Health and Bioengineering Conference (EHB), 2015. IEEE, 1--4.

[20]

Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, and Shwetak N Patel. 2012. SpiroSmart: using a microphone to measure lung function on a mobile phone. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM, 280--289.

Digital Library

[21]

Albert B Levin. 1966. A simple test of cardiac function based upon the heart rate changes induced by the Valsalva maneuver. The American journal of cardiology 18, 1 (1966), 90--99.

[22]

Wilburta Q Lindh, Marilyn Pooler, Carol D Tamparo, Barbara M Dahl, and Julie Morris. 2013. Delmar’s comprehensive medical assisting: administrative and clinical competencies. Cengage Learning.

[23]

Jian Liu, Yan Wang, Yingying Chen, Jie Yang, Xu Chen, and Jerry Cheng. 2015. Tracking vital signs during sleep leveraging off-the-shelf wifi. In Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. ACM, 267--276.

Digital Library

[24]

NH Mahmood, N Uyop, N Zulkarnain, FK Che Harun, MF Kamarudin, and A Linoby. 2011. LED indicator for heart rate monitoring system in sport application. In Signal Processing and its Applications (CSPA), 2011 IEEE 7th International Colloquium on. IEEE, 64--66.

[25]

Alison M McManus, Rich SW Masters, Raija MT Laukkanen, CW Clare, Cindy HP Sit, and Fiona CM Ling. 2008. Using heart-rate feedback to increase physical activity in children. Preventive medicine 47, 4 (2008), 402--408.

[26]

Satoshi Nakatani. 2011. Left ventricular rotation and twist: why should we learn? Journal of Cardiovascular Ultrasound 19, 1 (2011).

[27]

Alexandros Pantelopoulos and Nikolaos G Bourbakis. 2010. Prognosis: a wearable health-monitoring system for people at risk: Methodology and modeling. IEEE Transactions on Information Technology in Biomedicine 14, 3 (2010), 613--621.

Digital Library

[28]

N. Patwari, L. Brewer, Q. Tate, O. Kaltiokallio, and M. Bocca. 2014. Breathfinding: A Wireless Network That Monitors and Locates Breathing in a Home. IEEE Journal of Selected Topics in Signal Processing 8, 1 (2014), 30--42.

[29]

Panagiotis Pelegris, K Banitsas, T Orbach, and Kostas Marias. 2010. A novel method to detect heart beat rate using a mobile phone. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. IEEE, 5488--5491.

[30]

Rong-Chao Peng, Wen-Rong Yan, Ning-Ling Zhang, Wan-Hua Lin, Xiao-Lin Zhou, and Yuan-Ting Zhang. 2015. Cuffless and Continuous Blood Pressure Estimation from the Heart Sound Signals. Sensors 15, 9 (2015), 23653--23666.

[31]

Tauhidur Rahman, Alexander T Adams, Ruth Vinisha Ravichandran, Mi Zhang, Shwetak N Patel, Julie A Kientz, and Tanzeem Choudhury. 2015. Dopplesleep: A contactless unobtrusive sleep sensing system using short-range doppler radar. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 39--50.

Digital Library

[32]

Ltd. Samsung Electronics Co. 2014. Samsung Health {Mobile application software}. (2014). Retrieved from https://play.google.com.

[33]

Partho P Sengupta, A Jamil Tajik, Krishnaswamy Chandrasekaran, and Bijoy K Khandheria. 2008. Twist mechanics of the left ventricle: principles and application. Journal of the American College of Cardiology 1, 3 (2008), 366--376.

[34]

Ghufran Shafiq and Kalyana C Veluvolu. 2014. Surface chest motion decomposition for cardiovascular monitoring. Scientific reports 4 (2014).

[35]

Li Shen, Joshua A Hagen, and Ian Papautsky. 2012. Point-of-care colorimetric detection with a smartphone. Lab on a Chip 12, 21 (2012), 4240--4243.

[36]

AF Silva, JP Carmo, PM Mendes, and JH Correia. 2011. Simultaneous cardiac and respiratory frequency measurement based on a single fiber Bragg grating sensor. Measurement Science and Technology 22, 7 (2011), 075801.

[37]

Francisco Torrent-Guasp, Mladen J. Kocica, Antonio F. Corno, Masashi Komeda, Francesc Carreras-Costa, A. Flotats, Juan Cosin-Aguillar, and Han Wen. 2005. Towards new understanding of the heart structure and function. Eurpean Journal of Cardiothoracic Surgery 27, 2 (2005), 191--201.

[38]

Aly Waninge, Annette AJ Van der Putten, Roy E Stewart, Bert Steenbergen, Ruud Van Wijck, and Cees P Van der Schans. 2013. Heart rate and physical activity patterns in persons with profound intellectual and multiple disabilities. The Journal of Strength 8 Conditioning Research 27, 11 (2013), 3150--3158.

[39]

Goya Wannamethee, A Gerald Shaper, and Peter W Macfarlane. 1993. Heart rate, physical activity, and mortality from cancer and other noncardiovascular diseases. American journal of epidemiology 137, 7 (1993), 735--748.

[40]

Hongjiang Wei, Magalie Viallon, Benedicte MA Delattre, Kevin Moulin, Feng Yang, Pierre Croisille, and Yuemin Zhu. 2015. Free-breathing diffusion tensor imaging and tractography of the human heart in healthy volunteers using wavelet-based image fusion. IEEE transactions on medical imaging 34, 1 (2015), 306--316.

[41]

Lide Zhang, Birjodh Tiwana, Robert P Dick, Zhiyun Qian, Z Morley Mao, Zhaoguang Wang, and Lei Yang. 2010. Accurate online power estimation and automatic battery behavior based power model generation for smartphones. In Hardware/Software Codesign and System Synthesis (CODES+ ISSS), 2010 IEEE/ACM/IFIP International Conference on. IEEE, 105--114.

Digital Library

Cited By

Xia KLi WShao YLu SCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Vi2ACT:Video-enhanced Cross-modal Co-learning with Representation Conditional Discriminator for Few-shot Human Activity RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681664(1848-1856)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681664
Wang LWang XZhang YMa XDai HZhang YLi ZGu T(2024)Accurate Blood Pressure Measurement Using Smartphone's Built-in AccelerometerProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595998:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659599
Wu WElgendi MFletcher RBomberg HEichenberger UGuan CMenon C(2023)Detection of heart rate using smartphone gyroscope data: a scoping reviewFrontiers in Cardiovascular Medicine10.3389/fcvm.2023.132929010Online publication date: 18-Dec-2023
https://doi.org/10.3389/fcvm.2023.1329290
Show More Cited By

Index Terms

HeartSense: Ubiquitous Accurate Multi-Modal Fusion-based Heart Rate Estimation Using Smartphones
1. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

HeartSense: estimating blood pressure and ECG from photoplethysmograph using smart phones
SenSys '13: Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems

Regular monitoring of certain vital parameters like heart-rate (HR), blood pressure (BP), Electrocardiogram (ECG) are the basic needs for elderly people and patients with chronic diseases residing at home. In this demo, authors would like to demonstrate ...
HeartSense: smart phones to estimate blood pressure from photoplethysmography
SenSys '14: Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems

In this paper we propose to demonstrate a smart phone application, that estimates human blood pressure (BP) values from photoplethysmography (PPG) signal using Windkessel model. PPG signal is extracted from a video sequence of a user's index fingertip, ...
The anisotropic structure of the human left and right ventricles

An important determinant of cardiac output derives from the structure of the ventricular wall given by the arrangement of the cardiac muscle fibres. A key feature of this arrangement is both a global and local anisotropy. First, a preparation method ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 3

September 2017

2023 pages

EISSN:2474-9567

DOI:10.1145/3139486

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017

Accepted: 01 June 2017

Revised: 01 May 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
930
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)7

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xia KLi WShao YLu SCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Vi2ACT:Video-enhanced Cross-modal Co-learning with Representation Conditional Discriminator for Few-shot Human Activity RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681664(1848-1856)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681664
Wang LWang XZhang YMa XDai HZhang YLi ZGu T(2024)Accurate Blood Pressure Measurement Using Smartphone's Built-in AccelerometerProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595998:2(1-28)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659599
Wu WElgendi MFletcher RBomberg HEichenberger UGuan CMenon C(2023)Detection of heart rate using smartphone gyroscope data: a scoping reviewFrontiers in Cardiovascular Medicine10.3389/fcvm.2023.132929010Online publication date: 18-Dec-2023
https://doi.org/10.3389/fcvm.2023.1329290
Rawassizadeh RRong Y(2023)ODSearchProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35694886:4(1-25)Online publication date: 11-Jan-2023
https://dl.acm.org/doi/10.1145/3569488
Mehmood ASarouji ARahman MAl-Naffouri T(2023)Your smartphone could act as a pulse-oximeter and as a single-lead ECGScientific Reports10.1038/s41598-023-45933-313:1Online publication date: 6-Nov-2023
https://doi.org/10.1038/s41598-023-45933-3
Kateu FJakllari GChaput E(2023)Unobstrusive smartphone-based oxygen saturation measurement using a Meta-Region of interestPervasive and Mobile Computing10.1016/j.pmcj.2022.10174188:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.pmcj.2022.101741
Sabir IBaber JAhmed ASheikh NBakhtyar MKhan ADevi VBalas V(2022)Deep transferable learning on heartbeat classification for imbalance datasetJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21930543:2(2057-2067)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-219305
Wang LLi WSun KZhang FGu TXu CZhang D(2022)LoEarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502936:3(1-24)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550293
Xie ZZhou BCheng XSchoenfeld EYe F(2022)Passive and Context-Aware In-Home Vital Signs Monitoring Using Co-Located UWB-Depth Sensor FusionACM Transactions on Computing for Healthcare10.1145/35499413:4(1-31)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3549941
Wang DWeng JZou YWu K(2022)EmoTracer: A Wearable Physiological and Psychological Monitoring System With Multi-modal SensorsAdjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers10.1145/3544793.3560409(444-449)Online publication date: 11-Sep-2022
https://dl.acm.org/doi/10.1145/3544793.3560409
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents