research-article

Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring

Authors:

Alessio Burrello,

Daniele Jahier Pagliari,

Pierangelo Maria Rapa,

Matilde Semilia,

Tommaso Polonelli,

Massimo Poncino,

Simone BenattiAuthors Info & Claims

ACM Transactions on Computing for Healthcare (HEALTH), Volume 3, Issue 2

Article No.: 19, Pages 1 - 25

https://doi.org/10.1145/3487910

Published: 03 March 2022 Publication History

Abstract

Photoplethysmography (PPG) sensors allow for non-invasive and comfortable heart rate (HR) monitoring, suitable for compact wrist-worn devices. Unfortunately, motion artifacts (MAs) severely impact the monitoring accuracy, causing high variability in the skin-to-sensor interface. Several data fusion techniques have been introduced to cope with this problem, based on combining PPG signals with inertial sensor data. Until now, both commercial and reasearch solutions are computationally efficient but not very robust, or strongly dependent on hand-tuned parameters, which leads to poor generalization performance. In this work, we tackle these limitations by proposing a computationally lightweight yet robust deep learning-based approach for PPG-based HR estimation. Specifically, we derive a diverse set of Temporal Convolutional Networks for HR estimation, leveraging Neural Architecture Search. Moreover, we also introduce ActPPG, an adaptive algorithm that selects among multiple HR estimators depending on the amount of MAs, to improve energy efficiency. We validate our approaches on two benchmark datasets, achieving as low as 3.84 beats per minute of Mean Absolute Error on PPG-Dalia, which outperforms the previous state of the art. Moreover, we deploy our models on a low-power commercial microcontroller (STM32L4), obtaining a rich set of Pareto optimal solutions in the complexity vs. accuracy space.

References

[1]

2020. Neurosky. Retrieved from http://neurosky.com/wp-content/uploads/2016/06/TOF-side-by-side-competitor-comparison.pdf.

[2]

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Retrieved from https://www.tensorflow.org/.

[3]

Rasha M. Al-Eidan, Hend Al-Khalifa, and Abdul Malik Al-Salman. 2018. A review of wrist-worn wearable: Sensors, models, and challenges. J. Sens. 2018 (2018).

[4]

Apple. [n.d.]. Apple Watch Series. Retrieved from https://www.apple.com/lae/watch/.

[5]

K. R. Arunkumar and M. Bhaskar. 2020. Robust de-noising technique for accurate heart rate estimation using wrist-type PPG signals. IEEE Sens. J. 20, 14 (2020), 7980–7987.

[6]

Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv:1803.01271. Retrieved from https://arxiv.org/abs/1803.01271.

[7]

BiosignalsPLUX. [n.d.]. Respiban Professional 2019. Retrieved from https://biosignalsplux.com/products/wearables/respiban-pro.html.

[8]

Dwaipayan Biswas, Luke Everson, Muqing Liu, Madhuri Panwar, Bram-Ernst Verhoef, Shrishail Patki, Chris H. Kim, Amit Acharyya, Chris Van Hoof, Mario Konijnenburg, et al. 2019. CorNET: Deep learning framework for PPG-based heart rate estimation and biometric identification in ambulant environment. IEEE Trans. Biomed. Circ. Syst. 13, 2 (2019), 282–291.

[9]

D. Biswas, N. Simões-Capela, C. Van Hoof, and N. Van Helleputte. 2019. Heart rate estimation from wrist-worn photoplethysmography: A review. IEEE Sens. J. 19, 16 (2019), 6560–6570.

[10]

Alessio Burrello, Simone Benatti, Kaspar Anton Schindler, Luca Benini, and Abbas Rahimi. 2020. An ensemble of hyperdimensional classifiers: Hardware-friendly short-latency seizure detection with automatic iEEG electrode selection. IEEE J. Biomed. Health Inf. 25, 4 (2020), 935–946.

[11]

Alessio Burrello, Kaspar Anton Schindler, Luca Benini, and Abbas Rahimi. 2020. Hyperdimensional computing with local binary patterns: One-shot learning for seizure onset detection and identification of ictogenic brain regions from short-time iEEG recordings. IEEE Trans. Biomed. Eng. 67, 2 (2020), 601–613.

[12]

Han Cai, Ligeng Zhu, and Song Han. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv:1812.00332. Retrieved from https://arxiv.org/abs/1812.00332.

[13]

Denisse Castaneda, Aibhlin Esparza, Mohammad Ghamari, Cinna Soltanpur, and Homer Nazeran. 2018. A review on wearable photoplethysmography sensors and their potential future applications in health care. Int. J. Biosens. Bioelectr. 4, 4 (2018), 195–202.

[14]

Xiangmao Chang, Gangkai Li, Guoliang Xing, Kun Zhu, and Linlin Tu. 2021. DeepHeart: A deep learning approach for accurate heart rate estimation from PPG signals. ACM Trans. Sen. Netw. 17, 2, Article 14 (Jan. 2021), 18 pages.

Digital Library

[15]

Heewon Chung, Hoon Ko, Hooseok Lee, and Jinseok Lee. 2020. Deep learning for heart rate estimation from reflectance photoplethysmography with acceleration power spectrum and acceleration intensity. IEEE Access 8 (2020), 63390–63402.

[16]

Heewon Chung, Hooseok Lee, and Jinseok Lee. 2018. Finite state machine framework for instantaneous heart rate validation using wearable photoplethysmography during intensive exercise. IEEE J. Biomed. Health Inform. 23, 4 (2018), 1595–1606.

[17]

Francesco Conti. 2020. Technical Report: NEMO DNN Quantization for Deployment Model. arxiv:2004.05930 [cs.LG]. Retrieved from https://arxiv.org/abs/2004.05930.

[18]

Francesco Daghero, Alessio Burrello, Daniele Jahier Pagliari, Luca Benini, Enrico Macii, and Massimo Poncino. 2020. Energy-efficient adaptive machine learning on IoT end-nodes with class-dependent confidence. In Proceedings of the International Conference on Electronics Circuits and Systems (ICECS’20). IEEE, 1–4.

[19]

Empatica. [n.d.]. E4 Wristband 2014. Retrieved from https://www.empatica.com/en-eu/research/e4/.

[20]

Fitbit. [n.d.]. Fitbit Charge 4. Retrieved from https://www.fitbit.com/global/us/products/trackers/charge4.

[21]

Z. Ge, P. W. C. Prasad, N. Costadopoulos, Abeer Alsadoon, A. K. Singh, and A. Elchouemi. 2016. Evaluating the accuracy of wearable heart rate monitors. In Proceedings of the 2nd International Conference on Advances in Computing, Communication, & Automation (ICACCA’16). IEEE, 1–6.

[22]

Ariel Gordon, Elad Eban, Ofir Nachum, Bo Chen, Hao Wu, Tien-Ju Yang, and Edward Choi. 2018. Morphnet: Fast & simple resource-constrained structure learning of deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1586–1595.

[23]

Nicholas Huang and Nandakumar Selvaraj. 2020. Robust PPG-based ambulatory heart rate tracking algorithm. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 5929–5934.

[24]

T. M. Ingolfsson, M. Hersche, X. Wang, N. Kobayashi, L. Cavigelli, and L. Benini. 2020. EEG-TCNet: An accurate temporal convolutional network for embedded motor-imagery brain-machine interfaces. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC’20). 2958–2965.

Digital Library

[25]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167. Retrieved from https://arxiv.org/abs/1502.03167.

[26]

Daniele Jahier Pagliari, Enrico Macii, and Massimo Poncino. 2018. Dynamic bit-width reconfiguration for energy-efficient deep learning hardware. In Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED’18). ACM, New York, NY.

Digital Library

[27]

Hao-Yu Jan, Mei-Fen Chen, Tieh-Cheng Fu, Wen-Chen Lin, Cheng-Lun Tsai, and Kang-Ping Lin. 2019. Evaluation of coherence between ECG and PPG derived parameters on heart rate variability and respiration in healthy volunteers with/without controlled breathing. J. Med. Biol. Eng. 39, 5 (2019), 783–795.

[28]

Colin Lea, Rene Vidal, Austin Reiter, and Gregory D. Hager. 2016. Temporal convolutional networks: A unified approach to action segmentation. In Proceedings of the European Conference on Computer Vision. Springer, 47–54.

[29]

Ji Lin, Wei-Ming Chen, Yujun Lin, Chuang Gan, Song Han, et al. 2020. Mcunet: Tiny deep learning on iot devices. In Advances in Neural Information Processing Systems, Vol. 33.

[30]

Mahdi Boloursaz Mashhadi, Ehsan Asadi, Mohsen Eskandari, Shahrzad Kiani, and Farokh Marvasti. 2015. Heart rate tracking using wrist-type photoplethysmographic (PPG) signals during physical exercise with simultaneous accelerometry. IEEE Sign. Process. Lett. 23, 2 (2015), 227–231.

[31]

ST Microelectornics. [n.d.]. STM32L476. Retrieved from https://www.st.com/resource/en/datasheet/stm32l476je.pdf.

[32]

L. Mocerino and A. Calimera. 2020. Fast and accurate inference on microcontrollers with boosted cooperative convolutional neural networks (BC-Net). IEEE Trans. Circ. Syst. I: Regul. Pap. (2020), 1–12.

[33]

Benjamin W. Nelson, Carissa A. Low, Nicholas Jacobson, Patricia Areán, John Torous, and Nicholas B. Allen. 2020. Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research. NPJ Dig. Med. 3, 1 (2020), 1–9.

[34]

Ralph Neuneier and Hans Georg Zimmermann. 1998. How to train neural networks. In Neural Networks: Tricks of the Trade. Springer, 373–423.

[35]

Eunhyeok Park, Dongyoung Kim, Soobeom Kim, Yong-Deok Kim, Gunhee Kim, Sungroh Yoon, and Sungjoo Yoo. 2015. Big/little deep neural network for ultra low power inference. In Proceedings of the International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS’15). IEEE, 124–132.

Digital Library

[36]

Attila Reiss, Ina Indlekofer, Philip Schmidt, and Kristof Van Laerhoven. 2019. Deep PPG: Large-scale heart rate estimation with convolutional neural networks. Sensors 19, 14 (2019), 3079.

[37]

Lei Ren, Yuxin Liu, Xiaokang Wang, Jinhu Lü, and M Jamal Deen. 2021. Cloud-edge based lightweight temporal convolutional networks for remaining useful life prediction in IIoT. IEEE IoT J. 8, 16 (2021), 12578–12587.

[38]

Matteo Risso, Alessio Burrello, Daniele Jahier Pagliari, Simone Benatti, Enrico Macii, Luca Benini, and Massimo Poncino. 2021. Robust and energy-efficient PPG-based heart-rate monitoring. In Proceedings of the International Symposium on Circuit and Systems (ISCAS’21). IEEE.

[39]

Leandro Giacomini Rocha, Dwaipayan Biswas, Bram-Ernst Verhoef, Sergio Bampi, Chris Van Hoof, Mario Konijnenburg, Marian Verhelst, and Nick Van Helleputte. 2020. Binary corNET: Accelerator for HR estimation from wrist-PPG. IEEE Trans. Biomed. Circ. Syst. 14, 4 (2020), 715–726.

[40]

Seyed Salehizadeh, Duy Dao, Jeffrey Bolkhovsky, Chae Cho, Yitzhak Mendelson, and Ki H. Chon. 2016. A novel time-varying spectral filtering algorithm for reconstruction of motion artifact corrupted heart rate signals during intense physical activities using a wearable photoplethysmogram sensor. Sensors 16, 1 (2016), 10.

[41]

Tim Schäck, Michael Muma, and Abdelhak M. Zoubir. 2017. Computationally efficient heart rate estimation during physical exercise using photoplethysmographic signals. In Proceedings of the 25th European Signal Processing Conference (EUSIPCO’17). IEEE, 2478–2481.

[42]

Hang Sik Shin, Chungkeun Lee, and Myoungho Lee. 2009. Adaptive threshold method for the peak detection of photoplethysmographic waveform. Comput. Biol. Med. 39, 12 (2009), 1145–1152.

Digital Library

[43]

A. Shyam, Vignesh Ravichandran, S. P. Preejith, Jayaraj Joseph, and Mohanasankar Sivaprakasam. 2019. PPGnet: Deep network for device independent heart rate estimation from photoplethysmogram. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’19). IEEE, 1899–1902.

[44]

Nina Sviridova and Kenshi Sakai. 2015. Human photoplethysmogram: New insight into chaotic characteristics. Chaos Solitons Fract. 77 (2015), 53–63.

[45]

Toshiyo Tamura, Yuka Maeda, Masaki Sekine, and Masaki Yoshida. 2014. Wearable photoplethysmographic sensors-past and present. Electronics 3, 2 (2014), 282–302.

[46]

Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le. 2019. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2820–2828.

[47]

Andriy Temko. 2017. Accurate heart rate monitoring during physical exercises using PPG. IEEE Trans. Biomed. Eng. 64, 9 (2017), 2016–2024.

[48]

Anjali S. Yeole and Dhananjay R. Kalbande. 2016. Use of internet of things (IoT) in healthcare: A survey. In Proceedings of the ACM Symposium on Women in Research 2016. 71–76.

Digital Library

[49]

Ming Yuan and Yi Lin. 2006. Model selection and estimation in regression with grouped variables. J. Roy. Stat. Soc.: Ser. B 68, 1 (2006), 49–67.

[50]

Marcello Zanghieri, Simone Benatti, Alessio Burrello, Victor Kartsch, Francesco Conti, and Luca Benini. 2020. Robust real-time embedded EMG recognition framework using temporal convolutional networks on a multicore IoT processor. IEEE Trans. Biomed. Circ. Syst. 14, 2 (2020), 244–256.

[51]

Xiang Zhang, Lina Yao, Xianzhi Wang, Jessica Monaghan, David Mcalpine, and Yu Zhang. 2019. A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv:1905.04149. Retrieved from https://arxiv.org/abs/1905.04149.

[52]

Yangsong Zhang, Benyuan Liu, and Zhilin Zhang. 2015. Combining ensemble empirical mode decomposition with spectrum subtraction technique for heart rate monitoring using wrist-type photoplethysmography. Biomed. Sign. Process. Contr. 21 (2015), 119–125.

[53]

Zhilin Zhang. 2015. Photoplethysmography-based heart rate monitoring in physical activities via joint sparse spectrum reconstruction. IEEE Trans. Biomed. Eng. 62, 8 (2015), 1902–1910.

[54]

Zhilin Zhang, Zhouyue Pi, and Benyuan Liu. 2014. TROIKA: A general framework for heart rate monitoring using wrist-type photoplethysmographic signals during intensive physical exercise. IEEE Trans. Biomed. Eng. 62, 2 (2014), 522–531.

[55]

Menglian Zhou and Nandakumar Selvaraj. 2020. Heart rate monitoring using sparse spectral curve tracing. In Proceedings of the 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC’20). IEEE, 5347–5352.

Cited By

Yang LAmin OShihada B(2024)Intelligent Wearable Systems: Opportunities and Challenges in Health and SportsACM Computing Surveys10.1145/364846956:7(1-42)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3648469
Zheng YWu CCai PZhong ZHuang HJiang Y(2024)Tiny-PPG: A lightweight deep neural network for real-time detection of motion artifacts in photoplethysmogram signals on edge devicesInternet of Things10.1016/j.iot.2023.10100725(101007)Online publication date: May-2024
https://doi.org/10.1016/j.iot.2023.101007
Xie CBurrello ADaghero FBenini LCalimera AMacii EPoncino MJahier Pagliari D(2023)Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic InferenceSensors10.3390/s2304206523:4(2065)Online publication date: 12-Feb-2023
https://doi.org/10.3390/s23042065
Show More Cited By

Index Terms

Embedding Temporal Convolutional Networks for Energy-efficient PPG-based Heart Rate Monitoring
1. Applied computing
  1. Life and medical sciences
    1. Consumer health

Recommendations

A Deep Learning–based PPG Quality Assessment Approach for Heart Rate and Heart Rate Variability
Photoplethysmography (PPG) is a non-invasive optical method to acquire various vital signs, including heart rate (HR) and heart rate variability (HRV). The PPG method is highly susceptible to motion artifacts and environmental noise. Unfortunately, such ...
Toward Hypertension Prediction Based on PPG-Derived HRV Signals: a Feasibility Study

Heart rate variability (HRV) is often used to assess the risk of cardiovascular disease, and data on this can be obtained via electrocardiography (ECG). However, collecting heart rate data via photoplethysmography (PPG) is now a lot easier. We ...
Robust CNN-based Respiration Rate Estimation for Smartwatch PPG and IMU
ICBRA '23: Proceedings of the 2023 10th International Conference on Bioinformatics Research and Applications

Respiratory rate (RR) serves as an indicator of various medical conditions, such as cardiovascular diseases and sleep disorders. Several studies have employed signal processing and machine learning techniques to extract RR from biosignals, such as ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing for Healthcare

ACM Transactions on Computing for Healthcare Volume 3, Issue 2

April 2022

292 pages

EISSN:2637-8051

DOI:10.1145/3505188

Editors:
Insup Lee
University of Pennsylvania, USA
,
John A. Stankovic
University of Virginia, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 March 2022

Accepted: 01 September 2021

Revised: 01 August 2021

Received: 01 March 2021

Published in HEALTH Volume 3, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

Hasler Foundation
EU’s Horizon 2020 Research and Innovation Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
792
Total Downloads

Downloads (Last 12 months)254
Downloads (Last 6 weeks)6

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang LAmin OShihada B(2024)Intelligent Wearable Systems: Opportunities and Challenges in Health and SportsACM Computing Surveys10.1145/364846956:7(1-42)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3648469
Zheng YWu CCai PZhong ZHuang HJiang Y(2024)Tiny-PPG: A lightweight deep neural network for real-time detection of motion artifacts in photoplethysmogram signals on edge devicesInternet of Things10.1016/j.iot.2023.10100725(101007)Online publication date: May-2024
https://doi.org/10.1016/j.iot.2023.101007
Xie CBurrello ADaghero FBenini LCalimera AMacii EPoncino MJahier Pagliari D(2023)Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic InferenceSensors10.3390/s2304206523:4(2065)Online publication date: 12-Feb-2023
https://doi.org/10.3390/s23042065
Burrello ARisso MTomasello NChen YBenini LMacii EPoncino MPagliari D(2023)Energy-efficient Wearable-to-Mobile Offload of ML Inference for PPG-based Heart-Rate Estimation2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)10.23919/DATE56975.2023.10137129(1-6)Online publication date: May-2023
https://doi.org/10.23919/DATE56975.2023.10137129
Amidei ARapa PTagliavini GRabbeni RPavan PBenatti S(2023)ANGELS - Smart Steering Wheel for Driver Safety2023 9th International Workshop on Advances in Sensors and Interfaces (IWASI)10.1109/IWASI58316.2023.10164505(15-20)Online publication date: 8-Jun-2023
https://doi.org/10.1109/IWASI58316.2023.10164505
Gao H(2023)Construction of a Motion Data Collection System for Mobile Device Heart Rate Monitoring2023 International Conference on Integrated Intelligence and Communication Systems (ICIICS)10.1109/ICIICS59993.2023.10420930(1-6)Online publication date: 24-Nov-2023
https://doi.org/10.1109/ICIICS59993.2023.10420930
Kasnesis PToumanidis LBurrello AChatzigeorgiou CPatrikakis C(2023)Feature-Level Cross-Attentional PPG and Motion Signal Fusion for Heart Rate Estimation2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC57700.2023.00267(1731-1736)Online publication date: Jul-2023
https://doi.org/10.1109/COMPSAC57700.2023.00267
Bellandi VD’Andrea GMaghool SSiccardi S(2022)Digital Health Data, a Way to Take Under Control the Quality During the Elaboration Processes2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS)10.1109/SITIS57111.2022.00015(37-44)Online publication date: Oct-2022
https://doi.org/10.1109/SITIS57111.2022.00015

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents