Abstract
Background
Advancements in wearable technology have provided practitioners and researchers with the ability to conveniently measure various health and/or fitness indices. Specifically, portable devices have been devised for convenient recordings of heart rate variability (HRV). Yet, their accuracies remain questionable.
Objective
The aim was to quantify the accuracy of portable devices compared to electrocardiography (ECG) for measuring a multitude of HRV metrics and to identify potential moderators of this effect.
Methods
This meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Articles published before July 29, 2017 were located via four electronic databases using a combination of the terms related to HRV and validity. Separate effect sizes (ESs), defined as the absolute standardized difference between the HRV value recorded using the portable device compared to ECG, were generated for each HRV metric (ten metrics analyzed in total). A multivariate, multi-level model, incorporating random-effects assumptions, was utilized to quantify the mean ES and 95% confidence interval (CI) and explore potential moderators.
Results
Twenty-three studies yielded 301 effects and revealed that HRV measurements acquired from portable devices differed from those obtained from ECG (ES = 0.23, 95% CI 0.05–0.42), although this effect was small and highly heterogeneous (I2 = 78.6%, 95% CI 76.2–80.7). Moderator analysis revealed that HRV metric (p <0.001), position (p = 0.033), and biological sex (β = 0.45, 95% CI 0.30–0.61; p <0.001), but not portable device, modulated the degree of absolute error. Within metric, absolute error was significantly higher when expressed as standard deviation of all normal–normal (R–R) intervals (SDNN) (ES = 0.44) compared to any other metric, but was no longer significantly different after a sensitivity analysis removed outliers. Likewise, the error associated with the tilt/recovery position was significantly higher than any other position and remained significantly different without outliers in the model.
Conclusions
Our results suggest that HRV measurements acquired using portable devices demonstrate a small amount of absolute error when compared to ECG. However, this small error is acceptable when considering the improved practicality and compliance of HRV measurements acquired through portable devices in the field setting. Practitioners and researchers should consider the cost–benefit along with the simplicity of the measurement when attempting to increase compliance in acquiring HRV measurements.
Similar content being viewed by others
Data Availability Statement
Data used for this analysis are available from the corresponding author upon request.
References
Thompson WR. Worldwide survey of fitness trends for 2017. ACSMs Health Fit J. 2016;20(6):8–17.
Thompson WR. Worldwide survey of fitness trends for 2018: the CREP edition. ACSMs Health Fit J. 2017;21(6):10–9.
Plews DJ, Laursen PB, Stanley J, Kilding AE, Buchheit M. Training adaptation and heart rate variability in elite endurance athletes: opening the door to effective monitoring. Sports Med. 2013;43(9):773–81.
Silva VP, Oliveira NA, Silveira H, Mello RGT, Deslandes AC. Heart rate variability indexes as a marker of chronic adaptation in athletes: a systematic review. Ann Noninvasive Electrocardiol. 2015;20(2):108–18.
Malik M, Camm AJ, Bigger JT Jr, Breithardt G, Cerutti S, Cohen RJ, et al. Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Eur Heart J. 1996;17(3):354–81.
Radespiel-Tröger M, Rauh R, Mahlke C, Gottschalk T, Muck-Weymann M. Agreement of two different methods for measurement of heart rate variability. Clin Auton Res. 2003;13(2):99–102.
Flatt AA, Esco MR, Nakamura FY, Plews DJ. Interpreting daily heart rate variability changes in collegiate female soccer players. J Sports Med Phys Fitness. 2017;57(6):907–15.
Flatt AA, Esco MR. Evaluating individual training adaptation with smartphone-derived heart rate variability in a collegiate female soccer team. J Strength Cond Res. 2016;30(2):378–85.
Esco MR, Flatt AA, Nakamura FY. Initial weekly HRV response is related to the prospective change in VO2max in female soccer players. Int J Sports Med. 2016;37(6):436–41.
Nakamura FY, Pereira LA, Esco MR, Flatt AA, Moraes JE, Abad CCC, et al. Intraday and interday reliability of ultra-short-term heart rate variability in rugby union players. J Strength Cond Res. 2017;31(2):548–51.
Flatt AA, Esco MR, Allen JR, Robinson JB, Earley RL, Fedewa MV, et al. Heart rate variability and training load among National Collegiate Athletic Association Division 1 college football players throughout spring camp. J Strength Cond Res. 2017. https://doi.org/10.1519/jsc.0000000000002241 (Epub ahead of print).
Flatt AA, Esco MR. Heart rate variability stabilization in athletes: towards more convenient data acquisition. Clin Physiol Funct Imaging. 2016;36(5):331–6.
Berntson GG, Bigger JT Jr, Eckberg DL, Grossman P, Kaufmann PG, Malik M, et al. Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology. 1997;34(6):623–48.
Board L, Ispoglou T, Ingle L. Validity of telemetric-derived measures of heart rate variability: a systematic review. J Exerc Physiol Online. 2016;19(6):64–84.
Barbosa MP, da Silva NT, de Azevedo FM, Pastre CM, Vanderlei LC. Comparison of Polar(R) RS800G3 heart rate monitor with Polar(R) S810i and electrocardiogram to obtain the series of RR intervals and analysis of heart rate variability at rest. Clin Physiol Funct Imaging. 2016;36(2):112–7.
Bolkhovsky JB, Scully CG, Chon KH, editors. Statistical analysis of heart rate and heart rate variability monitoring through the use of smart phone cameras. In: Engineering in Medicine and Biology Society (EMBC), 2012 annual international conference of the IEEE; 2012: IEEE.
Wallen MB, Hasson D, Theorell T, Canlon B, Osika W. Possibilities and limitations of the Polar RS800 in measuring heart rate variability at rest. Eur J Appl Physiol. 2012;112(3):1153–65.
Hernando D, Garatachea N, Almeida R, Casajús JA, Bailón R. Validation of heart rate monitor Polar RS800 for heart rate variability analysis during exercise. J Strength Cond Res. 2018;32(3):716–25.
Kingsley M, Lewis MJ, Marson RE. Comparison of Polar 810 s and an ambulatory ECG System for RR interval measures during progressive exercise. Int J Sports Med. 2005;26(1):39–44.
Romagnoli M, Alis R, Guillen J, Basterra J, Villacastin J, Guillen S. A novel device based on smart textile to control heart’s activity during exercise. Australas Phys Eng Sci Med. 2014;37(2):377–84.
Weippert M, Kumar M, Kreuzfeld S, Arndt D, Rieger A, Stoll R. Comparison of three mobile devices for measuring R-R intervals and heart rate variability: Polar S810i, Suunto t6 and an ambulatory ECG system. Eur J Appl Physiol. 2010;109(4):779–86.
Chellakumar PJ, Brumfield A, Kunderu K, Schopper AW. Heart rate variability: comparison among devices with different temporal resolutions. Physiol Meas. 2005;26(6):979–86.
Russoniello CV, Zhirnov YN, Pougatchev VI, Gribkov EN. Heart rate variability and biological age: implications for health and gaming. Cyberpsychol Behav Soc Netw. 2013;16(4):302–8.
Esco MR, Flatt AA, Nakamura FY. Agreement between a smartphone pulse sensor application and electrocardiography for determining lnRMSSD. J Strength Cond Res. 2017;31(2):380–5.
Flatt AA, Esco MR. Validity of the ithlete™ smart phone application for determining ultra-short-term heart rate variability. J Hum Kinet. 2013;39:85–92.
Vanderlei LCM, Silva RA, Pastre CM, Azevedo FMD, Godoy MF. Comparison of the Polar S810i monitor and the ECG for the analysis of heart rate variability in the time and frequency domains. Braz J Med Biol Res. 2008;41(10):854–9.
Gamelin FX, Berthoin S, Bosquet L. Validity of the Polar S810 heart rate monitor to measure R-R intervals at rest. Med Sci Sports Exerc. 2006;38(5):887–93.
Giles D, Draper N, Neil W. Validity of the Polar V800 heart rate monitor to measure RR intervals at rest. Eur J Appl Physiol. 2016;116(3):563–71.
Montaño A, Brown F, Credeur DP, Williams MA, Stoner L. Telemetry-derived heart rate variability responses to a physical stressor. Clin Physiol Funct Imaging. 2017;37(4):421–7.
Vasconcellos FV, Seabra A, Cunha FA, Montenegro RA, Bouskela E, Farinatti P. Heart rate variability assessment with fingertip photoplethysmography and Polar RS800cx as compared with electrocardiography in obese adolescents. Blood Press Monit. 2015;20(6):351–60.
Nunan D, Jakovljevic DG, Donovan G, Hodges LD, Sandercock GR, Brodie DA. Levels of agreement for RR intervals and short-term heart rate variability obtained from the Polar S810 and an alternative system. Eur J Appl Physiol. 2008;103(5):529–37.
Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:332–6.
Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6(11):e012799.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for reporting of diagnostic accuracy. Clin Chem. 2003;49(1):1–6.
Quintana DS, Alvares GA, Heathers JA. Guidelines for reporting articles on psychiatry and heart rate variability (GRAPH): recommendations to advance research communication. Transl Psychiatry. 2016;6:e803.
Laborde S, Mosley E, Thayer JF. Heart rate variability and cardiac vagal tone in psychophysiological research—recommendations for experiment planning, data analysis, and data reporting. Front Psychol. 2017;8:213.
Cohen J. A power primer. Psychol Bull. 1992;112(1):155–9.
Hedges L, Olkin I. Statistical methods for meta-analysis. 6th ed. San Diego: Academic Press; 1985. p. 79–201.
Crossingham IR, Nethercott DR, Columb MO. Comparing cardiac output monitors and defining agreement: a systematic review and meta-analysis. J Intensive Care Soc. 2016;17(4):302–13.
Zaki R, Bulgiba A, Ismail R, Ismail NA. Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review. PLoS One. 2012;7(5):e37908.
Plews DJ, Scott B, Altini M, Wood M, Kilding AE, Laursen PB. Comparison of heart rate variability recording with smart phone photoplethysmographic, Polar H7 chest strap and electrocardiogram methods. Int J Sports Physiol Perform. 2017;12(10):1324–8.
Boos CJ, Bakker-Dyos J, Watchorn J, Woods DR, O’Hara JP, Macconnachie L, et al. A comparison of two methods of heart rate variability assessment at high altitude. Clin Physiol Funct Imaging. 2017;37(6):582–7.
Hox J. Multilevel analysis: techniques and applications. 2nd edn. New York, NY: Taylor & Francis; 2010. p. 205–32.
Singer JD, Using SAS. PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat. 1998;23(4):323–55.
Higgins J, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–60.
Schäfer A, Vagedes J. How accurate is pulse rate variability as an estimate of heart rate variability? A review on studies comparing photoplethysmographic technology with an electrocardiogram. Int J Cardiol. 2013;166(1):15–29.
Egger M, Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629–34.
Rosenberg M. The file-drawer problem revisited: a general weighted method for calculating fail-safe numbers in meta-analysis. Evolution. 2005;59(2):464–8.
Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1–48.
R Development Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. 2017.
Gamelin FX, Baquet G, Berthoin S, Bosquet L. Validity of the Polar S810 to measure R-R intervals in children. Int J Sports Med. 2008;29(2):134–8.
Heathers JAJ. Smartphone-enabled pulse rate variability: an alternative methodology for the collection of heart rate variability in psychophysiological research. Int J Psychophysiol. 2013;89:297–304.
Nunan D, Donovan G, Jakovljevic DG, Hodges LD, Sandercock GR, Brodie DA. Validity and reliability of short-term heart-rate variability from the Polar S810. Med Sci Sports Exerc. 2009;41(1):243–50.
Porto LGG, Junqueira J. Comparison of time-domain short-term heart interval variability analysis using a wrist-worn heart rate monitor and the conventional electrocardiogram. Pacing Clin Electrophysiol. 2009;32(1):43–51.
Guzik P, Piekos C, Pierog O, Fenech N, Krauze T, Piskorski J, et al. Classic electrocardiogram-based and mobile technology derived approaches to heart rate variability are not equivalent. Int J Cardiol. 2018;258:154–6.
Sterne JAC, Sutton AJ, Ioannidis JPA, Terrin N, Jones DR, Lau J, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002.
Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data. J R Stat Soc Ser A. 1988;151(3):419–63.
Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol. 2001;54(10):1046–55.
Rosenthal R. Meta-analytic procedures for social research. 2nd edn. Newbury Park, CA: SAGE Publications; 1991.
Sammito S, Böckelmann I. Options and limitations of heart rate measurement and analysis of heart rate variability by mobile devices: a systematic review. Herzschrittmacherther Elektrophysiol. 2016;27(1):38–45.
Buchheit M, Mendez-Villanueva A. Improbable effect of carbohydrate diet on cardiac autonomic modulation during exercise. Eur J Appl Physiol. 2010;109(3):571–4.
Schmitt L, Regnard J, Millet GP. Monitoring fatigue status with HRV measures in elite athletes: an avenue beyond RMSSD? Front Physiol. 2015;6:343.
Penttila J, Helminen A, Jartti T, Kuusela T, Huikuri HV, Tulppo MP, et al. Time domain, geometrical and frequency domain analysis of cardiac vagal outflow: effects of various respiratory patterns. Clin Physiol. 2001;21(3):365–76.
Buchheit M. Monitoring training status with HR measures: do all roads lead to Rome? Front Physiol. 2014;5:73.
Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231–40.
Esco MR, Flatt AA. Ultra-short-term heart rate variability indexes at rest and post-exercise in athletes: evaluating the agreement with accepted recommendations. J Sports Sci Med. 2014;13(3):535–41.
Altini M, Amft O. HRV4Training: large-scale longitudinal training load analysis in unconstrained free-living settings using a smartphone application. Conf Proc IEEE Eng Med Biol Soc. 2016;2016:2610–3.
Plews DJ, Laursen PB, Buchheit M. Day-to-day heart-rate variability recordings in world-champion rowers: appreciating unique athlete characteristics. Int J Sports Physiol Perform. 2017;12(5):697–703.
Nuuttila OP, Nikander A, Polomoshnov D, Laukkanen JA, Hakkinen K. Effects of HRV-guided vs. predetermined block training on performance, HRV and serum hormones. Int J Sports Med. 2017;38(12):909–20.
Plews DJ, Laursen PB, Kilding AE, Buchheit M. Evaluating training adaptation with heart-rate measures: a methodological comparison. Int J Sports Physiol Perform. 2013;8(6):688–91.
Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.
Whiting P, Westwood M, Burke M, Sterne J, Glanville J. Systematic reviews of test accuracy should search a range of databases to identify primary studies. J Clin Epidemiol. 2008;61(4):357–64.
Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.
Linder SK, Kamath GR, Pratt GF, Saraykar SS, Volk RJ. Citation searches are more sensitive than keyword searches to identify studies using specific measurement instruments. J Clin Epidemiol. 2015;68(4):412–7.
Allen IE, Olkin I. Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA. 1999;282(7):634–5.
Author information
Authors and Affiliations
Contributions
Ward Dobbs designed the study, coded and analyzed effects, carried out the initial analysis, drafted the initial manuscript, and approved the final manuscript as submitted. Michael Fedewa conceptualized and designed the study, coded and analyzed effects, carried out the initial analysis, drafted the initial manuscript, and approved the final manuscript as submitted. Hayley MacDonald designed the study, coded and analyzed effects, reviewed and revised the initial manuscript, and approved the final manuscript as submitted. Clifton Holmes coded and analyzed effects, reviewed and revised the initial manuscript, and approved the final manuscript as submitted. Zackary Cicone coded and analyzed effects, reviewed and revised the initial manuscript, and approved the final manuscript as submitted. Daniel Plews reviewed and revised the initial manuscript, and approved the final manuscript as submitted. Michael Esco conceptualized the study, coded and analyzed effects, drafted the initial manuscript, and approved the final manuscript as submitted.
Corresponding author
Ethics declarations
Funding
No sources of funding were used to assist in the preparation of this article.
Conflict of interest
Ward Dobbs, Michael Fedewa, Hayley MacDonald, Clifton Holmes, Zackary Cicone, Daniel Plews and Michael Esco declare that they have no conflicts of interest relevant to the content of this review.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Electronic Supplementary Material Table S1a.
Standard for Reporting Diagnostic Accuracy Studies Guidelines for Heart Rate Variability Research (STARDHRV) Methodology Study Quality Assessment Tool for Primary-Level Evidence. BMI, body mass index; ES, effect size; ICC, intra-class correlation; LOA, limits of agreement; SD, standard deviation
Electronic Supplementary Material Table S1b.
Item Description of the Standard for Reporting Diagnostic Accuracy Studies Guidelines for Heart Rate Variability Research (STARDHRV) Methodology Study Quality Assessment Tool for Primary-Level Evidence. The source of the STARDHRV quality item is provided, and if applicable, describes whether the item was used in its original form, modified, or newly added by authors. Abbreviations: BMI, body mass index; ECG, electrocardiogram; GRAPH, Guidelines for Reporting Articles on Psychiatry and Heart rate variability; ICC, intra-class correlation; LOA, limits of agreement; STARD, Standard for Reporting Diagnostic Accuracy Studies Guidelines
Electronic Supplementary Material Table S2.
Description of selected studies (k = 23) examining the validity of heart rate variability measurements obtained from portable heart rate monitors compared to a criterion electrocardiogram. %, percentage; a, calculated from data provided; ApEn, approximate entropy; AR, autoregressive; b/w, between; BMI, body mass index; kg/m, kilograms/meters; CWT, continuous wavelet transform; ECG, electrocardiogram; FFT, fast Fourier transform; HF, high frequency; HR, heart rate; HRV, heart rate variability; LF, low frequency; lnRMSSD, log transformed RMSSD; LoA, limits of agreement; N, study population size; NR, not reported; PA, physical activity; pNN50, percentage of consecutive N-N intervals that deviate from one another by more than 50 ms; RR, RR interval; RMSSD, square root of the mean squared differences between normal adjacent R-R intervals; SaEn, sample entropy; SD1, dispersion of points perpendicular to the line of identity; SD2, dispersion of points along the line of identity; SDNN, standard deviation of all normal–normal (R-R) intervals; TP, total power; VLF, very low frequency; WP, Welch’s paradigm; yrs, year
Electronic Supplementary Material Table S3.
Item-by-item summary of methodology study quality for the included studies (k = 23) using a version of the Standard for Reporting Diagnostic Accuracy Studies modified for the use of heart rate variability (STARDHRV). n/a, not applicable due to the study design
Electronic Supplementary Material Table S4.
Individual moderator analyses for categorical variables of interest after removal of outliers (n = 275 total effects). %, proportion of effects accounted for; a, significant omnibus test; b, significantly different from all other measurements within the variable; ES, estimated absolute standardized mean difference effect size; HF, high frequency; HR, heart rate; HRV, heart rate variability; LF, low frequency; LF:HF, LF to HF ratio; n, number of effects; pNN50, percentage of consecutive N-N intervals that deviate from one another by more than 50 ms; PPG, photoplethysmogrophy; RMSSD, square root of the mean squared differences between normal adjacent R-R intervals; SD1, dispersion of points perpendicular to the line of identity; SD2, dispersion of points along the line of identity; SDNN, standard deviation of all normal–normal (R-R) intervals; SE, standard error; TP, total power; VLF, very low frequency
Electronic Supplementary Material Table S5:
Moderating relationships greater than zero within the multiple moderator model after removal of outliers (n = 275 total effects). CI, confidence interval; ES, estimated absolute standardized mean difference effect size; HF, high frequency power; LF, low frequency power; SDNN, standard deviation of all normal–normal (R-R) intervals; SE, standard error; RMSSD, square root of the mean squared differences between normal adjacent R-R intervals
Rights and permissions
About this article
Cite this article
Dobbs, W.C., Fedewa, M.V., MacDonald, H.V. et al. The Accuracy of Acquiring Heart Rate Variability from Portable Devices: A Systematic Review and Meta-Analysis. Sports Med 49, 417–435 (2019). https://doi.org/10.1007/s40279-019-01061-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40279-019-01061-5