research-article

Open access

Variationally regularized graph-based representation learning for electronic health records

Authors:

Narges RazavianAuthors Info & Claims

CHIL '21: Proceedings of the Conference on Health, Inference, and Learning

Pages 1 - 13

https://doi.org/10.1145/3450439.3451855

Published: 08 April 2021 Publication History

Abstract

Electronic Health Records (EHR) are high-dimensional data with implicit connections among thousands of medical concepts. These connections, for instance, the co-occurrence of diseases and lab-disease correlations can be informative when only a subset of these variables is documented by the clinician. A feasible approach to improving the representation learning of EHR data is to associate relevant medical concepts and utilize these connections. Existing medical ontologies can be the reference for EHR structures, but they place numerous constraints on the data source. Recent progress on graph neural networks (GNN) enables end-to-end learning of topological structures for non-grid or non-sequential data. However, there are problems to be addressed on how to learn the medical graph adaptively and how to understand the effect of medical graph on representation learning. In this paper, we propose a variationally regularized encoder-decoder graph network that achieves more robustness in graph structure learning by regularizing node representations. Our model outperforms the existing graph and non-graph based methods in various EHR predictive tasks based on both public data and real-world clinical data. Besides the improvements in empirical experiment performances, we provide an interpretation of the effect of variational regularization compared to standard graph neural network, using singular value analysis.

References

[1]

Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer Normalization. CoRR abs/1607.06450 (2016). arXiv:1607.06450 http://arxiv.org/abs/1607.06450

[2]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. http://arxiv.org/abs/1409.0473 cite arxiv:1409.0473Comment: Accepted at ICLR 2015 as oral presentation.

[3]

Muhammet Balcilar, Guillaume Renton, Pierre Heroux, Benoit Gauzere, Sebastien Adam, and Paul Honeine. 2020. Bridging the Gap Between Spectral and Spatial Domains in Graph Neural Networks. arXiv:2003.11702 [cs.LG]

[4]

Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (2001), 5--32.

Digital Library

[5]

Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. 2013. Spectral Networks and Locally Connected Networks on Graphs. CoRR abs/1312.6203 (2013).

[6]

Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David A. Sontag, and Yan Liu. 2016. Recurrent Neural Networks for Multivariate Time Series with Missing Values. CoRR abs/1606.01865 (2016). arXiv:1606.01865 http://arxiv.org/abs/1606.01865

[7]

Zhiqian Chen, Fanglan Chen, Lei Zhang, Taoran Ji, Kaiqun Fu, Liang Zhao, Feng Chen, and Chang-Tien Lu. 2020. Bridging the Gap between Spatial and Spectral Domains: A Survey on Graph Neural Networks. arXiv:2002.11867 [cs.LG]

[8]

Yu Cheng, Feng Wang, Ping Zhang, and Jianying Hu. 2016. Risk Prediction with Electronic Health Records: A Deep Learning Approach. In SDM.

[9]

Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. 2016a. Multi-Layer Representation Learning for Medical Concepts. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 1495--1504.

Digital Library

[10]

Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F. Stewart, and Jimeng Sun. 2016b. GRAM: Graph-based Attention Model for Healthcare Representation Learning. CoRR abs/1611.07012 (2016). arXiv:1611.07012 http://arxiv.org/abs/1611.07012

[11]

Edward Choi, Mohammad Taha Bahadori, and Jimeng Sun. 2015. Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. CoRR abs/1511.05942 (2015). arXiv:1511.05942 http://arxiv.org/abs/1511.05942

[12]

Edward Choi, Cao Xiao, Walter F. Stewart, and Jimeng Sun. 2018. MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare. CoRR abs/1810.09593 (2018). arXiv:1810.09593 http://arxiv.org/abs/1810.09593

Digital Library

[13]

Edward Choi, Zhen Xu, Yujia Li, Michael W. Dusenberry, Gerardo Flores, Yuan Xue, and Andrew M. Dai. 2019. Graph Convolutional Transformer: Learning the Graphical Structure of Electronic Health Records. CoRR abs/1906.04716 (2019). arXiv:1906.04716 http://arxiv.org/abs/1906.04716

[14]

Youngduck Choi, Chill Chiu, and David Sontag. 2016c. Learning Low-Dimensional Representations of Medical Concepts. AMIA Joint Summits on Translational Science proceedings. AMIA Summit on Translational Science 2016 (07 2016), 41--50.

[15]

Hsusan Chou, Jiunn-Tay Lee, Chun-Chieh Lin, Yueh-Feng Sung, Che-Chen Lin, Chih-Hsin Muo, Fu-Chi Yang, Chi Pang Wen, I-Kuan Wang, Chia-Hung Kao, Chung Hsu, and Chun-Hung Tseng. 2017. Septicemia is associated with increased risk for dementia: A population-based longitudinal study. Oncotarget 8 (09 2017).

[16]

Jean-Baptiste Cordonnier, Andreas Loukas, and Martin Jaggi. 2020. On the Relationship between Self-Attention and Convolutional Layers. In International Conference on Learning Representations. https://openreview.net/forum?id=HJlnC1rKPB

[17]

Ehsan Hajiramezanali, Arman Hasanzadeh, Nick Duffield, Krishna R Narayanan, Mingyuan Zhou, and Xiaoning Qian. 2019. Variational Graph Recurrent Neural Networks. arXiv:1908.09710 [cs.LG]

[18]

Hrayr Harutyunyan, Hrant Khachatrian, David Kale, and Aram Galstyan. 2017. Multitask Learning and Benchmarking with Clinical Time Series Data. Scientific Data 6 (03 2017).

[19]

Arman Hasanzadeh, Ehsan Hajiramezanali, Nick Duffield, Krishna R. Narayanan, Mingyuan Zhou, and Xiaoning Qian. 2019. Semi-Implicit Graph Variational AutoEncoders. arXiv:1908.07078 [cs.LG]

[20]

Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep Convolutional Networks on Graph-Structured Data. CoRR abs/1506.05163 (2015). arXiv:1506.05163 http://arxiv.org/abs/1506.05163

[21]

I. Higgins, Loïc Matthey, A. Pal, C. Burgess, Xavier Glorot, M. Botvinick, S. Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In ICLR.

[22]

Chih-Yen Hsiao, Huang-Yu Yang, Chih-Hsiang Chang, Hsing-Lin Lin, Chao-Yi Wu, Meng-Chang Hsiao, Peir-Haur Hung, Su-Hsun Liu, Cheng-Hao Weng, cheng-chia Lee, Tzung-Hai Yen, Yung-Chang Chen, and Tzu-ChinWu. 2015. Risk Factors for Development of Septic Shock in Patients with Urinary Tract Infection. BioMed Research International 2015 (07 2015), 7 pages.

[23]

Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep Unordered Composition Rivals Syntactic Methods for Text Classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Beijing, China, 1681--1691.

[24]

Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data 3 (2016), 160035.

[25]

Diederik P Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. http://arxiv.org/abs/1312.6114 cite arxiv:1312.6114.

[26]

Thomas N. Kipf and Max Welling. 2016a. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907 (2016). arXiv:1609.02907 http://arxiv.org/abs/1609.02907

[27]

Thomas N Kipf and Max Welling. 2016b. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016).

[28]

Hee Lee, Hye Seo, Hee Cha, Yun Yang, Soo Kwon, and Soo Jin Yang. 2018. Diabetes and Alzheimer's Disease: Mechanisms and Nutritional Aspects. Clinical Nutrition Research 7 (10 2018), 229.

[29]

Yikuan Li, Shishir Rao, Jose Roberto Ayala Solares, Abdelaali Hassaïne, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi Khorshidi. 2019. BEHRT: Transformer for Electronic Health Records. CoRR abs/1907.09538 (2019). arXiv:1907.09538 http://arxiv.org/abs/1907.09538

[30]

Zachary Lipton, David Kale, Charles Elkan, and Randall Wetzel. 2015. Learning to Diagnose with LSTM Recurrent Neural Networks. (11 2015).

[31]

Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. CoRR abs/1508.04025 (2015). arXiv:1508.04025 http://arxiv.org/abs/1508.04025

[32]

Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. CoRR abs/1301.3781 (2013).

[33]

Riccardo Miotto, Li Li, Brian A. Kidd, and Joel T. Dudley. 2016. Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records. Scientific Reports 6 (17 May 2016), 26094 EP -. Article.

[34]

Tom Pollard, Alistair Johnson, Jesse Raffa, Leo Celi, Roger Mark, and Omar Badawi. 2018. The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Scientific Data 5 (09 2018), 180178.

[35]

Victor Prokhorov, Ehsan Shareghi, Yingzhen Li, Mohammad Taher Pilehvar, and Nigel Collier. 2019. On the Importance of the Kullback-Leibler Divergence Term in Variational Autoencoders for Text Generation. In Proceedings of the 3rd Workshop on Neural Generation and Translation. Association for Computational Linguistics, Hong Kong, 118--127.

[36]

Sanjay Purushotham, Chuizheng Meng, Zhengping Che, and Yan Liu. 2018. Benchmarking deep learning models on large healthcare datasets. Journal of Biomedical Informatics 83 (2018), 112 -- 134.

[37]

Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jonathon Shlens. 2019. Stand-Alone Self-Attention in Vision Models. CoRR abs/1906.05909 (2019). arXiv:1906.05909 http://arxiv.org/abs/1906.05909

[38]

Ali Razavi, Aäron van den Oord, Ben Poole, and Oriol Vinyals. 2019. Preventing Posterior Collapse with delta-VAEs. CoRR abs/1901.03416 (2019). arXiv:1901.03416 http://arxiv.org/abs/1901.03416

[39]

Narges Razavian et al. 2016. Multi-task Prediction of Disease Onsets from Longitudinal Lab Tests. CoRR abs/1608.00647 (2016). arXiv:1608.00647 http://arxiv.org/abs/1608.00647

[40]

Takaya Saito and Marc Rehmsmeier. 2015. The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLOS ONE 10, 3 (03 2015), 1--21.

[41]

Benjamin Shickel, Patrick Tighe, Azra Bihorac, and Parisa Rashidi. 2017. Deep EHR: A Survey of Recent Advances on Deep Learning Techniques for Electronic Health Record (EHR) Analysis. CoRR abs/1706.03446 (2017). arXiv:1706.03446 http://arxiv.org/abs/1706.03446

[42]

Huan Song, Deepta Rajan, Jayaraman J. Thiagarajan, and Andreas Spanias. 2017. Attend and Diagnose: Clinical Time Series Analysis using Attention Models. arXiv:1711.03905 [stat.ML]

[43]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929--1958. http://jmlr.org/papers/v15/srivastava14a.html

Digital Library

[44]

Navdeep Tangri et al. 2008. Predicting technique survival in peritoneal dialysis patients: Comparing artificial neural networks and logistic regression. Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association 23 (05 2008), 2972--81.

[45]

Louis C. Tiao, Pantelis Elinas, Harrison Nguyen, and Edwin V. Bonilla. 2019. Variational Spectral Graph Convolutional Networks. CoRR abs/1906.01852 (2019). arXiv:1906.01852 http://arxiv.org/abs/1906.01852

[46]

Truyen Tran, Trang Pham, Dinh Phung, and Svetha Venkatesh. 2016. DeepCare: A Deep Dynamic Memory Model for Predictive Medicine.

[47]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 5998--6008. http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

Digital Library

[48]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. International Conference on Learning Representations (2018). https://openreview.net/forum?id=rJXMpikCZ

Cited By

Hu XSun ZNian YWang YDang YLi FFeng JYu ETao C(2024)Self-Explainable Graph Neural Network for Alzheimer Disease and Related Dementias Risk Prediction: Algorithm Development and Validation StudyJMIR Aging10.2196/547487(e54748)Online publication date: 8-Jul-2024
https://doi.org/10.2196/54748
Xie YWang KZheng JLiu FWang XHuang GHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)OEHR: An Orthopedic Electronic Health Record DatasetProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657885(1126-1135)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657885
Tang THan ZYu SBagirov AZhang Q(2024)Fuzzy Multiview Graph Learning on Sparse Electronic Health RecordsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341573032:10(5520-5532)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3415730
Show More Cited By

Index Terms

Variationally regularized graph-based representation learning for electronic health records
1. Applied computing
  1. Life and medical sciences
    1. Health informatics
2. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Learning latent representations
      2. Neural networks

Recommendations

Electronic health records: how can IS researchers contribute to transforming healthcare?

Electronic health records (EHR) facilitate integration of patient health history for planning safe and proper treatment. Combined with data analytics, aggregate-level EHR enable examination and development of effective medicines and therapies for ...
Meaningful Use of Electronic Health Records for Physician Collaboration: A Patient Centered Health Care Perspective
HICSS '14: Proceedings of the 2014 47th Hawaii International Conference on System Sciences

EHRs (Electronic Health Records), can contribute greatly to improving care and managing the rising costs of healthcare. The use and the integration of EHRs (Electronic Health Records) in supporting collaboration to increase the efficiency and ...
Concept-based electronic health records: opportunities and challenges
MM '06: Proceedings of the 14th ACM international conference on Multimedia

Healthcare is a data-rich but information-poor domain. Terabytes of multimedia medical data are being generated on a monthly basis in a typical healthcare organization in order to document patients' health status and care process. Government and health-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIL '21: Proceedings of the Conference on Health, Inference, and Learning

April 2021

309 pages

ISBN:9781450383592

DOI:10.1145/3450439

General Chair:
Marzyeh Ghassemi
University of Toronto and Vector Institute
,
Program Chairs:
Tristan Naumann
Microsoft Research Redmond
,
Emma Pierson
Stanford University and Microsoft Research New England

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ACM CHIL '21

Sponsor:

ACM

ACM CHIL '21: ACM Conference on Health, Inference, and Learning

April 8 - 10, 2021

Virtual Event, USA

Acceptance Rates

CHIL '21 Paper Acceptance Rate 27 of 110 submissions, 25%;

Overall Acceptance Rate 27 of 110 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1,848
Total Downloads

Downloads (Last 12 months)396
Downloads (Last 6 weeks)32

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hu XSun ZNian YWang YDang YLi FFeng JYu ETao C(2024)Self-Explainable Graph Neural Network for Alzheimer Disease and Related Dementias Risk Prediction: Algorithm Development and Validation StudyJMIR Aging10.2196/547487(e54748)Online publication date: 8-Jul-2024
https://doi.org/10.2196/54748
Xie YWang KZheng JLiu FWang XHuang GHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)OEHR: An Orthopedic Electronic Health Record DatasetProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657885(1126-1135)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657885
Tang THan ZYu SBagirov AZhang Q(2024)Fuzzy Multiview Graph Learning on Sparse Electronic Health RecordsIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341573032:10(5520-5532)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TFUZZ.2024.3415730
J PPrincess P(2024)Multi-Criteria-based Graph Neural Networks for a Medical Emergency Response System2024 International Conference on Inventive Computation Technologies (ICICT)10.1109/ICICT60155.2024.10544521(1071-1076)Online publication date: 24-Apr-2024
https://doi.org/10.1109/ICICT60155.2024.10544521
Verma RMugala VDharmadhikary S(2024)Advancements in Graph-Based Machine Learning for Electronic Health Record Analysis2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT61001.2024.10724775(1-7)Online publication date: 24-Jun-2024
https://doi.org/10.1109/ICCCNT61001.2024.10724775
Yang EHu PHan XNing Y(2024)MPLite: Multi-Aspect Pretraining for Mining Clinical Health Records2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825511(5096-5102)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10825511
Shen DHu CJiang XWei XZhang ZYao FWang T(2024)A Hypergraph-based temporal model for Electronic Health Records2024 10th International Conference on Big Data and Information Analytics (BigDIA)10.1109/BigDIA63733.2024.10808271(721-728)Online publication date: 25-Oct-2024
https://doi.org/10.1109/BigDIA63733.2024.10808271
Wang CYang XSun MGu YNiu JZhang W(2024)Multimodal fusion network for ICU patient outcome predictionNeural Networks10.1016/j.neunet.2024.106672180:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106672
Chan TYin GBae KYu L(2024)Multi-task heterogeneous graph learning on electronic health recordsNeural Networks10.1016/j.neunet.2024.106644(106644)Online publication date: Aug-2024
https://doi.org/10.1016/j.neunet.2024.106644
Sun MYang XNiu JGu YWang CZhang W(2024)A cross-modal clinical prediction system for intensive care unit patient outcomeKnowledge-Based Systems10.1016/j.knosys.2023.111160283(111160)Online publication date: Jan-2024
https://doi.org/10.1016/j.knosys.2023.111160
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten