Semi-supervised biomedical translation with cycle wasserstein regression GANs
AUTHORs: 
Matthew B. A. McDermott, Tom Yan, 
Tristan Naumann, Nathan Hunt, Harini Suresh, 
Peter Szolovits, Marzyeh GhassemiAuthors Info & Claims
Matthew B. A. McDermott, Tom Yan, Tristan Naumann, Nathan Hunt, Harini Suresh, Peter Szolovits, Marzyeh GhassemiAuthors Info & ClaimsArticle No.: 288, Pages 2363 - 2370
Abstract
The biomedical field offers many learning tasks that share unique challenges: large amounts of unpaired data, and a high cost to generate labels. In this work, we develop a method to address these issues with semi-supervised learning in regression tasks (e.g., translation from source to target). Our model uses adversarial signals to learn from unpaired data-points, and imposes a cycle-loss reconstruction error penalty to regularize mappings in either direction against one another. We first evaluate our method on synthetic experiments, demonstrating two primary advantages of the system: 1) distribution matching via the adversarial loss and 2) regularization towards invertible mappings via the cycle loss. We then show a regularization effect and improved performance when paired data is supplemented by additional unpaired data on two real biomedical regression tasks: estimating the physiological effect of medical treatments, and extrapolating gene expression (transcriptomics) signals. Our proposed technique is a promising initial step towards more robust use of adversarial signals in semi-supervised regression, and could be useful for other tasks (e.g., causal inference or modality translation) in the biomedical field.
References
[1]
Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G. S.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Goodfellow, I.; Harp, A.; Irving, G.; Isard, M.; Jia, Y.; Jozefowicz, R.; Kaiser, L.; Kudlur, M.; Levenberg, J.; Mane, D.; Monga, R.; Moore, S.; Murray, D.; Olah, C.; Schuster, M.; Shlens, J.; Steiner, B.; Sutskever, I.; Talwar, K.; Tucker, P.; Vanhoucke, V.; Vasudevan, V.; Viegas, F.; Vinyals, O.; Warden, P.; Wattenberg, M.; Wicke, M.; Yu, Y.; and Zheng, X. 2016. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv:1603.04467 [cs]. arXiv: 1603.04467.
[2]
Arjovsky, M.; Chintala, S.; and Bottou, L. 2017. Wasserstein GAN. arXiv:1701.07875 [cs, stat]. arXiv: 1701.07875.
[3]
Bellemare, M. G.; Danihelka, I.; Dabney, W.; Mohamed, S.; Lakshminarayanan, B.; Hoyer, S.; and Munos, R. 2017. The Cramer Distance as a Solution to Biased Wasserstein Gradients. arXiv:1705.10743 [cs, stat]. arXiv: 1705.10743.
[4]
Broad Connectivity Map Team. 2016. L1000 connectivity map perturbational profiles from broad institute lincs center for transcriptomics lincs phase II.
[5]
Che, Z.; Purushotham, S.; Cho, K.; Sontag, D.; and Liu, Y. 2016. Recurrent Neural Networks for Multivariate Time Series with Missing Values. arXiv:1606.01865 [cs, stat]. arXiv: 1606.01865.
[6]
Choi, E.; Biswal, S.; Malin, B.; Duke, J.; Stewart, W. F.; and Sun, J. 2017. Generating Multi-label Discrete Electronic Health Records using Generative Adversarial Networks. arXiv:1703.06490 [cs]. arXiv: 1703.06490.
[7]
D'Aragon, F.; Belley-Cote, E. P.; Meade, M. O.; Lauzier, F.; Adhikari, N. K.; Briel, M.; Lalu, M.; Kanji, S.; Asfar, P.; Turgeon, A. F.; et al. 2015. Blood pressure targets for vasopressor therapy: A systematic review. Shock 43(6):530-539.
[8]
Eisen, M. B.; Spellman, P. T.; Brown, P. O.; and Botstein, D. 1998. Cluster analysis and display of genome-wide expression patterns. Proceedings of the National Academy of Sciences 95(25):14863-14868.
[9]
Esteban, C.; Hyland, S. L.; and Rtsch, G. 2017. Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs. arXiv:1706.02633 [cs, stat]. arXiv: 1706.02633.
[10]
Ghassemi, M.; Naumann, T.; Doshi-Velez, F.; Brimmer, N.; Joshi, R.; Rumshisky, A.; and Szolovits, P. 2014. Unfolding Physiological State: Mortality Modelling in Intensive Care Units. In Proc. of the 20th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, KDD '14, 75-84. New York, NY, USA: ACM.
[11]
Ghassemi, M.; Wu, M.; Hughes, M. C.; Szolovits, P.; and Doshi-Velez, F. 2017. Predicting intervention onset in the ICU with switching state space models. AMIA Summits on Trans. Sci. Proc. 2017:82-91.
[12]
Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2014. Generative Adversarial Nets. In Ghahramani, Z.; Welling, M.; Cortes, C.; Lawrence, N. D.; and Weinberger, K. Q., eds., Advances in Neural Information Processing Systems 27. Curran Associates, Inc. 2672-2680.
[13]
Goodfellow, I. 2016. NIPS 2016 Tutorial: Generative Adversarial Networks. arXiv:1701.00160 [cs]. arXiv: 1701.00160.
[14]
Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; and Courville, A. 2017. Improved Training of Wasserstein GANs. arXiv:1704.00028 [cs, stat]. arXiv: 1704.00028.
[15]
Isola, P.; Zhu, J.; Zhou, T.; and Efros, A. A. 2016. Image-to-image translation with conditional adversarial networks. CoRR abs/1611.07004.
[16]
Johnson, A. E.; Pollard, T. J.; Shen, L.; Lehman, L.-w. H.; Feng, M.; Ghassemi, M.; Moody, B.; Szolovits, P.; Anthony Celi, L.; and Mark, R. G. 2016. MIMIC-III, a freely accessible critical care database. Scientific Data 3.
[17]
Kingma, D. P., and Ba, J. 2014. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs]. arXiv: 1412.6980.
[18]
Lipton, Z. C.; Kale, D. C.; Elkan, C.; and Wetzel, R. 2015. Learning to Diagnose with LSTM Recurrent Neural Networks. arXiv:1511.03677 [cs]. arXiv: 1511.03677.
[19]
Ngiam, J.; Khosla, A.; Kim, M.; Nam, J.; Lee, H.; and Ng, A. Y. 2011. Multimodal deep learning.
[20]
Reed, S.; Akata, Z.; Yan, X.; Logeswaran, L.; Schiele, B.; and Lee, H. 2016. Generative Adversarial Text to Image Synthesis. In PMLR, 1060-1069.
[21]
Salimans, T.; Goodfellow, I.; Zaremba, W.; Cheung, V.; Radford, A.; Chen, X.; and Chen, X. 2016. Improved Techniques for Training GANs. In Lee, D. D.; Sugiyama, M.; Luxburg, U. V.; Guyon, I.; and Garnett, R., eds., Advances in Neural Information Processing Systems 29. Curran Associates, Inc. 2234-2242.
[22]
Sohn, K.; Shang, W.; and Lee, H. 2014. Improved Multimodal Deep Learning with Variation of Information. In Ghahramani, Z.; Welling, M.; Cortes, C.; Lawrence, N. D.; and Weinberger, K. Q., eds., Advances in Neural Information Processing Systems 27. Curran Associates, Inc. 2141-2149.
[23]
Springenberg, J. T. 2015. Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks. arXiv:1511.06390 [cs, stat]. arXiv: 1511.06390.
[24]
Subramanian, A.; Narayan, R.; Corsello, S. M.; Peck, D. D.; Natoli, T. E.; Lu, X.; Gould, J.; Davis, J. F.; Tubelli, A. A.; Asiedu, J. K.; Lahr, D. L.; Hirschman, J. E.; Liu, Z.; Donahue, M.; Julian, B.; Khan, M.; Wadden, D.; Smith, I.; Lam, D.; Liberzon, A.; Toder, C.; Bagul, M.; Orzechowski, M.; Enache, O. M.; Piccioni, F.; Berger, A. H.; Shamji, A.; Brooks, A. N.; Vrcic, A.; Flynn, C.; Rosains, J.; Takeda, D.; Davison, D.; Lamb, J.; Ardlie, K.; Hogstrom, L.; Gray, N. S.; Clemons, P. A.; Silver, S.; Wu, X.; Zhao, W.-N.; Read-Button, W.; Wu, X.; Haggarty, S. J.; Ronco, L. V.; Boehm, J. S.; Schreiber, S. L.; Doench, J. G.; Bittker, J. A.; Root, D. E.; Wong, B.; and Golub, T. R. 2017. A Next Generation Connectivity Map: L1000 Platform And The First 1,000,000 Profiles. bioRxiv 136168.
[25]
Sundararajan, K.; Flabouris, A.; and Thompson, C. 2016. Diurnal variation in the performance of rapid response systems: the role of critical care servicesa review article. Journal of intensive care 4(1):15.
[26]
Suresh, H.; Szolovits, P.; and Ghassemi, M. 2017. The Use of Autoencoders for Discovering Patient Phenotypes. arXiv:1703.07004 [cs]. arXiv: 1703.07004.
[27]
Tobin, M. J. 2006. Principles and practice of mechanical ventilation. LWW.
[28]
Wu, M.; Ghassemi, M.; Feng, M.; Celi, L. A.; Szolovits, P.; and Doshi-Velez, F. 2017. Understanding vasopressor intervention and weaning: risk prediction in a public heterogeneous clinical time series database. J of the American Med Inform Assoc 24(3):488-495.
[29]
Yang, K. L., and Tobin, M. J. 1991. A prospective study of indexes predicting the outcome of trials of weaning from mechanical ventilation. New England Journal of Medicine 324.
[30]
Zhu, J.-Y.; Park, T.; Isola, P.; and Efros, A. A. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. arXiv:1703.10593 [cs]. arXiv: 1703.10593.
Index Terms
- Semi-supervised biomedical translation with cycle wasserstein regression GANs
Index terms have been assigned to the content through auto-classification.
Recommendations
Sparse Representation-Based Semi-Supervised Regression for People Counting
Label imbalance and the insufficiency of labeled training samples are major obstacles in most methods for counting people in images or videos. In this work, a sparse representation-based semi-supervised regression method is proposed to count people in ...
Semi-Supervised Kernel Regression
ICDM '06: Proceedings of the Sixth International Conference on Data MiningInsufficiency of training data is a major obstacle in machine learning and data mining applications. Many different semi-supervised learning algorithms have been proposed to tackle this difficulty by leveraging a large amount of unlabeled data. However, ...
Semi-supervised regression with co-training
IJCAI'05: Proceedings of the 19th international joint conference on Artificial intelligenceIn many practical machine learning and data mining applications, unlabeled training examples are readily available but labeled ones are fairly expensive to obtain. Therefore, semi-supervised learning algorithms such as co-training have attracted much ...
Comments
Information & Contributors
Information
Published In
Copyright © 2018 Association for the Advancement of Artificial Intelligence.
Sponsors
- Association for the Advancement of Artificial Intelligence
Publisher
AAAI Press
Publication History
Published: 02 February 2018
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 48Total Downloads
- Downloads (Last 12 months)23
- Downloads (Last 6 weeks)6
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in