Abstract
Modeling the mapping between mind and brain is the key towards understanding how brain works. More specifically, the question can be formatted as modeling the posterior distribution of the latent psychological state given the observed brain, and the likelihood of brain observation given the latent psychological state. Generative adversarial network (GAN) is known for learning implicitly distributions over data which are hard to model with an explicit likelihood. To utilize GAN for the brain mapping modeling, we propose a novel representation learning framework to explore brain representations of different functions. With a linear regression, the learned representations are interpreted as functional brain networks (FBNs), which characterize the mapping between mind and brain. The proposed framework is evaluated on Human Connectome Project (HCP) task functional MRI (tfMRI) data. This novel framework proves that GAN can learn meaningful representations of tfMRI and promises better understanding of the brain function.
Q. Dong and N. Qiang ━ Equally contribution to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Huettel, S.A., et al.: Functional Magnetic Resonance Imaging, vol. 1. Sinauer Associates Sunderland, MA (2004)
Shimony, J.S., et al.: Resting-state spontaneous fluctuations in brain activity: a new paradigm for presurgical planning using fMRI. Acad. Radiol. 16(5), 578–583 (2009)
Smith, S.M., et al.: Correspondence of the brain’s functional architecture during activation and rest. Proc. Nat. Acad. Sci. 106(31), 13040–13045 (2009)
Kanwisher, N.: Functional specificity in the human brain: a window into the functional architecture of the mind. Proc. Nat. Acad. Sci. 107(25), 11163–11170 (2010)
Pessoa, L.: Understanding brain networks and brain organization. Phys. Rev. 11(3), 400–435 (2014)
Archbold, K.H., et al.: Neural activation patterns during working memory tasks and OSA disease severity: preliminary findings. J. Clin. Sleep Med. 5(01), 21–27 (2009)
Barch, D.M., et al.: Function in the human connectome: task-fMRI and individual differences in behavior. Neuroimage 80, 169–189 (2013)
Binder, J.R., et al.: Mapping anterior temporal lobe language areas with fMRI: a multicenter normative study. Neuroimage 54(2), 1465–1475 (2011)
Dosenbach, N.U., et al.: A core system for the implementation of task sets. Neuron 50(5), 799–812 (2006)
Beckmann, C.F., et al.: General multilevel linear modeling for group analysis in FMRI. Neuroimage 20(2), 1052–1063 (2003)
McKeown, M.J.: Detection of consistently task-related activations in fMRI data with hybrid independent component analysis. NeuroImage 11(1), 24–35 (2000)
Beckmann, C.F., et al.: Investigations into resting-state connectivity using independent component analysis. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360(1457), 1001–1013 (2005)
Calhoun, V.D., et al.: A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. Neuroimage 45(1), S163–S172 (2009)
Calhoun, V.D., et al.: Multisubject independent component analysis of fMRI: a decade of intrinsic networks, default mode, and neurodiagnostic discovery. IEEE Rev. Biomed. Eng. 5, 60–73 (2012)
Jiang, X., et al.: Sparse representation of HCP grayordinate data reveals novel functional architecture of cerebral cortex. Hum. Brain Mapp. 36(12), 5301–5319 (2015)
Lv, J., et al.: Holistic atlases of functional networks and interactions reveal reciprocal organizational architecture of cortical function. IEEE Trans. Biomed. Eng. 62(4), 1120–1131 (2015)
Li, X., et al.: Multple-demand system identification and characterization via sparse representations of fMRI data. In: Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on. IEEE (2016)
Ge, F., et al.: Exploring intrinsic networks and their interactions using group wise temporal sparse coding. In: Biomedical Imaging (ISBI 2018), 2018 IEEE 15th International Symposium on. IEEE (2018)
Ge, F., et al.: Deriving ADHD biomarkers with sparse coding based network analysis. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). IEEE (2015)
Zhao, Y., et al.: Connectome-scale group-wise consistent resting-state network analysis in autism spectrum disorder. NeuroImage Clin. 12, 23–33 (2016)
Grabner, G., Janke, A.L., Budge, M.M., Smith, D., Pruessner, J., Collins, D.L.: Symmetric atlasing and model based segmentation: an application to the hippocampus in older adults. In: Larsen, R., Nielsen, M., Sporring, J. (eds.) MICCAI 2006. LNCS, vol. 4191, pp. 58–66. Springer, Heidelberg (2006). https://doi.org/10.1007/11866763_8
Huang, H., et al.: Modeling task fMRI data via mixture of deep expert networks. In: Biomedical Imaging (ISBI 2018), 2018 IEEE 15th International Symposium on. IEEE (2018)
Huang, H., et al.: Modeling task fMRI data via deep convolutional autoencoder. IEEE Trans. Med. Imaging 37(7), 1551–1561 (2018)
Zhao, Y., et al.: 4D modeling of fMRI data via spatio-temporal convolutional neural networks (ST-CNN). In: IEEE Transactions on Cognitive and Developmental Systems. IEEE (2019)
Li, Q., et al.: Simultaneous spatial-temporal decomposition of connectome-scale brain networks by deep sparse recurrent auto-encoders. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 579–591. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_45
Wang, H., et al.: Recognizing brain states using deep sparse recurrent neural network. In: IEEE transactions on medical imaging. IEEE (2018)
Ng, A.Y., et al.: On discriminative vs. generative classifiers: a comparison of logistic regression and naive bayes. In: Advances in neural information processing systems, (2002)
Calimeri, F., Marzullo, A., Stamile, C., Terracina, G.: Biomedical data augmentation using generative adversarial neural networks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017. LNCS, vol. 10614, pp. 626–634. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68612-7_71
Nie, Dong., et al.: Medical image synthesis with context-aware generative adversarial networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 417–425. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_48
Chen, X., et al.: Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in neural information processing systems, (2016)
Costa, P., et al.: Towards adversarial retinal image synthesis. arXiv preprint, (2017) arXiv:1701.08974
Dimsdale-Zucker, H.R., et al.: Representational similarity analyses: a practical guide for functional MRI applications. In: Handbook of Behavioral Neuroscience, Elsevier. pp. 509–525 (2019)
Guibas, J.T., et al.: Synthetic medical images from dual generative adversarial networks. arXiv preprint, (2017) arXiv:1709.01872
Radford, A., et al.: Unsupervised representation learning with deep convolutional generative adversarial networks. (2015)
Bengio, Y., et al.: Representation learning: a review and new perspectives. 35(8), 1798–1828 (2013)
Bengio, Y., et al.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Barch, D.M., et al.: Function in the human connectome: task-fMRI and individual differences in behavior. Neuroimage 80, 169–189 (2013)
Jenkinson, M., et al.: Fsl. Neuroimage 62(2), 782–790 (2012)
Kingma, D.P., et al.: Adam: a method for stochastic optimization. arXiv preprint, (2014) arXiv:1412.6980
Arjovsky, M., et al.: Wasserstein gan, (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Dong, Q. et al. (2020). A Novel fMRI Representation Learning Framework with GAN. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds) Machine Learning in Medical Imaging. MLMI 2020. Lecture Notes in Computer Science(), vol 12436. Springer, Cham. https://doi.org/10.1007/978-3-030-59861-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-59861-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59860-0
Online ISBN: 978-3-030-59861-7
eBook Packages: Computer ScienceComputer Science (R0)
