Abstract
The performance of multi-task learning hinges on the design of feature sharing between tasks; a process which is combinatorial in the network depth and task count. Hand-crafting an architecture based on human intuitions of task relationships is therefore suboptimal. In this paper, we present a probabilistic approach to learning task-specific and shared representations in Convolutional Neural Networks (CNNs) for multi-task learning of semantic tasks. We introduce Stochastic Filter Groups; which is a mechanism that groups convolutional kernels into task-specific and shared groups to learn an optimal kernel allocation. They facilitate learning optimal shared and task specific representations. We employ variational inference to learn the posterior distribution over the possible grouping of kernels and CNN weights. Experiments on MRI-based prostate radiotherapy organ segmentation and CT synthesis demonstrate that the proposed method learns optimal task allocations that are inline with human-optimised networks whilst improving performance over competing baselines.
F. J. S. Bragman and R. Tanno—Contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Singla, S., Gong, M., Ravanbakhsh, S., Sciurba, F., Poczos, B., Batmanghelich, K.N.: Subject2Vec: generative-discriminative approach from a set of image patches to a vector. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 502–510. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_57
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)
Bragman, F., et al.: Uncertainty in multitask learning: joint representations for probabilistic MR-only radiotherapy planning. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_1
Tanno, R., et al.: AutoDVT: joint real-time classification for vein compressibility analysis in deep vein thrombosis ultrasound diagnostics. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 905–912. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_100
Caruana, R.: Multitask learning. Mach. Learn. 28(1), 41–75 (1997)
Meyerson, E., Miikkulainen, R.: Beyond shared hierarchies: deep multitask learning through soft layer ordering. In: ICLR (2018)
Misra, I., Shrivastava, A., Gupta, A., Hebert, M.: Cross-stitch networks for multi-task learning. In: CVPR (2016)
Zamir, A.R., Sax, A., Shen, W.B., Guibas, L.J., Malik, J., Savarese, S.: Taskonomy: disentangling task transfer learning. In: CVPR, IEEE (2018)
Ruder, S., Bingel, J., Augenstein, I., Søgaard, A.: Latent multi-task architecture learning (2019)
Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., Feris, R.S.: Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. In: CVPR (2017)
Mejjati, Y.A., Cosker, D., In Kim, K.: Multi-task learning by maximizing statistical dependence. In: CVPR (2018)
Ioannou, Y., Robertson, D., Cipolla, R., Criminisi, A., et al.: Deep roots: improving CNN efficiency with hierarchical filter groups (2017)
Gal, Y., Hron, J., Kendall, A.: Concrete dropout. In: NIPS, pp. 3581–3590 (2017)
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016)
Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., Vercauteren, T.: On the compactness, efficiency, and representation of 3D convolutional networks: brain parcellation as a pretext task (2017)
Acknowledgements
FB and MJC were supported by CRUK Grant A21993. RT was supported by Microsoft Scholarship. DA was supported by EU Horizon 2020 Research and Innovation Grant 666992, EPSRC Grant M020533, R014019, and R006032, and the NIHR UCLH BRC. We thank NVIDIA Corporation for hardware donation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Bragman, F.J.S., Tanno, R., Ourselin, S., Alexander, D.C., Cardoso, M.J. (2019). Learning Task-Specific and Shared Representations in Medical Imaging. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11767. Springer, Cham. https://doi.org/10.1007/978-3-030-32251-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-32251-9_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32250-2
Online ISBN: 978-3-030-32251-9
eBook Packages: Computer ScienceComputer Science (R0)
