Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

An end-to-end approach to combine attention feature extraction and Gaussian Process models for deep multiple instance learning in CT hemorrhage detection

Published: 08 August 2024 Publication History

Abstract

Intracranial hemorrhage (ICH) is a serious life-threatening emergency caused by blood leakage inside the brain. Radiologists usually confirm the presence of ICH by analyzing computed tomography (CT) scans, so, developing an automated diagnosis system that can process this type of images has become an important research problem. One of the main challenges to apply AI algorithms in this setting is the lack of labeled data. To mitigate the labeling burden, Multiple Instance Learning (MIL) algorithms group instances into bags, relying solely on bag-level labels for model training. Due to their capacity to handle uncertainty and deliver accurate predictions, Gaussian Processes (GPs) stand out as promising classifiers for MIL problems. Recent research has also demonstrated the effectiveness of combining attention mechanisms with GPs for ICH detection. Nonetheless, existing methods have a notable limitation: they train the attention mechanism and the GP separately, resulting in suboptimal feature extraction for GP-based classification. In this study, we introduce an innovative end-to-end MIL model that concurrently trains the CNN backbone and attention mechanism along with the GP classifier. Our approach enhances the robustness and accuracy of bag predictions by optimizing feature extraction for GP-based classification. We validate our method experimentally by focusing on two ICH detection datasets. Our results reveal a significant performance advantage in terms of accuracy, F1-score, precision, and ROC-AUC score over existing MIL approaches, especially two-stage GP approaches. Additionally, we offer empirical insights into the functionality and effectiveness of our novel model.

Highlights

End-to-end multiple instance learning (MIL) method for hemorrhage detection.
It combines attention layer, CNN and Gaussian Process for robust training.
The end-to-end MIL approach outperforms the two-phase MIL training strategy.

References

[1]
An S.J., Kim T.J., Yoon B.-W., Epidemiology, risk factors, and clinical features of intracerebral hemorrhage: An update, Journal of the Stroke 19 (1) (2017) 3–10.
[2]
Arbabshirani M.R., Fornwalt B.K., Mongelluzzo G.J., Suever J.D., Geise B.D., Patel A.A., Moore G.J., Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration, NPJ Digital Medicine 1 (1) (2018) 1–7.
[3]
Bi Q., Qin K., Li Z., Zhang H., Xu K., Xia G.-S., A multiple-instance densely-connected ConvNet for aerial scene classification, IEEE Transactions on Image Processing 29 (2020) 4911–4926,.
[4]
Bi Q., Yu S., Ji W., Bian C., Gong L., Liu H., Ma K., Zheng Y., Local-global dual perception based deep multiple instance learning for retinal disease classification, in: de Bruijne M., Cattin P.C., Cotin S., Padoy N., Speidel S., Zheng Y., Essert C. (Eds.), Medical image computing and computer assisted intervention – MICCAI 2021, Springer International Publishing, Cham, ISBN 978-3-030-87237-3, 2021, pp. 55–64.
[5]
Blomqvist K., Kaski S., Heinonen M., Deep convolutional Gaussian processes, in: Joint European conference on machine learning and knowledge discovery in databases, Springer, 2019, pp. 582–597.
[6]
Campanella G., Hanna M.G., Geneslaw L., Miraflor A., Werneck Krauss Silva V., Busam K.J., Brogi E., Reuter V.E., Klimstra D.S., Fuchs T.J., Clinical-grade computational pathology using weakly supervised deep learning on whole slide images, Nature Medical 25 (8) (2019) 1301–1309.
[7]
Campanella G., Silva V.W.K., Fuchs T.J., Terabyte-scale deep multiple instance learning for classification and localization in pathology, 2018, CoRR abs/1805.06983 arXiv:1805.06983.
[8]
Carbonneau M., Cheplygina V., Granger E., Gagnon G., Multiple instance learning: A survey of problem characteristics and applications, 2016, CoRR abs/1612.03365 arXiv:1612.03365.
[9]
Chang P.D., Kuoy E., Grinband J., Weinberg B.D., Thompson M., Homo R., Chen J., Abcede H., Shafie M., Sugrue L., Filippi C.G., Su M.-Y., Yu W., Hess C., Chow D., Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT, AJNR American Journal of the Neuroradiolorgy 39 (9) (2018) 1609–1616.
[10]
Chilamkurthy S., Ghosh R., Tanamala S., Biviji M., Campeau N.G., Venugopal V.K., Mahajan V., Rao P., Warier P., Development and validation of deep learning algorithms for detection of critical findings in head CT scans, 2018, CoRR abs/1803.05854 arXiv:1803.05854.
[11]
Gawlikowski J., Tassi C.R.N., Ali M., Lee J., Humt M., Feng J., Kruspe A.M., Triebel R., Jung P., Roscher R., Shahzad M., Yang W., Bamler R., Zhu X.X., A survey of uncertainty in deep neural networks, 2021, CoRR abs/2107.03342 arXiv:2107.03342.
[12]
Goodfellow I., Bengio Y., Courville A., Deep learning, MIT Press, 2016.
[13]
Han Z., Wei B., Hong Y., Li T., Cong J., Zhu X., Wei H., Zhang W., Accurate screening of COVID-19 using Attention-Based deep 3D multiple instance learning, IEEE Transactions on Medical Imaging 39 (8) (2020) 2584–2594.
[14]
Haußmann, M., Hamprecht, F. A., & Kandemir, M. (2017). Variational Bayesian multiple instance learning with Gaussian processes. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6570–6579).
[15]
Hensman, J., De G. Matthews, A., & Ghahramani, Z. (2015). Scalable Variational Gaussian Process Classification. In International conference on artificial intelligence and statistics (pp. 351–360).
[16]
Huang C., Chen J.-C., The Long-Term survival of intracranial hemorrhage patients successfully weaned from prolonged mechanical ventilation, International Journal of the General Medical 14 (2021) 1197–1203.
[17]
Ilse M., Tomczak J., Welling M., Attention-based deep multiple instance learning, in: International conference on machine learning - ICML, 2018, pp. 2127–2136.
[18]
Javed S.A., Juyal D., Padigela H., Taylor-Weiner A., Yu L., Prakash A., Additive MIL: Intrinsically interpretable multiple instance learning for pathology, in: Neural Information Processing Systems, 2022, URL https://api.semanticscholar.org/CorpusID:252917546.
[19]
Jnawali K., Arbabshirani M.R., Rao N., Patel A.A., Deep 3D convolution neural network for CT brain hemorrhage classification, in: Medical Imaging, 2018.
[20]
Kandemir M., Haußmann M., Diego F., Rajamani K.T., Van Der Laak J., Hamprecht F.A., Variational weakly supervised Gaussian processes, in: Wilson R.C., Hancock E.R., Smith W.A.P. (Eds.), Proceedings of the British machine vision conference (BMVC), BMVA Press, ISBN 1-901725-59-6, 2016, pp. 71.1–71.12,.
[21]
Khan M.E.E., Immer A., Abedi E., Korzepa M., Approximate inference turns deep networks into Gaussian processes, Advances in Neural Information Processing Systems 32 (2019).
[22]
Kim M., De la Torre F., Gaussian processes multiple instance learning, in: ICML, 2010.
[23]
Krishnamurthi R.V., Ikeda T., Feigin V.L., Global, regional and Country-Specific burden of ischaemic stroke, intracerebral haemorrhage and subarachnoid haemorrhage: A systematic analysis of the global burden of disease study 2017, Neuroepidemiology 54 (2) (2020) 171–179.
[24]
Kukacka J., Golkov V., Cremers D., Regularization for deep learning: A taxonomy, 2017.
[25]
Li S., Liu Y., Sui X., Chen C., Tjio G., Ting D.S.W., Goh R.S.M., Multi-instance multi-scale CNN for medical image classification, in: Shen D., Liu T., Peters T.M., Staib L.H., Essert C., Zhou S., Yap P., Khan A.R. (Eds.), Medical image computing and computer assisted intervention - MICCAI 2019 - 22nd international conference, Shenzhen, China, October 13-17, 2019, proceedings, part IV, in: Lecture notes in computer science, Vol. 11767, Springer, 2019, pp. 531–539,.
[26]
López-Pérez M., Schmidt A., Wu Y., Molina R., Katsaggelos A.K., Deep Gaussian processes for multiple instance learning: Application to CT intracranial hemorrhage detection, Computer Methods and Programs in Biomedicine (ISSN ) 219 (2022),.
[27]
Ober S.W., Aitchison L., Global inducing point variational posteriors for Bayesian neural networks and deep Gaussian processes, in: International conference on machine learning, PMLR, 2021, pp. 8248–8259.
[28]
Ober S.W., Rasmussen C.E., van der Wilk M., The promises and pitfalls of deep kernel learning, in: Uncertainty in artificial intelligence, PMLR, 2021, pp. 1206–1216.
[29]
Patel A., Van De Leemput S.C., Prokop M., Van Ginneken B., Manniesing R., Image level training and prediction: Intracranial hemorrhage identification in 3D non-contrast CT, IEEE Access 7 (2019) 92355–92364.
[30]
Phong T.D., Duong H.N., Nguyen H.T., Trong N.T., Nguyen V.H., Van Hoa T., Snasel V., Brain hemorrhage diagnosis by using deep learning, in: Proceedings of the 2017 international conference on machine learning and soft computing, Association for Computing Machinery, New York, NY, USA, ISBN 9781450348287, 2017, pp. 34–39,.
[31]
Qi S., Xu C., Li C., Tian B., Xia S., Ren J., Yang L., Wang H., Yu H., DR-MIL: deep represented multiple instance learning distinguishes COVID-19 from community-acquired pneumonia in CT images, Computer Methods Programs Biomedical 211 (2021).
[32]
Rajashekar D., Liang J.W., Intracerebral hemorrhage, 2020.
[33]
Rosner B., Glynn R.J., Lee M.-L.T., The Wilcoxon signed rank test for paired comparisons of clustered data, Biometrics 62 (1) (2006) 185–192.
[34]
Ruiz P., Morales-Álvarez P., Molina R., Katsaggelos A.K., Learning from crowds with variational Gaussian processes, Pattern Recognition 88 (2019) 298–311.
[35]
Saab K., Dunnmon J., Goldman R., Ratner A., Sagreiya H., Ré C., Rubin D., Doubly weak supervision of deep learning models for head CT, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2019, pp. 811–819.
[36]
Salimbeni H., Dutordoir V., Hensman J., Deisenroth M., Deep Gaussian processes with importance-weighted variational inference, in: International Conference on Machine Learning, PMLR, 2019, pp. 5589–5598.
[37]
Sato D., Hanaoka S., Nomura Y., Takenaga T., Miki S., Yoshikawa T., Hayashi N., Abe O., A primitive study on unsupervised anomaly detection with an autoencoder in emergency head CT volumes, in: Medical Imaging 2018: Computer-Aided Diagnosis, 10575, International Society for Optics and Photonics, 2018, p. 105751P.
[38]
Schmidt A., Morales-Álvarez P., Molina R., Probabilistic attention based on Gaussian processes for deep multiple instance learning, IEEE Transactions on Neural Networks and Learning Systems (2023) 1–14,.
[39]
Snelson E., Ghahramani Z., Sparse Gaussian processes using Pseudo-inputs, in: Advances in Neural Information Processing Systems, 18, 2006.
[40]
Struski Y., Rymarczyk D., Lewicki A., Sabiniewicz R., Tabor J., Zieliński B., ProMIL: Probabilistic multiple instance learning for medical imaging, 2023, arXiv:2306.10535.
[41]
Svendsen D.H., Morales-Álvarez P., Ruescas A.B., Molina R., Camps-Valls G., Deep Gaussian processes for biogeophysical parameter retrieval and model inversion, ISPRS Journal of Photogrammetry and Remote Sensing 166 (2020) 68–81.
[42]
Titano J.J., Badgeley M., Schefflein J., Pain M., Su A., Cai M., Swinburne N., Zech J., Kim J., Bederson J., Mocco J., Drayer B., Lehar J., Cho S., Costa A., Oermann E.K., Automated deep-neural-network surveillance of cranial images for acute neurologic events, Nat Med 24 (9) (2018) 1337–1341.
[43]
Wang F., Pinar A., The multiple instance learning Gaussian process probit model, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2021, pp. 3034–3042.
[44]
Wang X., Yan Y., Tang P., Bai X., Liu W., Revisiting multiple instance neural networks, Pattern Recognition 74 (2018) 15–24.
[45]
Williams C., Rasmussen C., Gaussian processes for machine learning, Vol. 2, MIT press Cambridge, MA, 2006.
[46]
Wilson A.G., Hu Z., Salakhutdinov R.R., Xing E.P., Stochastic variational deep kernel learning, Advances in Neural Information Processing Systems 29 (2016).
[47]
Wu Y., Castro-Macías F.M., Morales-Álvarez P., Molina R., Katsaggelos A.K., Smooth attention for deep multiple instance learning: Application to CT intracranial hemorrhage detection, 2023, arXiv preprint arXiv:2307.09457.
[48]
Wu Y., Schmidt A., Hernández-Sánchez E., Molina R., Katsaggelos A.K., Combining attention-based multiple instance learning and Gaussian processes for CT hemorrhage detection, in: Medical image computing and computer assisted intervention – MICCAI, ISBN 978-3-030-87195-6, 2021, pp. 582–591,.
[49]
Wu Z., Yang Y., Gu J., Tresp V., Quantifying predictive uncertainty in medical image analysis with deep kernel learning, in: 2021 IEEE 9th international conference on healthcare informatics (ICHI), 2021, pp. 63–72,.
[50]
Yao J., Zhu X., Jonnagaddala J., Hawkins N., Huang J., Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks, Medical Image Analysis (ISSN ) 65 (2020),. URL https://www.sciencedirect.com/science/article/pii/S1361841520301535.
[51]
Yousefi F., Smith M.T., Alvarez M., Multi-task learning for aggregated data using Gaussian processes, Advances in Neural Information Processing Systems 32 (2019).

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal
Expert Systems with Applications: An International Journal  Volume 240, Issue C
Apr 2024
1601 pages

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 08 August 2024

Author Tags

  1. End-to-end multiple instance learning
  2. Gaussian process
  3. Attention
  4. CT hemorrhage detection

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 06 Oct 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media