Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

BG-3DM2F: Bidirectional gated 3D multi-scale feature fusion for Alzheimer’s disease diagnosis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A computer-aided diagnosis system is one of the crucial decision support tools under the medical imaging scope. It has recently emerged as a powerful way to diagnose Alzheimer’s Disease (AD) from structural magnetic resonance imaging scans. However, due to the deficit of recognition memory in the Mild Cognitive Impairment (MCI) stage, semantic feature ambiguity, and high inter-class visual similarities problems, computer-aided diagnosis of AD remains challenging. To bridge these gaps, this paper proposed a hippocampus analysis method based on a novel 3D convolutional neural network fusion strategy, called Bidirectional Gated 3D Multi-scale Feature Fusion (BG-3DM2F). The suggested BG-3DM2F framework consists of two modules: 3D Multi-Scale Chained Network (3DMS-ChaineNet) and Bidirectional Gated Recurrent Fusion Unit (Bi-GRFU). The 3DMS-ChaineNet architecture is introduced to design the subtle features and capture the variations in hippocampal atrophy, while the Bi-GRFU scheme is investigated to store 3DMS-ChaineNet levels in the forward and backward fashion and retain them in the decision-making process. For validation, our solution is completely evaluated on the public Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset. Practically, we conducted empirical evaluations to verify the effect of BG-3DM2F components. In comparison with the current state-of-the-art methods, the experiments show that our proposed approach provides efficient results, achieving the accuracies of 98.12%, 95.26%, and 96.97% for binary classification of Normal Control (NC) versus AD, AD versus MCI, and NC versus MCI, respectively. Therefore, we can conclude that our proposed BG-3DM2F system has the potential to dramatically improve the conventional classification methods for assisting clinical decision-making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abdelaziz Ismael S, Mohammed A, Hefny H (2020) An enhanced deep learning approach for brain cancer MRI images classification using residual networks. Artif Intell Med 102:101779. https://doi.org/10.1016/j.artmed.2019.101779

    Article  Google Scholar 

  2. Aderghal K, Boissenin M, Benois-Pineau J, Catheline G, Afdel K (2016) Classification of sMRI for AD Diagnosis with Convolutional Neuronal Networks: A Pilot 2-D+ 𝜖 Study on ADNI. MultiMedia Modeling 690–701. https://doi.org/10.1007/978-3-319-51811-4∖_56

  3. Aderghal K, Khvostikov A, Krylov A, Benois-Pineau J, Afdel K, Catheline G (2018) Classification of Alzheimer disease on imaging modalities with deep CNNs using cross-modal transfer learning. In: 2018 IEEE 31st international symposium on computer-based medical systems (CBMS). https://doi.org/10.1109/cbms.2018.00067

  4. Amin S, Alsulaiman M, Muhammad G, Mekhtiche M, Shamim Hossain M (2019) Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion. Futur Gener Comput Syst 101:542–554. https://doi.org/10.1016/j.future.2019.06.027

    Article  Google Scholar 

  5. Arifoglu D, Bouchachia A (2019) Detection of abnormal behaviour for dementia sufferers using Convolutional Neural Networks. Artif Intell Med 94:88–95. https://doi.org/10.1016/j.artmed.2019.01.005

    Article  Google Scholar 

  6. Bakkouri I, Afdel K (2018) Multi-scale CNN based on region proposals for efficient breast abnormality recognition. Multimed Tools Appl 78(10):12939–12960. https://doi.org/10.1007/s11042-018-6267-z

    Article  Google Scholar 

  7. Bakkouri I, Afdel K (2019) Computer-aided diagnosis (CAD) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images. Multimed Tools Appl 79(29-30):20483–20518. https://doi.org/10.1007/s11042-019-07988-1

    Article  Google Scholar 

  8. Bakkouri I, Afdel K, Benois-Pineau J, Catheline G (2019) Recognition of alzheimer’s disease on sMRI based on 3D multi-scale CNN features and a gated recurrent fusion unit. In: 2019 international conference on content-based multimedia indexing (CBMI). https://doi.org/10.1109/cbmi.2019.8877477

  9. Baldi P, Sadowski P (2014) The dropout learning algorithm. Artif Intell 210:78–122. https://doi.org/10.1016/j.artint.2014.02.004

    Article  MathSciNet  MATH  Google Scholar 

  10. Banning L, Ramakers I, Köhler S, Bron E, Verhey F, de Deyn P, Claassen J, Koek H, Middelkoop H, van der Flier W, van der Lugt A, Aalten P (2020) The association between biomarkers and neuropsychiatric symptoms across the alzheimer’s disease spectrum. Amer J Geriatric Psychiatry 28:735–744. https://doi.org/10.1016/j.jagp.2020.01.012

    Article  Google Scholar 

  11. Basaia S, Agosta F, Wagner L, Canu E, Magnani G, Santangelo R, Filippi M (2019) Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks. NeuroImage: Clinical 21:101645. https://doi.org/10.1016/j.nicl.2018.101645

    Article  Google Scholar 

  12. Ben Ahmed O, Benois-Pineau J, Allard M, Ben Amar C, Catheline G (2014) Classification of Alzheimer’s disease subjects from MRI using hippocampal visual features. Multimed Tools Appl 74:1249–1266. https://doi.org/10.1007/s11042-014-2123-y

    Article  Google Scholar 

  13. Ben Ahmed O, Benois-Pineau J, Allard M, Catheline G, Ben Amar C (2017) Recognition of Alzheimer’s disease and Mild Cognitive Impairment with multimodal image-derived biomarkers and Multiple Kernel Learning. Neurocomputing 220:98–110. https://doi.org/10.1016/j.neucom.2016.08.041

    Article  Google Scholar 

  14. Ben Ahmed O, Mizotin M, Benois-Pineau J, Allard M, Catheline G, Ben Amar C (2015) Alzheimer’s disease diagnosis on structural MR images using circular harmonic functions descriptors on hippocampus and posterior cingulate cortex. Comput Med Imaging Graph 44:13–25. https://doi.org/10.1016/j.compmedimag.2015.04.007

    Article  Google Scholar 

  15. Ben Miled Z, Haas K, Black C, Khandker R, Chandrasekaran V, Lipton R, Boustani M (2020) Predicting dementia with routine care EMR data. Artif Intell Med 102:101771. https://doi.org/10.1016/j.artmed.2019.101771

    Article  Google Scholar 

  16. Budson A, Solomon P (2016) Evaluating the Patient with Memory Loss or Dementia. Memory Loss Alzheimer’s Disease, and Dementia, 5–38. https://doi.org/10.1016/b978-0-323-28661-9.00002-0

  17. Budson A, Solomon P (2016) Alzheimer’s disease dementia and mild cognitive impairment due to alzheimer’s disease. Memory loss Alzheimer’s Disease, and Dementia, 47–69. https://doi.org/10.1016/b978-0-323-28661-9.00004-4

  18. Bui T, Lee J, Shin J (2019) Incorporated region detection and classification using deep convolutional networks for bone age assessment. Artif Intell Med 97:1–8. https://doi.org/10.1016/j.artmed.2019.04.005

    Article  Google Scholar 

  19. Cahall D, Rasool G, Bouaynaya N, Fathallah-Shaykh H (2019) Inception Modules Enhance Brain Tumor Segmentation. Frontiers in Computational Neuroscience. https://doi.org/10.3389/fncom.2019.00044

  20. Castro A, Coria-Lucero C, Anzulovich A, Navigatore-Fonzo L, 2019 Effects of experimental intracerebral ventricular injection of amyloid beta peptide (1-42) aggregates on daily rhythms of A-degrading enzymes in the hippocampus: Relevance to Alzheimer’s disease pathophysiology. Pathophysiology. https://doi.org/10.1016/j.pathophys.2019.07.003

  21. Cattaud V, Bezzina C, Rey C, Lejards C, Dahan L, Verret L (2018) Early disruption of parvalbumin expression and perineuronal nets in the hippocampus of the Tg2576 mouse model of Alzheimer’s disease can be rescued by enriched environment. Neurobiol Aging 72:147–158. https://doi.org/10.1016/j.neurobiolaging.2018.08.024

    Article  Google Scholar 

  22. Cheng D, Liu M (2017) Classification of Alzheimer’s Disease by Cascaded Convolutional Neural Networks Using PET Images. Machine Learning in Medical Imaging, 106–113. https://doi.org/10.1007/978-3-319-67389-9∖_13

  23. Cheng H, Xie Z, Wu L, Yu Z, Li R (2019) Data prediction model in wireless sensor networks based on bidirectional LSTM. EURASIP Journal on Wireless Communications and Networking. https://doi.org/10.1186/s13638-019-1511-4

  24. Cogan T, Cogan M, Tamil L (2019) MAPGI: Accurate Identification of anatomical landmarks and diseased tissue in gastrointestinal tract using deep learning. Comput Biol Med 111:103351. https://doi.org/10.1016/j.compbiomed.2019.103351

    Article  Google Scholar 

  25. Cruz-Alonso M, Fernandez B, Navarro A, Junceda S, Astudillo A, Pereiro R (2019) Laser ablation ICP-MS for simultaneous quantitative imaging of iron and ferroportin in hippocampus of human brain tissues with Alzheimer’s disease. Talanta 197:413–421. https://doi.org/10.1016/j.talanta.2019.01.056

    Article  Google Scholar 

  26. Cui R, Liu M (2018) Hippocampus analysis based on 3D CNN for Alzheimer’s disease diagnosis. Tenth International Conference on Digital Image Processing (ICDIP), 2018. https://doi.org/10.1117/12.2503194

  27. Cui R, Liu M (2019) Hippocampus analysis by combination of 3-D DenseNet and shapes for Alzheimer’s disease diagnosis. IEEE J Biomed Health Inform 23:2099–2107. https://doi.org/10.1109/jbhi.2018.2882392

    Article  Google Scholar 

  28. Dai Y, Wang C, Dong J, Sun C (2019) Visual relationship detection based on bidirectional recurrent neural network. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-019-7732-z

  29. Dey R, Salemt F (2017) Gate-variants of Gated Recurrent Unit (GRU) neural networks. 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS). https://doi.org/10.1109/mwscas.2017.8053243

  30. Du X, Qu X, He Y, Guo D (2018) Single image Super-Resolution based on Multi-Scale competitive convolutional neural network. Sensors 18:789. https://doi.org/10.3390/s18030789

    Article  Google Scholar 

  31. Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Machine Learn Res 12:2121–2159

    MathSciNet  MATH  Google Scholar 

  32. Esmaeilzadeh S, Belivanis D, Pohl K, Adeli E (2018) End-To-End Alzheimer’s Disease Diagnosis and Biomarker Identification. Machine Learning in Medical Imaging, 337–345. https://doi.org/10.1007/978-3-030-00919-9∖_39

  33. Fawzi A, Moosavi-Dezfooli S, Frossard P (2017) The robustness of deep networks: a geometrical perspective. IEEE Signal Proc Mag 34:50–62. https://doi.org/10.1109/msp.2017.2740965

    Article  Google Scholar 

  34. Feng C, Elazab A, Yang P, Wang T, Lei B, Xiao X (2018) 3D convolutional neural network and stacked bidirectional recurrent neural network for Alzheimer’s Disease diagnosis. PRedictive Intelligence in MEdicine, 138–146. https://doi.org/10.1007/978-3-030-00320-3∖_17

  35. Feng C, Elazab A, Yang P, Wang T, Zhou F, Hu H, Xiao X, Lei B (2019) Deep learning framework for Alzheimer’s disease diagnosis via 3d-CNN and FSBi-LSTM. IEEE Access 7:63605–63618. https://doi.org/10.1109/access.2019.2913847

    Article  Google Scholar 

  36. Frisoni G, Testa C, Sabattoli F, Beltramello A, Soininen H, Laakso M (2005) Structural correlates of early and late onset Alzheimer’s disease: voxel based morphometric study. J Neurology Neurosurg Psychiatry 76:112–114. https://doi.org/10.1136/jnnp.2003.029876

    Article  Google Scholar 

  37. Gessert N, Beringhoff J, Otte C, Schlaefer A (2018) Force estimation from OCT volumes using 3D CNNs. Int J CARS 13:1073–1082. https://doi.org/10.1007/s11548-018-1777-8

    Article  Google Scholar 

  38. Grajski K, Bressler S (2019) Differential medial temporal lobe and default-mode network functional connectivity and morphometric changes in Alzheimer’s disease. NeuroImage: Clinical 23:101860. https://doi.org/10.1016/j.nicl.2019.101860

    Article  Google Scholar 

  39. Hosseini Asl E, Ghazal M, Mahmoud A, Aslantas A, Shalaby A, Casanova M, Barnes G, Gimel’farb G, Keynton R, El Baz A (2018) Alzheimer’s disease diagnostics by a 3D deeply supervised adaptable convolutional network. Front Biosci 23:584–596. https://doi.org/10.2741/4606

    Article  Google Scholar 

  40. Hsiao T, Chang Y, Chou H, Chiu C (2019) Filter-based deep-compression with global average pooling for convolutional networks. J Syst Archit 95:9–18. https://doi.org/10.1016/j.sysarc.2019.02.008

    Article  Google Scholar 

  41. Hu X, Teunissen C, Spottke A, Heneka M, Düzel E, Peters O, Li S, Priller J, Buerger K, Teipel S, Laske C, Verfaillie S, Barkhof F, Coll-Padrós N, Rami L, Molinuevo J, van der Flier W, Jessen F (2019) Smaller medial temporal lobe volumes in individuals with subjective cognitive decline and biomarker evidence of Alzheimer’s disease—Data from three memory clinic studies. Alzheimer’s Dementia 15:185–193. https://doi.org/10.1016/j.jalz.2018.09.002

    Article  Google Scholar 

  42. Huang Y, Xu J, Zhou Y, Tong T, Zhuang X (2019) Diagnosis of Alzheimer’s Disease via Multi-Modality 3D Convolutional Neural Network. Frontiers in Neuroscience. https://doi.org/10.3389/fnins.2019.00509

  43. Huang S, Zhong Z, Jin L, Zhang S, Wang H (2018) Dropregion training of inception font network for high-performance Chinese font recognition. Pattern Recogn 77:395–411. https://doi.org/10.1016/j.patcog.2017.10.018

    Article  Google Scholar 

  44. Islam M, Foysal F, Neehal N, Karim E, Hossain S (2018) Inceptb: A CNN based classification approach for recognizing traditional Bengali games. Procedia Computer Science 143:595–602. https://doi.org/10.1016/j.procs.2018.10.436

    Article  Google Scholar 

  45. Islam J, Zhang Y (2018) Deep convolutional neural networks for automated diagnosis of Alzheimer’s disease and mild cognitive impairment using 3D Brain MRI. Brain Informatics, 359–369. https://doi.org/10.1007/978-3-030-05587-5∖_34

  46. Jack C, Bernstein M, Fox N, Thompson P, Alexander G, Harvey D, Borowski B, Britson P, Whitwell LJ, Ward C, Dale A, Felmlee J, Gunter J, Hill D, Killiany R, Schuff N, Fox-Bosetti S, Lin C, Studholme C, DeCarli C, Krueger G, Ward H, Metzger G, Scott K, Mallozzi R, Blezek D, Levy J, Debbins J, Fleisher A, Albert M, Green R, Bartzokis G, Glover G, Mugler J, Weiner M (2008) The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J Magn Reson Imaging 27:685–691. https://doi.org/10.1002/jmri.21049

    Article  Google Scholar 

  47. Jagannatha A, Yu H (2016) Bidirectional RNN for medical event detection in electronic health records. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.18653/v1/n16-1056

  48. Kam T, Zhang H, Shen D (2018) A Novel Deep Learning Framework on Brain Functional Networks for Early MCI Diagnosis. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018, 293–301. https://doi.org/10.1007/978-3-030-00931-1∖_34

  49. Karasawa H, Liu C, Ohwada H (2018) Deep 3D Convolutional Neural Network Architectures for Alzheimer’s Disease Diagnosis. Intelligent Information and Database Systems, 287–296. https://doi.org/10.1007/978-3-319-75417-8∖_27

  50. Kingma D, Ba J (2015) Adam: A method for stochastic optimization. 2015 international conference on learning representations (iCLR). arXiv:1412.6980

  51. Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. In: 2017 Annual conference on neural information processing systems (NIPS). arXiv:1706.02515

  52. Kleinberg R, Li Y, Yuan Y (2018) An Alternative View: When Does SGD Escape Local Minima?. In: 2018 international conference on machine learning (ICML). arXiv:1802.06175

  53. Korolev S, Safiullin A, Belyaev M, Dodonova Y (2017) Residual and plain convolutional neural networks for 3D brain MRI classification 2017. IEEE 14th International Symposium on Biomedical Imaging (ISBI. https://doi.org/10.1109/isbi.2017.7950647

  54. Krizhevsky A, Sutskever I, Hinton G (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  55. Li H, Habes M, Wolk D, Fan Y (2019) A deep learning model for early prediction of Alzheimer’s disease dementia based on hippocampal magnetic resonance imaging data. Alzheimer’s Dementia 15:1059–1070. https://doi.org/10.1016/j.jalz.2019.02.007

    Article  Google Scholar 

  56. Li F, Liu M (2019) A hybrid convolutional and recurrent neural network for hippocampus analysis in Alzheimer’s disease. J Neurosci Methods 323:108–118. https://doi.org/10.1016/j.jneumeth.2019.05.006

    Article  Google Scholar 

  57. Li J, Liu X, Zhang M, Wang D (2019) Spatio-temporal deformable 3D ConvNets with attention for action recognition. Pattern Recogn 98:107037. https://doi.org/10.1016/j.patcog.2019.107037

    Article  Google Scholar 

  58. Li F, Zurada J, Liu Y, Wu W (2017) Input layer regularization of multilayer feedforward neural networks. IEEE Access 5:10979–10985. https://doi.org/10.1109/access.2017.2713389

    Article  Google Scholar 

  59. Liu G, Guo J (2019) Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325–338. https://doi.org/10.1016/j.neucom.2019.01.078

    Article  Google Scholar 

  60. Liu J, Li M, Lan W, Wu F, Pan Y, Wang J (2018) Classification of alzheimer’s disease using whole brain hierarchical network. IEEE/ACM Trans Comput Biology Bioinform 15:624–632. https://doi.org/10.1109/tcbb.2016.2635144

    Article  Google Scholar 

  61. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi F (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26. https://doi.org/10.1016/j.neucom.2016.12.038

    Article  Google Scholar 

  62. Liu B, Wei Y, Zhang Y, Yang Q (2017) Deep neural networks for high dimension low sample size data. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence. https://doi.org/10.24963/ijcai.2017/318

  63. Liu F, Zheng J, Zheng L, Chen C (2019) Combining attention-based bidirectional gated recurrent neural network and two-dimensional convolutional neural network for document-level sentiment classification. Neurocomputing 371:39–50. https://doi.org/10.1016/j.neucom.2019.09.012

    Article  Google Scholar 

  64. Lyu C, Chen B, Ren Y, Ji D (2017) Long short-term memory RNN for biomedical named entity recognition. BMC Bioinformatics. https://doi.org/10.1186/s12859-017-1868-5

  65. Marteau P, Gibet S, Reverdy C (2016) Adaptive Down-Sampling and Dimension Reduction in Time Elastic Kernel Machines for Efficient Recognition of Isolated Gestures. Advances in Knowledge Discovery and Management, 39–59. https://doi.org/10.1007/978-3-319-45763-5∖_3

  66. McNeely-White D, Beveridge J, Draper B (2019) Inception and ResNet features are (almost) equivalent. Cogn Syst Res 59:312–318. https://doi.org/10.1016/j.cogsys.2019.10.004

    Article  Google Scholar 

  67. Mufson E, Mahady L, Waters D, Counts S, Perez S, DeKosky S, Ginsberg S, Ikonomovic M, Scheff S, Binder L (2015) Hippocampal plasticity during the progression of Alzheimer’s disease. Neuroscience 309:51–67. https://doi.org/10.1016/j.neuroscience.2015.03.006

    Article  Google Scholar 

  68. Mukkamala M, Hein M (2017) Variants of RMSProp and adagrad with logarithmic regret bounds. In: 2017 international conference on machine learning (ICML). arXiv:1706.05507

  69. Nalepa J, Ribalta Lorenzo P, Marcinkiewicz M, Bobek-Billewicz B, Wawrzyniak P, Walczak M, Kawulok M, Dudzik W, Kotowski K, Burda I, Machura B, Mrukwa G, Ulrych P, Hayball M (2020) Fully-automated deep learning-powered system for DCE-MRI analysis of brain tumors. Artif Intell Med 102:101769. https://doi.org/10.1016/j.artmed.2019.101769

    Article  Google Scholar 

  70. Nam D, Barrack R, Potter H (2014) What Are the Advantages and Disadvantages of Imaging Modalities to Diagnose Wear-related Corrosion Problems? Clin Orthop Relat Res\(^{{{\circledR }}}\) 472:3665–3673. https://doi.org/10.1007/s11999-014-3579-9

    Article  Google Scholar 

  71. Nickerson P, Tighe P, Shickel B, Rashidi P (2016) Deep neural network architectures for forecasting analgesic response. In: 2016 38th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). https://doi.org/10.1109/embc.2016.7591352

  72. Ning X, Duan P, Li W, Zhang S (2020) Real-Time 3D Face alignment using an Encoder-Decoder network with an efficient deconvolution layer. IEEE Signal Processing Letters 27:1944–1948. https://doi.org/10.1109/lsp.2020.3032277

    Article  Google Scholar 

  73. Ning X, Gong K, Li W, Zhang L, Bai X, Tian S (2020) Feature Refinement and Filter Network for Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 1–1. https://doi.org/10.1109/tcsvt.2020.3043026

  74. Ning X, Li W, Tang B, He H (2018) BULDP: Biomimetic Uncorrelated locality discriminant projection for feature extraction in face recognition. IEEE Trans Image Process 27:2575–2586. https://doi.org/10.1109/tip.2018.2806229

    Article  MathSciNet  MATH  Google Scholar 

  75. Ofori E, DeKosky S, Febo M, Colon-Perez L, Chakrabarty P, Duara R, Adjouadi M, Golde T, Vaillancourt D (2019) Free-water imaging of the hippocampus is a sensitive marker of Alzheimer’s disease. NeuroImage: Clinical 24:101985. https://doi.org/10.1016/j.nicl.2019.101985

    Article  Google Scholar 

  76. Ohzeki M, Okada S, Terabe M, Taguchi S (2018) Optimization of neural networks via finite-value quantum fluctuations. Scientific Reports. https://doi.org/10.1038/s41598-018-28212-4

  77. Ouyang X, Xu S, Zhang C, Zhou P, Yang Y, Liu G, Li X (2019) A 3d-CNN and LSTM based Multi-Task learning architecture for action recognition. IEEE Access 7:40757–40770. https://doi.org/10.1109/access.2019.2906654

    Article  Google Scholar 

  78. Pagola M, Forcen J, Barrenechea E, Fernández J, Bustince H (2017) A Study on the Cardinality of Ordered Average Pooling in Visual Recognition. Pattern Recognition and Image Analysis, 437–444. https://doi.org/10.1007/978-3-319-58838-4∖_48

  79. Pirzada S, Uddin M, Figley T, Kornelsen J, Puig J, Marrie R, Mazerolle E, Fisk J, Helmick C, O’Grady C, Patel R, Figley C (2020) Spatial normalization of multiple sclerosis brain MRI data depends on analysis method and software package. Magn Reson Imaging 68:83–94. https://doi.org/10.1016/j.mri.2020.01.016

    Article  Google Scholar 

  80. Pluta R, Bogucka-Kocka A, Ułamek-Kozioł M, Bogucki J, Januszewski S, Kocki J, Czuczwar S (2018) Ischemic tau protein gene induction as an additional key factor driving development of Alzheimer’s phenotype changes in CA1 area of hippocampus in an ischemic model of Alzheimer’s disease. Pharmacol Rep 70:881–884. https://doi.org/10.1016/j.pharep.2018.03.004

    Article  Google Scholar 

  81. Rabai F, Ramani R (2017) Magnetic resonance imaging: Anesthetic implications. Essentials of Neuroanesthesia, 519–532. https://doi.org/10.1016/b978-0-12-805299-0.00031-2

  82. Rao C, Liu Y (2020) Three-dimensional convolutional neural network (3d-CNN) for heterogeneous material homogenization. Comput Mater Sci 184:109850. https://doi.org/10.1016/j.commatsci.2020.109850

    Article  Google Scholar 

  83. Rolls E, Huang C, Lin C, Feng J, Joliot M (2020) Automated anatomical labelling atlas 3. NeuroImage 206:116189. https://doi.org/10.1016/j.neuroimage.2019.116189

    Article  Google Scholar 

  84. Rolls E, Joliot M, Tzourio-Mazoyer N (2015) Implementation of a new parcellation of the orbitofrontal cortex in the automated anatomical labeling atlas. NeuroImage 122:1–5. https://doi.org/10.1016/j.neuroimage.2015.07.075

    Article  Google Scholar 

  85. Scheltens P, Blennow K, Breteler M, de Strooper B, Frisoni G, Salloway S, Van der Flier W (2016) Alzheimer’s disease. Lancet 388:505–517. https://doi.org/10.1016/s0140-6736(15)01124-1

    Article  Google Scholar 

  86. Selkoe D (2015) Alzheimer disease. Rosenberg’s Molecular and Genetic Basis of Neurological and Psychiatric Disease, 753–768. https://doi.org/10.1016/b978-0-12-410529-4.00067-x

  87. Shen J, Wang D, Huang Y, Wen M, Zhang C (2019) Scale-out Acceleration for 3D CNN-based Lung Nodule Segmentation on a Multi-FPGA System. In: Proceedings of the 56th annual design automation conference 2019 on - DAC ’19. https://doi.org/10.1145/3316781.3317906

  88. Shinagawa S, Kobayashi N, Nagata T, Kusaka A, Yamada H, Kondo K, Nakayama K (2016) DNA Methylation in the NCAPH2/LMF2 promoter region is associated with hippocampal atrophy in Alzheimer’s disease and amnesic mild cognitive impairment patients. Neurosci Lett 629:33–37. https://doi.org/10.1016/j.neulet.2016.06.055

    Article  Google Scholar 

  89. Shmulev Y, Belyaev M (2018) Predicting Conversion of Mild Cognitive Impairments to Alzheimer’s Disease and Exploring Impact of Neuroimaging. Lecture Notes in Computer Science, 83–91. https://doi.org/10.1007/978-3-030-00689-1∖_9

  90. Slot R, Sikkes S, Berkhof J, Brodaty H, Buckley R, Cavedo E, Dardiotis E, Guillo-Benarous F, Hampel H, Kochan N, Lista S, Luck T, Maruff P, Molinuevo J, Kornhuber J, Reisberg B, Riedel-Heller S, Risacher S, Roehr S, Sachdev P, Scarmeas N, Scheltens P, Shulman M, Saykin A, Verfaillie S, Visser P, Vos S, Wagner M, Wolfsgruber S, Jessen F, van der Flier W (2018) Subjective cognitive decline and rates of incident Alzheimer’s disease and non-Alzheimer’s disease dementia. Alzheimer’s Dementia 15:465–476. https://doi.org/10.1016/j.jalz.2018.10.003

    Article  Google Scholar 

  91. Sørensen L, Igel C, Pai A, Balas I, Anker C, Lillholm M, Nielsen M (2017) Differential diagnosis of mild cognitive impairment and Alzheimer’s disease using structural MRI cortical thickness, hippocampal shape, hippocampal texture, and volumetry. NeuroImage: Clinical 13:470–482. https://doi.org/10.1016/j.nicl.2016.11.025

    Article  Google Scholar 

  92. Tang H, Yao E, Tan G, Guo X (2018) A Fast and Accurate 3D Fine-Tuning Convolutional Neural Network for Alzheimer’s Disease Diagnosis. Communications in Computer and Information Science, 115–126. https://doi.org/10.1007/978-981-13-2122-1∖_9

  93. Thapar D, Jaswal G, Nigam A, Arora C (2019) Gait metric learning siamese network exploiting dual of spatio-temporal 3d-CNN intra and LSTM based inter gait-cycle-segment features. Pattern Recogn Lett 125:646–653. https://doi.org/10.1016/j.patrec.2019.07.008

    Article  Google Scholar 

  94. Tipps M, Buck K (2015) GIRK Channels: a potential link between learning and addiction. Int Rev Neurobiol, 239–277. https://doi.org/10.1016/bs.irn.2015.05.012

  95. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M (2002) Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI Single-Subject brain. NeuroImage 15:273–289. https://doi.org/10.1006/nimg.2001.0978

    Article  Google Scholar 

  96. Umamakeswari A, Angelus J, Kannan M, Rashikha BS (2020) Action Recognition Using 3D CNN and LSTM for Video Analytics. Intelligent Computing and Communication 531–539. https://doi.org/10.1007/978-981-15-1084-7∖_51

  97. Vorugunti C, Pulabaigari V, Gorthi R, Mukherjee P (2020) OSVFUsenet: Online Signature Verification by feature fusion and depth-wise separable convolution based deep learning. Neurocomputing 409:157–172. https://doi.org/10.1016/j.neucom.2020.05.072

    Article  Google Scholar 

  98. Wang H, Shen Y, Wang S, Xiao T, Deng L, Wang X, Zhao X (2019) Ensemble of 3D densely connected convolutional network for diagnosis of mild cognitive impairment and Alzheimer’s disease. Neurocomputing 333:145–156. https://doi.org/10.1016/j.neucom.2018.12.018

    Article  Google Scholar 

  99. Wijnands J, Thompson J, Nice K, Aschwanden G, Stevenson M (2019) Real-time monitoring of driver drowsiness on mobile platforms using 3D neural networks. Neural Computing and Applications. https://doi.org/10.1007/s00521-019-04506-0

  100. Xu X, Caulfield S, Amaro J, Falcao G, Moloney D (2019) 1.2 Watt Classification of 3D Voxel Based Point-clouds using a CNN on a Neural Compute Stick. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.10.114

  101. Xu Q, Wang Z, Wang F, Gong Y (2019) Multi-feature fusion CNNs for Drosophila embryo of interest detection. Physica A Stat Mechan Appl 531:121808. https://doi.org/10.1016/j.physa.2019.121808

    Article  Google Scholar 

  102. Xu J, Zhang Z, Friedman T, Liang Y, Broeck G (2018) A semantic loss function for deep learning with symbolic knowledge. In: Proceedings of machine learning research (PMLR). arXiv:1711.11157

  103. Yan Y, Lee H, Somer E, Grau V (2018) Generation of Amyloid PET Images via Conditional Adversarial Training for Predicting Progression to Alzheimer’s Disease. PRedictive Intelligence in MEdicine 26–33. https://doi.org/10.1007/978-3-030-00320-3∖_4

  104. Yang H, Yuan C, Li B, Du Y, Xing J, Hu W, Maybank S (2019) Asymmetric 3D Convolutional Neural Networks for action recognition. Pattern Recogn 85:1–12. https://doi.org/10.1016/j.patcog.2018.07.028

    Article  Google Scholar 

  105. Yildirim Ö (2018) A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal classification. Comput Biol Med 96:189–202. https://doi.org/10.1016/j.compbiomed.2018.03.016

    Article  Google Scholar 

  106. Zhang K, Guo L, Gao C (2018) Optimization method of residual networks of residual networks for image classification. In: 2018 IEEE international conference on big data and smart computing (BigComp). https://doi.org/10.1109/bigcomp.2018.00054

  107. Zhang Y, Muhammad K, Tang C (2018) Twelve-layer deep convolutional neural network with stochastic pooling for tea category classification on GPU platform. Multimed Tools Appl 77(17):22821–22839. https://doi.org/10.1007/s11042-018-5765-3

    Article  Google Scholar 

  108. Zhao K, Ding Y, Han Y, Fan Y, Alexander-Bloch A, Han T, Jin D, Liu B, Lu J, Song C, Wang P, Wang D, Wang Q, Xu K, Yang H, Yao H, Zheng Y, Yu C, Zhou B, Zhang X, Zhou Y, Jiang T, Zhang X, Liu Y (2020) Independent and reproducible hippocampal radiomic biomarkers for multisite Alzheimer’s disease: diagnosis, longitudinal progress and biological basis. Sci Bull 65:1103–1113. https://doi.org/10.1016/j.scib.2020.04.003

    Article  Google Scholar 

  109. Zhi S, Liu Y, Li X, Guo Y (2018) Toward real-time 3D object recognition: a lightweight volumetric CNN framework using multitask learning. Comput Graphics 71:199–207. https://doi.org/10.1016/j.cag.2017.10.007

    Article  Google Scholar 

  110. Zhou T, Ruan S, Canu S (2019) A review: Deep learning for medical image segmentation using multi-modality fusion. Array 3–4:100004. https://doi.org/10.1016/j.array.2019.100004

    Article  Google Scholar 

  111. Zhu Z, Dai W, Hu Y, Li J (2020) Speech emotion recognition model based on bi-GRU and Focal Loss. Pattern Recogn Lett 140:358–365. https://doi.org/10.1016/j.patrec.2020.11.009

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the PPR2-2015 project with the donation of the NVIDIA Geforce GTX 1080 Ti GPU used for this research under grant no 14UIZ2015, and in part by the Al Khawarizmi project financed by the Moroccan government through the CNRST funding program.

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.;Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.;Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Author information

Authors and Affiliations

  1. Laboratory of Computer Systems and Vision (LabSIV), Department of Computer Science, Faculty of Science, Ibn Zohr University, BP 8106, 80000, Agadir, Morocco

    Ibtissam Bakkouri & Karim Afdel

  2. Bordeaux Laboratory for Computer Science Research (LaBRI) UMR 5800, University of Bordeaux, CNRS, Bordeaux INP, F-33400, Talence, Cedex, France

    Jenny Benois-Pineau

  3. Aquitaine Institute for Cognitive and Integrative Neuroscience (INCIA) UMR 5287, University Victor Segalen Bordeaux 2, CNRS, F-33076, Bordeaux, Cedex, France

    Gwénaëlle Catheline For the Alzheimer’s Disease Neuroimaging Initiative

Authors
  1. Ibtissam Bakkouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karim Afdel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jenny Benois-Pineau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gwénaëlle Catheline For the Alzheimer’s Disease Neuroimaging Initiative
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ibtissam Bakkouri.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bakkouri, I., Afdel, K., Benois-Pineau, J. et al. BG-3DM2F: Bidirectional gated 3D multi-scale feature fusion for Alzheimer’s disease diagnosis. Multimed Tools Appl 81, 10743–10776 (2022). https://doi.org/10.1007/s11042-022-12242-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12242-2

Keywords

Search

Navigation