Detection of mild cognitive impairment based on attention mechanism and parallel dilated convolution

Attention gate mechanism

The attention mechanism is a technology that simulates the operation of the human brain. Its purpose is to imitate the selective attention characteristic phenomenon that humans present in the process of information processing. This mechanism allows humans to selectively focus on specific sources of information and thereby ignore information that is irrelevant to the current task. Using this ability, humans can gain a more refined and in-depth understanding of complex situations. With the continuous progress in the fields of computer science and artificial intelligence, researchers have introduced attention mechanisms into the field of deep learning to optimize the execution performance of various tasks. Among them, the most representative works include the channel attention mechanism proposed by Hu, Shen & Sun (2018) in 2017, the CBAM mechanism that integrates channel attention and spatial attention proposed by Woo et al. (2018) in 2018, and the 2020 (Wang et al., 2020) improved the channel attention mechanism, etc. By generating weighted attention maps in the two dimensions of channel and space, the network can more effectively focus on key channel features and spatial location information. This optimization method has achieved remarkable results in the field of natural image processing.

However, compared with natural images, medical images have the characteristic of localized target areas. Especially for the brain MRI images used in this study, the location of the lesions only occupies a small area of the entire brain MRI image. Currently, methods using channel attention mechanisms can automatically learn the importance between different channels (feature maps), allowing the network to focus on information useful for solving specific tasks. However, it often performs unsatisfactorily when processing tasks with fine-grained spatial information. While methods based on spatial attention mechanisms ignore the influence of global features. In the diagnosis of MCI, brain MRI lesions show obvious localization characteristics, and early lesions are not obvious. Therefore, during the diagnosis process, local information and global information need to be effectively combined.

To address this problem, this study introduces an attention mechanism method that targets the localization characteristics of targets in medical images, that is, the attention gate mechanism (AG) (Schlemper et al., 2019). The structure of the attention gate mechanism is shown in Fig. 2. Among them, x_l is the local feature vector extracted from the middle layer of the network, and g_l represents the global feature vector obtained from the coarse-scale part of the network, which contains abstract information of the target, such as size, position and orientation. The global feature g_l is used as a gating signal, and the local feature x_l is sent to the gated attention. The attention coefficient α^l is calculated, and its value is between the interval $\left[0,1\right]$. The calculation formula of attention coefficient α^l is as follows: (1)${\displaystyle q_{att,i}^l={\Psi }^T\left({\sigma }_1\left(W_x^T{x}_i^l+W_g^{T}g+b_{xg}\right)\right)+b_{\Psi }{\alpha }^l={\sigma }_2\left(q_{att}^l\left(x^l,g;{\Theta }_{att}\right)\right)}$ where σ₁ and σ₂ represent different activation functions. σ₁ is a rectified linear unit (ReLU), and σ₂ is generally a Softmax function to ensure that the sum of attention coefficient ${\alpha }_i^l=e^{q_{att,i}^{q^l}}/{\sum }_i{e}_{att,i}^{q^l}$ is 1. The output of the attention gate mechanism is completed by a channel-based 1 × 1 × 1 convolutional linear transformation, including three parts: W_x ∈ ℝ^F_l×F_int, W_g ∈ ℝ^F_g×F_int and Ψ ∈ ℝ^F_int×l. The corresponding bias terms are b_xg ∈ ℝ^F_int and b_Ψ ∈ ℝ, respectively. The final weighted output is ${\hat{\alpha }}^l={\left\{{\alpha }_i^l{x}_i^l\right\}}_{i=1}^n$. It can be seen that in the target area, the attention coefficient is larger, while in the background area, the attention coefficient is smaller. Therefore, the method is able to extract feature responses relevant to the target task from medical images and suppress the influence of useless features.

Parallel dilated convolution

An important challenge facing the field of MCI detection is that the lesion area appears localized and its size is usually extremely small. Therefore, ensuring that these small-size target features are not ignored during the recognition process has become one of the key factors to ensure accurate recognition of the model. Classic classification networks at this stage usually add pooling layers before and after the convolutional layer to increase the receptive field of the network. However, the pooling operation will lead to the loss of some spatial information, including important detailed features that may be included in the original image, and may even directly ignore small-scale lesion features.

Dilated convolution, also known as atrous convolution, was first proposed by Yu & Koltun (2015) in 2016. It was originally used to solve the problem of pooling operation reducing image resolution and causing the loss of some features in image semantic segmentation, as an alternative to the pooling layer. In a convolutional neural network, if you directly remove the pooling layer that may reduce the image size, you will not be able to increase the receptive field of the small-sized convolution kernel, and if you blindly increase the size of the convolution kernel, it will significantly increase the number of parameters of the network. By using a sparse kernel, dilated convolution expands the effective size of the convolution kernel without increasing network parameters, thereby expanding the receptive field of the network. This effect is similar to alternating convolutional and pooling layers, but does not reduce the size of the feature map.

The basic principle of dilated convolution is to expand the convolution kernel parameters that are originally closely connected according to a certain ratio, and use zero values to separate these parameters. The separation distance is determined by the expansion rate. The advantage of this design is that the receptive field of the network is expanded through the expansion of the convolution kernel, but the kernel parameters involved in the calculation do not increase, thus not increasing the computational burden. At the same time, the size of the feature map after dilation convolution remains unchanged, effectively retaining important detailed features.

Figure 3 shows a 3 × 3 dilated convolution kernel with different dilation rates. It can be observed from the figure that when the dilation rate is 1, dilated convolution is no different from ordinary convolution. When the dilation rate is 2, the interval between convolution kernel elements is 1. After filling, the dilated convolution kernel is equivalent to a 5 × 5 convolution kernel. When dilated = 3, the interval between convolution kernel elements is 2, which is equivalent to a receptive field of 7 × 7 convolution kernel. That is to say, for a convolution kernel with an original size of k × k, when the expansion rate is d, the actual side length of the convolution kernel will become $k+\left(k-1\right)\times \left(d-1\right)$.

For a two-dimensional input data, the formula of dilated convolution is defined as follows: (2)${\displaystyle y\left(m,n\right)=\sum _{i=1}^M\sum _{j=1}^{N}x\left(m+r\cdot i,n+r\cdot j\right)w\left(i,j\right)}$ where $x\left(m,n\right)$ represents the input of the network, $w\left(i,j\right)$ represents the filter of size M × N, and r represents the expansion rate of the convolution kernel, and $y\left(m,n\right)$ is the output of the network. By introducing dilated convolution, a larger receptive field can be obtained without the need for downsampling. This enables the retention of detailed information and the rich acquisition of contextual information on large-scale feature maps.

To solve the problem of small-size target feature recognition in MCI detection, this study introduces a parallel dilated convolution module, that is, Atrous Spatial Pyramid Pooling (ASPP), into the network structure. The parallel dilated convolution module contains multiple dilated convolution operations, and each operation uses a different dilation rate. The output feature map of each dilated convolution operation preserves different scale contextual information. By using the feature fusion method, the output feature maps of each dilated convolution operation are stacked in the channel dimension to obtain richer multi-scale feature representations. The ASPP is shown in Fig. 4.

Overall network architecture

To achieve good classification results for small target lesions, the network needs to obtain rich lesion area information, including the shape and size of the lesion and other features. These features usually exist in shallow networks, but because the lesion area is too small, these shallow features will lose a lot of detailed information during the feature extraction process. In this study, based on brain MRI images, we proposed an improved ResNet18 network for the detection of MCI with the help of an attention gate mechanism and parallel dilated convolution. As shown in Fig. 5, this network utilizes the deepest feature map g as a gating signal to provide contextual information for the feature maps of the second and third ResNet blocks. The purpose of this operation is to prune redundant information in shallow features to highlight the salient features of the lesion area. The deepest feature map g is selected as the gating signal to ensure that the extracted contextual information is highly correlated with the deep features, thereby helping to accurately identify the lesion area.

After eliminating redundant features, these features are input to the ASPP, and the detailed information and contextual information of the large-scale feature map are extracted through dilation convolution with different dilation rates. This step is crucial as it allows the network to focus on features at multiple scales simultaneously, thus better capturing the diversity of the lesion area. After obtaining the output of the ASPP, it is fused with the gating signal g to further obtain the features of fused gating information and multi-scale information. This fused feature has a larger scale range, which helps avoid the loss of detailed features and effectively solves the problem of poor performance of existing methods in classifying small target lesions.

Dataset

Image preprocessing and data augmentation

Due to the high dimensionality of MRI and relatively sparse medical data, deep learning algorithms face huge challenges in their training and convergence processes. Therefore, all images must be preprocessed to map brain image samples to a common coordinate space. This study uses the CAT12 toolkit of SPM12 (Gaser et al., 2022) to perform the necessary processing on the above MRI images: (1) Skull stripping. Use the built-in skull stripping function of CAT12 software to remove skull structures to exclude brain skull information that is not relevant to the experiment; (2) Tissue segmentation. Segment the MRI image into three tissue types: white matter, gray matter and cerebrospinal fluid. Since the gray matter tissue of the brain is susceptible to morphological changes caused by MCI, this study uses the gray matter tissue of the subject’s brain as input; (3) Spatial registration. Register the subject’s image to the MNI152 standard template established by the Montreal Neurological Institute-Hospital (MNI) to reduce the differences in individual brain spaces; (4) Modulation. Ensure that brain tissue is comparable in template space while retaining differences between individuals; (5) Smooth. Spatial smooth is performed on the image to reduce reconstruction errors and improve the consistency of the subject’s brain image. After the above preprocessing steps, a brain MRI gray matter image with a size of 84  ×  102  ×  84 was finally obtained. Figure 6 gives a schematic diagram of the preprocessing process.

Table 2:

Demographic information of subjects.

Diagnosis	Subjects	Gender (M /F)	Age (Mean ± SD)	Scans
NC	127	64/63	75.72 ± 6.87	265
MCI	75	46/29	77.99 ± 8.10	265

DOI: 10.7717/peerjcs.2056/table-2

Deep learning methods are known for their powerful potential and highly complex models, however, for these models to produce robust results, large image data sets are usually required. Due to the high cost and difficulty of obtaining clinical data, enough data cannot be obtained. In this study, we use data augmentation techniques such as flipping, rotating, and adding noise to improve the training effect of the neural network model. Among them, flipping the image vertically, horizontally and axial can increase the mirrored version of the data, helping the model better capture different perspectives and features in the image. The rotation operation can introduce more changes, making the model better able to recognize objects at different angles. By adding random noise to the image to simulate the uncertainty in the real world, it helps to improve the robustness of the model and make it better able to deal with noise and interference in the real world. In this study, we utilize these data augmentation techniques to expand the training data by 25 times. This significantly improves the model’s generalization ability on unknown data.

Experimental setup

The experimental running system of this study is Ubuntu 20.04.6 LTS, the CPU is 12th Gen Intel (R) Core (TM) i9-12900K with a main frequency of 3.9 GHz, the memory is 64 GB, the GPU model is NVIDIA GeForce RTX 3080*2, and the video memory is 24 GB. The experimental environment is python3.8 and pytorch1.7.1. In the experiment, the three-dimensional convolution kernel and fully connected weight parameters were initialized with truncated normal distributed random numbers with a standard deviation of 0.1. For the setting of hyperparameters, based on our experience and previous research, we set the batch size to 8. Compared to larger batch sizes, smaller batch sizes promote model convergence more effectively and are more efficient in terms of memory utilization. For the choice of optimizer, we use the Adam optimizer. The Adam optimizer combines the ideas of momentum and RMSprop, has the characteristics of adaptive learning rate, and has outstanding performance in handling many tasks. At the same time, compared to other optimizers, Adam is less sensitive to the selection requirements of hyperparameters. For the learning rate, we follow the default value used in most current studies, which is 0.001. In addition, we also adopt a learning rate decay strategy, that is, after every 10 epochs, the learning rate is adjusted to 0.1 times its original value. This strategy can improve the performance and stability of the optimization algorithm, allowing the model to converge to a better solution faster. The total training epochs are set to 50.

Evaluation indicators

This study uses common evaluation indicators in medical image classification tasks to evaluate the performance of the model, including sensitivity (SEN), specificity (SPE), accuracy (ACC) and F1-score (F1). The definitions of each indicator are as follows: (3)${\displaystyle SEN=\frac{TP}{TP+FN}}$ (4)${\displaystyle SPE=\frac{TN}{TN+FP}}$ (5)${\displaystyle ACC=\frac{TP+TN}{TP+TN+FP+FN}}$ (6)${\displaystyle f1=\frac{2\times \frac{TP}{TP+FP}\times SEN}{\frac{TP}{TP+FP}+SEN}.}$ Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative, respectively. In addition, the area under the curve (AUC) is also introduced to evaluate the overall classification performance of the model. AUC is the area enclosed by the receiver operating characteristic (ROC) curve in the $\left[0,1\right]$ interval and the X-axis, which reflects the overall performance of the model. The larger the AUC value, the better the model classification performance.

Experiment analysis

Figures 7 and 8 depict the training-validation loss and accuracy of the proposed method across iteration epochs, along with the confusion matrix and ROC curve. First, from Fig. 7, it can be found that as the number of iterations increases, the training and validation losses of the model gradually decrease and the accuracy gradually increases. In particular, the proposed method reaches stable accuracy after about 15 epochs. This shows that the proposed model can effectively reduce errors during the learning process and reach acceptable performance in a relatively short time. However, we also noticed a large difference in accuracy between the training and validation sets. This difference can be attributed to the smaller sample size used in the study. As the number of iterations increases, the model overfits on the training set. Further observing the confusion matrix and ROC curve in Fig. 8, we can find that out of 147 test samples, 120 were successfully predicted. This indicates that the proposed method achieves good performance in detecting MCI. In addition, we also showed the ROC curve with an AUC value of 83.06%, which further proved its effectiveness in detecting MCI.

Comparison of different ResNet architectures

To verify the performance differences on different ResNet architectures, we compared several typical ResNet basic frameworks, including ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152. These different ResNet architectures have obvious differences in depth, parameter volume and computational complexity, and have different impacts on MCI detection tasks. The experimental results are shown in Table 3. We used a variety of performance evaluation indicators to comprehensively evaluate the performance of different ResNet architectures, including sensitivity, specificity, accuracy, F1 and AUC.

From Table 3, it can be observed that ResNet18, ResNet34 and ResNet50 perform equally well in terms of accuracy. However, ResNet101 and ResNet152, two relatively deep ResNet architectures, show a clear downward trend in accuracy. This phenomenon is because as the depth of the model increases, the network overfits, resulting in a decrease in generalization performance on the test data. In addition, comparing ResNet18 and ResNet50, it can be found that they are generally equivalent in various indicators. However, compared with ResNet50, ResNet18 has a significantly lower number of parameters and computational complexity, and has more advantages in model lightweight, which is of great significance for MCI detection tasks in resource-constrained environments. Therefore, in this study, we choose ResNet18 as the backbone framework for the task.

Comparison of different attention modules

To verify the effectiveness of the attention module used in this study, we embedded different types of attention modules into the ResNet-18 backbone framework and compared their impact on model classification performance. In these experiments, we include a variety of widely used attention modules, such as SENet (Hu, Shen & Sun, 2018), ECANet (Wang et al., 2020) and CBAM (Woo et al., 2018), to ensure comprehensive evaluation, and the experimental results are shown in Table 4.

Combining the results in Tables 3 and 4, we can clearly observe that after integrating different types of attention modules into the network, the overall performance has improved. This performance improvement is reflected in multiple key performance indicators, including sensitivity, accuracy and F1. However, it is worth noting that compared to other attention modules, the attention gate mechanism (AG) adopted in this study shows compelling superiority in terms of accuracy and F1. This is because the attention gate mechanism is a method proposed for the localization characteristics of the target area of medical images. The method enables the network to focus more on the area of interest, extract feature responses that are closely related to the task target, and effectively suppress the influence of useless features, thereby better capturing key features in medical images and improving the overall network performance.

Ablation experiment

To prove the effectiveness of the module design in the model proposed in this study, we conducted an ablation experiment. The control variable method is used to gradually add different modules of the model and evaluate the impact of each module on the overall model performance. The experiment uses the ResNet18 basic network as the evaluation baseline. The results are shown in Table 5.

It can be clearly observed from Table 5 that with the gradual increase of different modules, the overall performance shows a gradual improvement trend. These results indicate that each module has a positive impact on the performance of the model. It is particularly worth noting that after adding the ASPP, the accuracy rate increased by about 4.76%. This proves that this module can better capture important information in images by introducing receptive fields of different scales, thereby improving classification performance.

Comparison with other methods

To further verify the effectiveness of the method, we compared it with other work in the literature. It is worth noting that the ADNI database contains longitudinal examination data. To avoid data leakage leading to model overfitting, in this study we divided the data based on subjects. In the comparison, we also only report methods that explicitly adopt this strategy in the literature. Table 6 shows the performance comparison between the method and existing work.

As can be seen from Table 6, the method used in this study has achieved significant improvements in specificity and accuracy. Specificity and accuracy are key indicators for measuring the performance of a classification model. High specificity means that the model is more able to correctly classify healthy samples as healthy, while high accuracy means that the model is more able to correctly classify diseased samples as diseased. These results indicate that our method is more accurate in distinguishing normal samples and MCI patients, which is of great significance for early diagnosis and treatment of MCI.

Visualization

To further analyze the performance of the model proposed in this study, we adopted the Gradient-weighted Class Activation Mapping (Grad-CAM) technology (Selvaraju et al., 2017) to visualize the model’s performance in key areas of interest, thereby qualitatively analyzing the effectiveness of the model in learning the characteristics of the lesion area. The Grad-CAM generated by this study’s model is presented in Fig. 9, where blue represents low-weighted regions and red represents high-weighted regions. It can be observed from Fig. 9 that subjects have significantly high weights in the frontal and parietal regions. These brain regions are closely related to key functions in cognitive processes, including decision-making, planning, attention, working memory, and language comprehension. The early stages of MCI are often accompanied by atrophy of these brain areas. To verify the findings of the model, we further cooperated with professional doctors to carefully confirm the brain lesions of the subjects. These results strongly demonstrate that the model proposed in this study performs well in learning and utilizing lesion area information. It can extract effective features from these key regions, enabling highly accurate classification of MCI.

Impact of backbone network on performance

Besides ResNet, deep learning networks such as LeNet, AlexNet, GoogLeNet and VGG are also widely used as backbone networks. To gain a deeper understanding of the impact of different backbone networks on model performance, we conducted a comparative analysis of LeNet5, AlexNet, GoogLeNet, VGG16 and ResNet18. The experimental results are shown in Table 7. First, as a classic deep learning model, LeNet5 shows relatively high sensitivity, but is slightly insufficient in other indicators. AlexNet performs well in terms of specificity and AUC, but its sensitivity and accuracy are slightly insufficient. GoogLeNet shows balanced performance, especially in AUC and F1 indicators. VGG16 achieved the best performance in specificity and AUC, showing its adaptability to MCI detection tasks. Finally, ResNet18 achieved excellent performance in accuracy and F1, demonstrating its advantages in handling complex tasks. It can be seen that choosing an appropriate backbone network is crucial to the performance of the model. One of the possible reasons why VGG16 performs well in this task is that its deep structure is better able to extract and characterize the features of the data. ResNet18, through its deep residual connection design, shows better stability and robustness when handling more complex tasks.

Impact of data augmentation on performance

Data augmentation plays a vital role in the field of deep learning. Especially when clinical data sets are small, data augmentation can not only improve classification accuracy but also reduce the risk of overfitting. To verify the effectiveness of the data augmentation operation used in this study, we show the classification results of our method before and after data augmentation in Fig. 10. As can be seen from the figure, after using data augmentation operations, the classification results have improved in various indicators. First, without data augmentation, the classification accuracy is 77.55%, and with data augmentation, the classification accuracy increases to 81.63%. This shows that data augmentation can significantly improve the accuracy, thereby improving the reliability of clinical data analysis. In addition, in terms of sensitivity and specificity, the data augmentation operation has also been improved, which further confirms the effect of data augmentation in improving model performance.

Impact of hyperparameters on performance

Hyperparameters play a vital role in deep learning, and they directly affect the performance and training process of the model. In this part, we analyze the impact of two hyperparameters, optimizer and batch size, on performance.

The batch size determines the number of samples used at each step in the training process. We explored the performance when batch sizes were 8, 16, 32, 64, 128, and 256. Figure 11 shows the performance of two key indicators (ACC and F1) at different batch sizes. As can be seen from Fig. 11, as the batch size increases, the accuracy and F1 decrease slightly. This is caused by the different gradient update frequencies under different batch sizes. Larger batch sizes usually mean fewer gradient updates, which can cause the model to converge slower during training, thus affecting performance indicators. In this study, the model performed best when the batch size was 8. Although the ACC are slightly lower at batch size 8 relative to some other batch sizes, the F1 performs best. This is because when the batch size is 8, the model can more fully capture the subtle features of the data and update the weights more frequently during the training process, resulting in better performance.

Besides batch size, the optimizer is another hyperparameter that often receives attention. In this study, we compare four common optimizers: stochastic gradient descent (SGD), root mean square propagation (RMSprop), Momentum, and Adam, to evaluate their impact on model performance. The experimental results are shown in Fig. 12. It can be seen that from Fig. 12, SGD showed good results in terms of sensitivity, accuracy and F1, but was slightly insufficient in specificity and AUC. RMSprop showed high specificity but relatively low sensitivity, accuracy and F1. The momentum optimizer performs better in terms of sensitivity and F1, but slightly lower specificity and accuracy. The Adam optimizer performs best in terms of accuracy and AUC, and is also competitive in other indicators. The Adam optimizer combines the ideas of momentum and RMSprop, has the characteristics of adaptive learning rate, and introduces a momentum term to accelerate convergence. This makes Adam perform well in many tasks, especially when dealing with complex non-convex optimization problems. Therefore, we choose to use the Adam optimizer in this study, and compared with other optimizers, it is less sensitive to the selection requirements of hyperparameters.

Robustness under different dataset distributions

Differences in data distribution may lead to changes in model performance, so verifying the performance of the model under different data distributions is crucial to evaluate its true effect. In this study, we used a four-fold cross-validation to validate the model, that is, each fold reserved 25% of the data for testing. Figure 13 shows the mean and standard deviation of different performance indicators. The proposed method achieved a sensitivity of 71.5% ± 7.74%, a specificity of 89.73% ± 2.12%, an accuracy of 80.7% ± 3.96%, and an AUC of 83.8% ± 5.63%, and an F1 of 78.13% ± 7.18%. These results show that our method can maintain good performance under different data distributions. The large standard deviation is due to the small data set, large inter-subject variability, and certain differences in the data distribution of the training set and the test set under different data splits.