A Review on Joint Carotid Intima-Media Thickness and Plaque Area Measurement in Ultrasound for Cardiovascular/Stroke Risk Monitoring: Artificial Intelligence Framework

Abstract

Cardiovascular diseases (CVDs) are the top ten leading causes of death worldwide. Atherosclerosis disease in the arteries is the main cause of the CVD, leading to myocardial infarction and stroke. The two primary image-based phenotypes used for monitoring the atherosclerosis burden is carotid intima-media thickness (cIMT) and plaque area (PA). Earlier segmentation and measurement methods were based on ad hoc conventional and semi-automated digital imaging solutions, which are unreliable, tedious, slow, and not robust. This study reviews the modern and automated methods such as artificial intelligence (AI)-based. Machine learning (ML) and deep learning (DL) can provide automated techniques in the detection and measurement of cIMT and PA from carotid vascular images. Both ML and DL techniques are examples of supervised learning, i.e., learn from “ground truth” images and transformation of test images that are not part of the training. This review summarizes (1) the evolution and impact of the fast-changing AI technology on cIMT/PA measurement, (2) the mathematical representations of ML/DL methods, and (3) segmentation approaches for cIMT/PA regions in carotid scans based for (a) region-of-interest detection and (b) lumen-intima and media-adventitia interface detection using ML/DL frameworks. AI-based methods for cIMT/PA segmentation have emerged for CVD/stroke risk monitoring and may expand to the recommended parameters for atherosclerosis assessment by carotid ultrasound.

Appendices

Appendix 1 Mathematical Representations of ML and DL Paradigms

The usage of the term learnability denotes the ability of computational models to discover patterns from unstructured data, infer logical constructs, and make decisions. Learning models, both ML and DL, have been thoroughly used in medical imaging to make life-saving decisions [82, 151,152,153]. In this review, we will delve deeper into ML and DL paradigms for carotid imaging and gauge the similarities and differences between them. Also, from this point onward, the first and second techniques for plaque segmentation will be referred to as conventional models. The third-generation techniques, ML, and DL will be addressed independently, which is the main focus of this study. A general discourse on ML is given in the next subsection.

Machine Learning: A Mathematical Representation

A mathematical approach to the concept of ML is as follows [154, 155]: if the instance space is denoted as ${\varvec{X}}=\{{x}_{1},{x}_{2},\dots ,{x}_{i}\}$ and the label/class space is denoted by ${\varvec{Y}}=\{{c}_{1},{c}_{2},\dots ,{c}_{j}\}$, then characterization job can be denoted by mapping of the function $\widehat{f}:{\varvec{X}}\to {\varvec{Y}}$, where $\widehat{f}$ is an estimate of the true function $f({x}_{i})$, and, $f\left({x}_{i}\right)$ denotes the true class of an instance ${x}_{i}$. The difference between the estimated version of the true function $\left(\widehat{f}\right)$ and the true function $(f)$ is used as a feedback to fine-tune the ML model to converge towards the correct output. The least-square (LS) method is the most common tool to compute the difference between the desired and actual output which is expressed mathematically as:

$$\varepsilon =\frac{1}{n}{\Vert f\left({\varvec{X}}\right)-\widehat{f}({\varvec{X}})\Vert }^{2}$$

(11)

where $\widehat{f}({\varvec{X}})$ is the output of the ML model $\mathrm{and} f\left({\varvec{X}}\right)$ is the input of the ML model. Different variations of error or loss-functions are employed in other AI paradigm. There are different variations of this form of learning, i.e., probability estimation, where the approximate function outputs a probability estimation over classes for any instance, i.e., $\widehat{f}:{\varvec{X}}\to {\left[\mathrm{0,1}\right]}^{\left|{\varvec{Y}}\right|}$. Regression is another learning job, where ${\varvec{X}}=R,$ and ${\varvec{Y}}=R,$ where, $R \epsilon \mathrm{real numbers}$, and $\widehat{f}:{\varvec{X}}\to {\varvec{Y}}$, is an approximation function mapping from ${\varvec{X}}$ to ${\varvec{Y}}$. The instances are defined by their attributes or features. Therefore, the attribute space can be defined as a mapping of function ${g}_{k}$ from ${g}_{k}:{\varvec{X}}\to {{\varvec{A}}}_{k}$, i.e., from instance space ${\varvec{X}}$ to attribute domain ${{\varvec{A}}}_{k}$. In other words, the instance space can be defined as the Cartesian product of $k$ attribute domains ${\varvec{X}}={{\varvec{A}}}_{1}\times {{\varvec{A}}}_{2}\times \dots \times {{\varvec{A}}}_{k}$. In the case of ML, the features are needed to be extracted separately from the instances. The features are generally statistical information that can be categorized as quantitative, ordinal, and categorical. Since we are mainly dealing with different imaging modalities, features are generally quantitative. Some of the features extracted in general from these images deal with orientation, direction, scale, and texture, etc. ML models that estimate the function can be geometric, probabilistic, or logical. Geometric models are built directly in instance space using geometric concepts of lines, planes, and distances. These models work by drawing a hyperplane between two classes, i.e., K-nearest neighbors [156], support vector machines (SVMs) [157], artificial neural networks (ANNs) [158], etc. Probabilistic models take into consideration most likelihood of an instance belonging to a class by computing maximum posterior probability e.g., Bayesian classifier [159]. Logical models compute the likelihood of a class by developing a series of rules based on logical operations, i.e., decision trees [160]. The training and testing of ML as well as DL models (described in the next section) follow the same pattern except for the feature extraction part which is the same as the learning paradigm of the latter. Initially, the given model is trained using offline labeled data. Once trained, the ML model is tested on an unknown online data to test its performance. An illustration of training and testing is shown in Fig. 3 (i).

Deep Learning: A Mathematical Representation

Deep learning models draw their inspiration from the working of brain neural networks. Unlike ML, deep learning models generate feature space directly from instance space, without the requirement of third-party feature extraction algorithms. These features can be said to be a downsampled representation of the original instances. The feature space, also called as representation space for DL techniques, is generated by using many layers of similar kernel functions, resulting in dimensionality reduction of the original instance space to the desired dimensionality of feature space which is given as follows:

$${{\varvec{A}}}_{{\varvec{i}}}={P}_{n}\left({Q}_{n}\left({R}_{n}\left({\dots (P}_{1}\left({Q}_{1}\left({R}_{1}\left({{\varvec{x}}}_{i}\right)\right)\right)\dots \right)\right)\right)$$

(12)

where, ${{\varvec{x}}}_{{\varvec{i}}}\in {\varvec{X}}$, $P$, $Q,$ and $R$ are different kernels applied repeatedly on the instance ${x}_{i}$. ${{\varvec{A}}}_{{\varvec{i}}}$ is the desired feature vector obtained after $n$ applications of $P$, $Q,$ and $R$. Therefore, the least square model (Eq. 11) can be used to backpropagate [161] the error within the network to fine-tune the DL model. Convolution neural networks (CNNs) [84] and fully convolutional networks (FCNs) [109] are the most common deep supervised learning models used widely in the characterization of carotid plaque and segmentation of the cIMT region. These networks apply a series of convolution and pooling operations to extract features from plaque images and characterize/segment them.

There is another form of learning in which outputs are not available and called unsupervised learning [162]. The unsupervised learning models try to find out interesting relationships within the input data to show important properties. K-means clustering [163] and autoencoders [107] are important ML and DL unsupervised learning algorithms, respectively. This review briefly describes all the models used in cIMT and PA measurement.

• JS Suri, D Kumar, Medical image enhancement system, US Patent App. 11/609,743

Appendix 2 Mathematical Representations of ML and DL Paradigms

A Short Note on Cardiovascular Risk Assessment

The cardiovascular risk assessment or stratification using intelligence paradigms such as ML and DL can help in both monitoring the CVD risk. Many studies have shown a strong association between covariates such as blood biomarkers, and conventional risk factors like age, grayscale median values, and stroke risk [20, 79, 89, 133]. In addition to blood biomarkers, behavioral patterns such as smoking, diets, and other image-based phenotypes such as cIMT, and PA values can be combined to enhance stroke risk assessment.

A note on Clinical Impact of AI Methods on cIMT/PA Techniques

Note that DL has just started to penetrate in the vascular area, especially vascular ultrasound. The prototypes have been attempted recently and our group has been leading this field. While we are able to undergo the designs, these designs have not passed the stage of its application to clinical world, unlike our non-AI-based models which are already in clinical practice (see AtheroEdge™ 2.0, AtheroPoint™, Roseville, CA, USA [79, 89, 133,134,135,136]). We however believe, as time progresses, we will see more applications of TL/DL/RL will reach the clinical world where diagnostic ultrasound community would start using this. Some of the scientific validation can be accomplished by matching the plaque regional information between cross modalities using registration methods [137].

A Note on Inter- and Intra-Observer Variability Analysis on Evaluation of AI Models

Inter- and intra-observer variability is certainly an important consideration during the performance evaluation of the cIMT/PA systems. Our group has attempted the variability analysis on cIMT and other applications [138,139,140] using AtheroEdge™ 1.0 and 2.0 systems. Recently, Biswas et al. [141] showed the effect of inter-observer variability using deep learning systems. The authors demonstrated inter-operator variability of the DL system using three set of manual tracings on 407 ultrasound scans for lumen boundary detection and performance evaluation. Each DL system was trained using the manual tracings, which underwent cross-validation runs for training and testing protocol. Using the Suri’s polyline distance metric, the precision of merit for three DL systems over 407 US scans was 99.61%, 97.75%, and 99.89%, respectively. The Jaccard index of the DL lumen segmented region against three ground truth (GT) regions were 0.94, 0.94, and 0.93, while the Dice similarity was 0.97, 0.97, and 0.97, respectively. The corresponding AUC for three DL systems was 0.95, 0.91, and 0.93. This DL system demonstrated superior performance of proposed DL system over conventional methods in literature. In another recent DL method using patch-based technique [142], the authors did the variability analysis using two manual tracings taken from two vascular radiologists. The authors showed that the absolute cIMT error of their AI-based system was $0.0935\pm 0.0637\mathrm{ mm}$ (i.e., 20% lesser than the earlier method [111]) using GT1 (i.e., $0.1164\pm 0.1122\mathrm{ mm}$) and 18% lesser using GT2 (i.e., $0.1146, 0.0940\mathrm{ mm}$), respectively. As part of performance evaluation of the system such as reliability, the authors in [48, 131] attempted studying the effect of ultrasound scanners on the performance of cIMT.

A Short Note on 10-year Risk Stimation Using cIMT and PA

Many studies have shown that cIMT and PA values are linked with diabetes and cardiovascular risks [79]. Different risk measures and calculators have been developed to compute the 10-year risks. A new measure, age-adjusted grayscale median (AAGSM) [28], has been developed as a function of age, PA, and conventional GSM. An association of HbA1c, AAGSM, and GSM is shown in Fig. 10

Fig. 9

Comparison of AtheroEdgeTM (a and b [121], c and d [122] (reproduced with permission)), SS (e and f [46]) and AI (g and h) [111] delineation of LI-and MA borders

 which illustrated low CC: $-0.05 (p=0.36)$ between HbA1c (male) and GSM, low CC: $-0.09 (p=0.10)$ between HbA1c (male) and AAGSM, low CC: $- 0.09 (p=0.372)$ between HbA1c (female) and GSM, high CC: − 0 $.29$ $(p=0.005)$ between HbA1c (female) and AAGSM. These relationships were instrumental in stratifying diabetes patients into low, medium, and high-risk groups.

Fig. 10

(i) GSM vs. HbA1c for male; (ii) AAGSM vs. HbA1c for male; (iii) GSM vs. HbA1c for female and (iv) AAGSM vs. HbA1c for female (image courtesy AtheroPoint™)

In another study, cIMT [134] has been used to calculate the risk of diabetic cohorts using logistic regression. Another risk score “AtheroEdge Composite Risk Score 1.0” (AECRS1.0_10 yr) [89] computed the 10-year risk of carotid image phenotypes by integrating conventional cardiovascular risk factors and reported the highest area-under-the-curve of 0.927.

Statistical Power Analysis and Diagnostic Odds Ratio

Statistical power analysis is performed to validate sample size for training. If z* is the standardized value taken from the z-table, $\widehat{p}$ is the data proportion, and MoE is the margin of error, then the number of samples (n) required is computed as $n=\left[{\left({z}^{*}\right)}^{2}\times \left(\frac{\widehat{p}\left(1-\widehat{p}\right)}{{MoE}^{2}}\right)\right]$. This is used with a 95% confidence interval with a 5% margin of error and 0.5 data proportion. In some recent AI studies, power analysis was performed for training data size validation [143,144,145]. The diagnostic odds ratio (DOR) [146] presents a meta-analysis of the diagnosis. The DOR is given by:

$$DOR=\left(\frac{sens}{1-sens}\right)/\left(\frac{1-spec}{spec}\right)$$

(13)

where $sens$ and $spec$ refers to the sensitivity and specificity of the study, respectively. Another popular tool to understand the power of the diagnostic tool is to know the reduction rate of type II error (false negative) [147]. If $\beta$ refers to a false negative rate, then $1-\beta$ refers to the power of the study.

A Short Note on Graphical Processing Units

The AI growth is powered by the availability of graphical processing units (GPUs). As experienced by our group, the AI usually involves millions of complex computations which if performed sequentially in multicore CPUs that take large time (sometimes even several hours) to converge. In contrast, GPUs break the complex problems into thousands of simple problems to be divided among its thousands of cores to solve them in parallel and combine them at once. Hence, convergence is much faster in GPUs compared to CPUs. In a comparative analysis [148] of ImageNet8 [84] implementation by different deep learning libraries such as MXNet [149], SINGA [150], Omnivore [148] on CPU-S (nine cores), CPU-L (33 cores), and GPU-S (36 GPUs) cluster systems, the GPU-S converged faster than the CPU-S and CPU-L for both.

Fig. 11

ImageNet implementation using different libraries on CPU-S, GPU-S, and CPU-L cluster (reproduced with permission[148])

Omnivore and MXNet. Among the libraries, Omnivore performed best (2.3X-CPU-S, 4.8X-GPU-S, 3.2X-CPU-L) while the SINGA library performed worst, and was not implemented in CPU-L and GPU-S. The comparison graph is shown in Fig.11