Machine-Learning-Based Outcome Prediction in Stroke Patients With Middle Cerebral Artery-M1 Occlusions and Early Thrombectomy

|
Received: 30 October 2020 Accepted: 16 November 2020
DOI: 10.1111/ene.14651
ORIGINAL ARTICLE
Machine-learning-based outcome prediction in stroke

patients with middle cerebral artery-M1 occlusions and early
thrombectomy
Janne Hamann1 | Lisa Herzog1,2,3 | Carina Wehrli1,4 | Tomas Dobrocky5 |

Andrea Bink4 | Marco Piccirelli4 | Leonidas Panos6 | Johannes Kaesmacher5,7 |
Urs Fischer6 | Christoph Stippich4 | Andreas R. Luft1 | Jan Gralla5 | Marcel Arnold6 |
Roland Wiest5 | Beate Sick2,3 | Susanne Wegener1
1
Department of Neurology and Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
2
Epidemiology, Biostatistics and Prevention Institute (EBPI), University of Zurich, Zurich, Switzerland
3
Institute of Data Analysis and Process Design, ZHAW Winterthur, Winterthur, Switzerland
4
Department of Neuroradiology and Clinical Neuroscience Center, University Hospital Zurich and University of Zurich, Zurich, Switzerland
5
Diagnostic and Interventional Neuroradiology, University Hospital of Berne, Berne, Switzerland
6
Department of Neurology, University Hospital of Berne, Berne, Switzerland
7
Department of Diagnostic, Interventional and Pediatric Radiology, University Hospital of Berne, Berne, Switzerland
Correspondence Abstract
Susanne Wegener, Department of
Neurology, University Hospital Zurich, Background and purpose: Clinical outcomes vary substantially among individuals with
Frauenklinikstrasse 26, 8091 Zurich, large vessel occlusion (LVO) stroke. A small infarct core and large imaging mismatch were
Switzerland.
Email: Susanne.Wegener@usz.ch found to be associated with good recovery. The aim of this study was to investigate
whether those imaging variables would improve individual prediction of functional out-
Funding information
Support by the Swiss National Science come after early (<6 h) endovascular treatment (EVT) in LVO stroke.
Foundation (SNSF PP00P3_170683) and Methods: We included 222 patients with acute ischemic stroke due to middle cerebral
Clinical Research Priority Program stroke
(University of Zurich) is acknowledged. artery (MCA)-M1 occlusion who received EVT. As predictors, we used clinical variables
The funders have no role in the study and region of interest (ROI)-based magnetic resonance imaging features. We developed
design; in the collection, analysis, and
interpretation of data; in the writing of different machine-learning models and quantified their prediction performance accord-
the report; and in the decision to submit ing to the area under the receiver-operating characteristic curves and the Brier score.
the article for publication. All researchers
are independent from funders and all Results: The rate of successful recanalization was 78%, with 54% patients having a fa-
authors had full access to all of the data vorable outcome (modified Rankin scale score 0–2). Small infarct core was associated
(including statistical reports and tables) in
the study and can take responsibility for with favorable functional outcome. Outcome prediction improved only slightly when im-
the integrity of the data and the accuracy aging was added to patient variables. Age was the driving factor, with a sharp decrease in
of the data analysis.
likelihood of favorable functional outcome above the age of 78 years.
Abbreviations: AUC, area under the curve; CI, confidence interval; EVT, endovascular treatment; IVT, intravenous thrombolysis; LVO, large vessel occlusion; MCA, middle cerebral
artery; MR, magnetic resonance; MRI, magnetic resonance imaging; mRS, modified Rankin Scale; NIHSS, National Institutes of Health Stroke Scale; OR, odds ratio; PtC, patient
characteristics; rCBF, regional cerebral blood flow; rCBV, regional cerebral blood volume; RF, random forest; ROC, receiver-operating characteristic; ROI, region of interest; THRIVE,
Totaled Health Risks in Vascular Events; TICI, Thrombolysis In Cerebral Infarction scale; Tmax, time to maximum; VISTA, Virtual International Stroke Trials Archive.
Janne Hamann and Lisa Herzog contributed equally
This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in
any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
© 2020 The Authors. European Journal of Neurology published by John Wiley & Sons Ltd on behalf of European Academy of Neurology
|
1234
wileyonlinelibrary.com/journal/ene Eur J Neurol. 2021;28:1234–1243.
|
14681331, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/ene.14651 by Readcube (Labtiva Inc.), Wiley Online Library on [31/03/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
MACHINE LEARNING STROKE OUTCOME PREDICTION 1235
Conclusions: In patients with MCA-M1 occlusion strokes referred to EVT within 6 h
of symptom onset, infarct core volume was associated with outcome. However, ROI-
based imaging variables led to no significant improvement in outcome prediction at
an individual patient level when added to a set of clinical predictors. Our study is in
concordance with current practice, where imaging mismatch or collateral readouts
are not recommended as factors for excluding patients with MCA-M1 occlusion for
early EVT.
KEYWORDS
machine learning, stroke outcome prediction
I NTRO D U C TI O N M E TH O D S
Ischemic stroke treatment aims at recanalization of the occluded Patient data

vessel to restore blood flow as fast as possible [1]. With the
introduction of stent retrievers, a breakthrough in large ves- We used encrypted clinical and imaging data from a cohort of is-
sel occlusion (LVO) stroke management was achieved in 2015, chemic stroke patients treated with EVT between January 2012
when five randomized controlled trials showed greater efficacy and August 2017 at the Inselspital Berne, Switzerland. The study
of endovascular treatment (EVT) compared with intravenous was performed according to the ethical guidelines of the Canton of
thrombolysis (IVT) alone [2]. However, successful recanalization Berne, with approval from the local ethics committee (no. 231/14).
of LVO stroke does not always translate into good clinical recov- Patients were included in the analysis if: (i) diagnosis of ischemic
ery. Despite high recanalization rates, only approximately 50% stroke due to an MCA-M1 occlusion was established by magnetic
of patients with LVO have a favorable functional outcome [3]. resonance (MR) angiography and MR perfusion; (ii) modified Rankin
Currently, reliable predictors of functional outcome for patients Scale (mRS) score at 3 months was documented; (iii) EVT was at-
undergoing EVT are lacking, but are needed to understand the tempted; and (iv) perfusion and diffusion images were complete
effect of treatment on the individual patient. and of sufficient quality for analysis. Patients were excluded if pre-
Imaging mismatch with estimates of a small infarct core and vious territorial infarction was evident on magnetic resonance im-
significant area of hypoperfusion (at least 120% of the core) have aging (MRI) or if additional occlusions other than clot extension to
been incorporated into previous trials as selection criteria for the internal carotid artery or distal MCA branches were revealed
likely treatment success (EXTEND-IA, DEFUSE 3) [4,5]. In gen- by angiography. Patient characteristics that were available for
eral, the evidence of imaging mismatch for functional outcome analyses included demographic information, previous medication,
prediction has so far relied on association studies showing that risk factors, past ischemic events, as well as baseline stroke admis-
the means of conditional outcome distributions depend on imag- sion information (Table 1). Treatment decisions were made accord-
ing mismatch variables [6]. However, a different mean does not ing to routine clinical guidelines of the Bernese Stroke Center. The
necessarily imply a clear separation of the conditional outcome study was reported according to the Strengthening the Reporting
distributions, which is the prerequisite for achieving reliable out- of Observational Studies in Epidemiology (STROBE) guidelines for
come predictions [7]. At present, there are no prediction mod- observational studies.
els integrating imaging predictors including imaging mismatch in
stroke patients subjected to EVT.
Our goal was to develop a robust machine-learning model to Imaging data
improve functional outcome prediction in LVO stroke patients,
integrating clinical and imaging variables. All patients had middle Stroke imaging was performed using predefined stroke proto-
cerebral artery (MCA)-M1 occlusions and were subjected to EVT cols with 1.5-T or 3-T MRI systems from one vendor (Siemens
within 6 h of symptom onset; therefore, all were deemed good Healthineers, Erlangen, Germany). The full stroke protocol is de-
treatment candidates. Using this homogenous cohort, we in- scribed in Appendix S1A.
tended to overcome interactions of occlusion site and treatment All image analysis was performed blinded to clinical outcome.
regimen in outcome prediction for LVO stroke and to reveal the The dataset used for predictive analysis was restricted to diffu-
most relevant factors indicating favorable functional outcome sion-weighted and perfusion images. Imaging data were post-pro-
after early EVT. cessed with Olea Sphere's Acute Stroke Care Plug-in (Olea Sphere™
|
1236 HAMANN et al.
TA B L E 1 Patient characteristics
All (n = 222) Favorable (n = 119) Unfavorable (n = 103) p
Age, years 73.54 (19.99) 68.80 (18.23) 79.20 (17.70) <0.001

Women, n(%) 134 (60) 72 (61) 62 (60) 1.000
Risk factors, n(%)
Diabetes mellitus 39 (18) 15 (13) 24 (23) 0.051
Atrial fibrillation 92 (42) 39 (33) 53 (51) 0.006
Hypertension 148 (67) 69 (58) 79 (77) 0.004
Dyslipidemia 133 (60) 75 (63) 58 (57) 0.410
Smoker 49 (24) 35 (31) 14 (16) 0.013
Coronary heart disease 35 (16) 19 (16) 16 (16) 1.000
Peripheral arterial disease 8 (4) 3 (3) 5 (5) 0.474
Past ischemic events 30 (14) 15 (13) 15 (15) 0.698
Previous medication, n(%)
Antiplatelet therapy 70 (32) 30 (25) 40 (39) 0.031
Oral anticoagulation 21 (9) 10 (8) 11 (11) 0.648
Statin therapy 49 (22) 25 (21) 24 (23) 0.747
Antihypertensive therapy 130 (59) 61 (51) 69 (67) 0.020
On admission, n(%)
Independent before stroke 184 (92) 104 (96) 80 (88) 0.032
NIHSS baseline 13 (9) 12 (7) 15 (9.5) 0.005
Systolic blood pressure, mmHg 153 (36) 147 (29) 160 (35) 0.004
Diastolic blood pressure, mmHg 82 (25) 81 (22) 84.5 (27.5) 0.585
Glucose, mmol/L 6.5 (2.02) 6.2 (1.7) 6.75 (2.5) 0.010
HbA1c, % 5.8 (0.6) 5.8 (0.5) 5.8 (0.9) 0.650
LDL, mmol/L 2.41 (1.41) 2.53 (1.34) 2.36 (1.38) 0.252
HDL, mmol/L 1.37 (0.57) 1.33 (0.56) 1.44 (0.6) 0.423
Triglycerides, mmol/L 1.32 (0.84) 1.33 (0.77) 1.27 (0.85) 0.825
C-reactive protein, mg/L 3 (5) 3 (3) 4 (8) 0.015
International Normalized Ratio 1.01 (0.07) 1.01 (0.05) 1.02 (0.07) 0.247
Infarct side (left), n(%) 99 (45) 54 (45) 45 (44) 0.892
IVT, n(%) 103 (46) 58 (49) 45 (44) 0.501
Onset to imaging, min 132 (210.5) 128 (187) 149.5 (243) 0.383
Onset to groin puncture, min 216 (231) 210.5 (190.75) 230 (271) 0.540
General anesthesia, n(%) 112 (52) 58 (51) 54 (53) 0.786
Collateralization status, n(%)
Good 115 (53) 69 (59) 46 (46) 0.124
Moderate 78 (36) 39 (33) 39 (39)
Poor 25 (11) 10 (8) 15 (15)
Perfusion imaging, n(%)
Volume core 19.90 (27.77) 16.50 (20.82) 23.83 (33.77) 0.543
Volume hypoperfusion 137.78 (72.25) 132.32 (69.94) 144.10 (74.68) 0.142
Volume of tissue at risk 117.88 (66.06) 115.82 (60.98) 120.26 (71.70) 0.692
rCBF core 11.83 (7.05) 13.04 (6.76) 10.43 (7.15) 0.002
rCBF penumbra 18.49 (7.59) 19.43 (7.93) 17.40 (7.06) 0.057
rCBV core 1.62 (0.93) 1.74 (0.90) 1.48 (0.95) 0.037
rCBV penumbra 3.11 (1.44) 3.11 (1.32) 3.11 (1.57) 0.728
(Continues)
|
TA B L E 1 (Continued)
All (n = 222) Favorable (n = 119) Unfavorable (n = 103) p
After treatment
TICI score, n(%)
0 10 (5) 2 (2) 8 (8) 0.011
1 10 (5) 2 (2) 8 (8)
2a 27 (12) 13 (11) 14 (13)
2b 75 (33) 39 (32) 36 (35)
3 100 (45) 63 (53) 37 (36)
NIHSS score after 24 h 6 (10) 4 (4.5) 11 (9.75) <0.001
ICH (SITS-MOST) , n(%) 5 (2) 1 (1) 4 (4) 0.185
Note: Data are frequencies (percentages), as indicated, for categorical variables and median (interquartile range) for continuous variables of clinical data
and patient characteristics. For perfusion imaging, mean (standard deviation) of core and penumbral volumes (cc), as well as their regional blood flow
(ml/100 ml/min) and regional blood volume (ml/100 ml) are shown. Differences in patients with favorable (modified Rankin scale [mRS] score 0–2) and
unfavorable outcome (mRS score 3–6) were tested with Fisher's (categorical) and Wilcoxon's (continuous) test using a significance level of 5%.
Abbreviations: HbA1c, glycated hemoglobin; HDL, high-density lipoprotein; ICH, intracerebral hemorrhage; IVT, intravenous thrombolysis; LDL, low-
density lipoprotein; NIHSS, National Institutes of Health Stroke Scale; rCBF, regional cerebral blood flow; rCBV, regional cerebral blood volume; TICI,
Thrombolysis In Cerebral Infarction scale.
3.0.13, La Ciotat, France). Quantitative (regional cerebral blood infarct core and volume mismatch on favorable functional outcome
flow [rCBF], regional cerebral blood volume [rCBV]), time to max- (mRS scores 0–2 vs. 3–6) and functional improvement (mRS score 0–6)
imum [Tmax], and mean transit time) and qualitative/semi-quanti- using a binary or an ordinal logistic regression model adjusted for age,
tative (time to peak and temporal maximum intensity projections) sex, National Institutes of Health Stroke Scale (NIHSS) score at base-
maps were calculated from perfusion images. Automatic volume line, onset-to-treatment time, thrombolysis and infarct size.
segmentation was used to define the core (with an apparent diffu-
sion coefficient threshold at values <600 × 10 −6 mm2/s) and pen-
umbral volumes (with a predefined Tmax threshold at values >6 s; Prediction models
Figure 1a,b).
We manually drew regions of interest (ROIs) for the lentiform Next, we developed prediction models for favorable functional
nucleus (MCA territory) and thalamus (mostly posterior cerebral outcome (mRS scores 0–2 vs. 3–6) assessing the predictive
artery territory) in one slice, where structures were best demar- power of different patient characteristics and imaging variables
cated on the raw perfusion images (Figure 1c). Additionally, a total and a combination of both. Therefore, we defined five predic-
of seven slices were automatically segmented into five ROIs in each tor sets. The first two sets incorporated only imaging variables
hemisphere using the Sector ROI template (Figure 1d). Here, we from (i) the stroke hemisphere (MRI(S)) or (ii) the entire brain
discarded slices that were two slices below and six slices above the (MRI). The third set (iii) contained only patient characteristics
level of the manual ROIs, because they were prone to artefacts. The (PtC). Two more predictor sets incorporated both PtC and imag-
ROIs in each slice corresponded to anterior cerebral artery territory, ing variables (iv) of the stroke hemisphere (PtC + MRI(S)) or (v)
MCA territory (M ant, M med and M post), and posterior cerebral the entire brain (PtC + MRI). All predictors were taken after clini-
artery territory (Post) of the stroke (S) and contralateral hemisphere. cal and MRI assessment but prior to thrombectomy (see Table
For each perfusion map, we pooled the perfusion values in each ROI S1 for an overview of the predictors). To test whether predic-
across the seven slices by calculating a weighted mean, adjusting for tion would improve if thrombectomy success (Thrombolysis In
the number of pixels per ROI. Cerebral Infarction [TICI] scale) and the early outcome variable
24-h NIHSS score were considered as variables, we computed all
models again after adding these variables (Figure S1). The imag-
Statistical and association analyses ing variables in the respective predictor set were extracted from
six perfusion maps (rCBF, rCBV, Tmax, mean transit time, time
Statistical analysis was performed in open source software R to peak, temporal maximum intensity projections) and, for each
(V3.5.2) [8]. perfusion map, ROI-based features including core and param-
To benchmark our dataset and findings, we first evaluated if infarct eter mismatch were summarized (Figure 1). That is, the following
core and perfusion imaging were associated with functional outcome analyses were repeated seven times while using in each iteration
after EVT. Therefore, we followed a recent meta-analysis of 309 pa- the imaging variables from one of the perfusion maps as well as a
tients of the HERMES collaboration [6]. We estimated the effect of combination of all imaging variables from all perfusion maps (All).
|
1238 HAMANN et al.
F I G U R E 1 Workflow for image-post-processing with Olea Sphere. (a) Diffusion-weighted image and automatic segmentation of core
volume (right) with apparent diffusion coefficient <600 × 10−6 mm2/s and (b) penumbral volume with a time-to-maximum threshold of >6 s.
(c) Manual segmentation of the lentiform nucleus (LN) and thalamus (Th) on stroke (S) and contralateral (CL) side. (d) Automatic region of
interest segmentation of anterior cerebral artery territory (S_Ant, CL_Ant), middle cerebral artery territory (S_M ant/med/post, CL_M ant/
med/post), and posterior cerebral artery territory (S_Post, CL_Post) in seven slices
Prediction models were evaluated in a fivefold cross-validation Three prediction models were used to analyse the automati-
setting. The dataset was split into five test sets and data not con- cally selected or the expert-selected predictors; an RF, a logistic
tained in the respective test set were used for training and valida- regression (logistic regression) with L1 regularization, and a neural
tion. In each of the five iterations, a random forest (RF) was learned network [11]. The trained models were applied to predict the cor-
for variable selection. Variable importance was assessed by ordering responding test set. As the baseline benchmark for our prediction
the variables with respect to the mean decrease in accuracy and by models, we considered Totaled Health Risks in Vascular Events
subsequently selecting the most important predictors for outcome (THRIVE) score [12]. Detailed information about the models and
prediction (Figure S4). For the predictor sets MRI(S) and MRI, all the analysis procedure is provided in Appendix S1B and Figure S1,
variables were used (n = 7). For the remaining predictor sets, the respectively.
number of predictors was reduced to the 10 most important ones. Overall performance was evaluated by computing the area under
Additionally, we evaluated an “Expert” predictor set including vari- the curve (AUC) value of the receiver-operating characteristic (ROC)
ables which were considered most relevant by experts based on curve as well as Brier scores across all test sets. Whereas AUC val-
literature and clinical judgment: age; NIHSS score on admission; sys- ues indicate the model's ability to discriminate the classes, Brier
tolic blood pressure; risk factors (hypertension, diabetes, smoking, scores measure the agreement between predicted and observed
previous ischemic event); preceding IVT; onset to groin puncture values. To test for significant differences between predictor sets,
time; collateralization status; perfusion value of the medial MCA ter- we performed 5% significance tests using bootstrapping to compare
ritory (S_M med); and volume of core and tissue at risk [9,10]. the AUCs of correlated ROC curves. To visualize the influence of a
|
selected variable on predicted outcome, we produced partial depen- 117.88 (66.06) cc, respectively. The only imaging parameters that
dence plots. were significantly different between the two groups were the mean
relative rCBF and rCBV values in the infarct core.
R E S U LT S
Association analyses
Patient and imaging data
The results of the association analysis between ischemic core, vol-
We screened 578 ischemic stroke patients with MCA-M1 occlusion ume mismatch and 90-day mRS score are summarized in Figure 3
and included 222 in the analysis (Figure 2). The distribution of all and were in good agreement with previous studies [6]. Increasing
available patient characteristics with regard to favorable and unfa- core volume was significantly associated with a decrease in odds of
vorable outcomes are shown in Table 1. The median (interquartile favorable outcome (odds ratio of 0.82 with 95% confidence interval
range) age was 73.54 (19.99), and 134 patients (60%) were women. (CI) [0.71, 0.94] per 10 ml) and functional improvement (odds ratio of
The median (interquartile range) NIHSS score on admission was 13 0.89 with 95%-CI [0.81, 0.97] per 10 ml). Tissue at risk was not as-
(9). Successful recanalization (TICI scores 2b-3) was achieved in 175 sociated with outcome. Age was associated with a decreased chance
patients (78%). Of the 222 patients, 119 (53.6%) had a favorable of favorable outcome and functional improvement.
functional outcome after 3 months.
Outcome assessed by mRS score at 3 months was 0 in 32
(14.41%), 1 in 48 (21.62%), 2 in 39 (17.58%), 3 in 32 (14.41%), 4 in 32 Prediction models
(14.42%), 5 in five (2.25%), and 6 in 34 patients (15.32%). Patients
with favorable outcome were younger, more often had no history Figure 4 visualizes the model performance assessed according to
of hypertension or atrial fibrillation, and had a lower NIHSS score at ROC curves and the corresponding AUC values, together with the
baseline. Onset-to-imaging or onset-to-groin-puncture times were Brier scores for favorable functional outcome prediction based on
similar between patients with favorable and unfavorable outcome, the automatically selected variables from the predictor sets, the
as was collateral status and frequency of IVT application. expert-selected predictors and the THRIVE score. For the sake of
The mean (standard deviation) core, hypoperfused area and simplicity, we only show the RF as a prediction model; results ob-
tissue-at-risk volumes were 19.90 (27.77) cc, 137.78 (72.25) cc, tained with the logistic regression and neural network were similar,
M1-Occlusions (Berne 2012-17)

n = 578
Missing mRS at 3 months (n=27)
Initial Imaging n = 551
CT Perfusion No Perfusion
n = 171 n = 96
MR Perfusion
n = 284
Multiple occlusion sites (n = 6)

Unsatisfactory image quality (n = 49)
Previous territorial infarction (n = 3)
No angiography (n = 4)
DSA n = 222; preceding IVT n = 103
F I G U R E 2 Flowchart for patient selection. We screened 578 ischemic stroke patients with middle cerebral artery (MCA)-M1 occlusion
and included 222 into the analysis. We excluded 27 patients due to missing modified Rankin Scale (mRS) values at the 3-month follow-up
time point (n = 551). In this study, we analysed only patients with acute stroke magnetic resonance (MR) imaging including perfusion imaging
(n = 284). A total of 62 patients were further excluded either due to insufficient perfusion data quality, for example, because of excessive
head motion and other artefacts or missing MR perfusion sequence data (n = 49), additional large vessel occlusion other than extension of
the occlusion to the internal carotid artery, or more distal MCA occlusions (n = 6), prior territorial infarction (n = 3), or missing angiography
(n = 4). CT, computed tomography; MT, mechanical thrombectomy; DSA, digital subtraction angiography; IVT, intravenous thrombolysis.
|
1240 HAMANN et al.
F I G U R E 3 Association of core and mismatch volumes with functional outcome. (a) Partial dependence plot derived from binary logistic
regression analysis illustrating the probability for favorable outcome at 3 months (modified Rankin Scale [mRS] scores 0–2) in relation to
ischemic core volume derived from diffusion-weighted imaging. Each red dot indicates an individual patient. (b and c) show the results
of a binary logistic regression for favorable functional outcome (mRS scores 0–2 vs. 3–6) and an ordinal logistic regression (proportional
odds model) for functional improvement (mRS scores 0–6) investigating the association between infarct core (b) and tissue at risk (c) on
the outcome. CI, confidence interval; NIHSS, National Institutes of Health Stroke Scale; OR, odds ratio. [Colour figure can be viewed at
wileyonlinelibrary.com]
F I G U R E 4 Receiver-operating
characteristic curves and corresponding
area under the curve (AUC) values (upper
row) together with Brier scores (lower
row) of the random forest classifier for
predictor sets with imaging variables from
all perfusion maps. Five predictor sets
-magnetic resonance imaging features
from the stroke hemisphere (MRI(S)),
from the entire brain (MRI), patient
characteristics (PtC), and a combination
of both (PtC + MRI(S) and PtC + MRI) -
were compared with the Totaled Health
Risks in Vascular Events (THRIVE) score
and the expert predictor set (Expert). The
dotted line in the upper panel indicates
the random model. In case of perfect
discrimination, the AUC value would reach
its maximum of one and the Brier score
it's minimum of 0. Models are compared
at time point baseline prediction (left
column) and after adding Thrombolysis
In Cerebral Infarction scale (TICI) and
National Institutes of Health Stroke Scale
(NIHSS).
|
highlighting the robustness of our findings. In addition, we visualized of 0.684 with 95%-CI [0.612, 0.756] vs. an AUC of 0.81 with 95%-CI
the predictor sets with imaging variables from all perfusion maps, [0.752, 0.867] and Brier scores of 0.223 vs. 0.176).
which yielded better results than individual perfusion maps in most In all experiments, the predicted probability of a favorable out-
settings. However, results based on the imaging variables from the come was strongly dependent on age (for a summary of the most
other perfusion parameters were similar. A summary of all results is important predictors see Figure S4). In addition, the partial depen-
provided in Figures S2 and S3. dence plots allowed us to elucidate the impact of the individual vari-
An AUC of 0.654 with 95%-CI [0.582, 0.726] and an average ables on outcome prediction (Figure 5). We had one very consistent
Brier score of 0.245 was achieved when using clinical (PtC) predic- finding across all predictor sets and models using any of the perfu-
tors. Imaging information alone performed relatively poorly as a sion parameters: at an age of approximately 78 years, there was a
predictor of functional outcome. Adding image information includ- drastic drop in favorable outcomes of approximately 20%, indepen-
ing infarct core and mismatch to patient variables yielded similar or dent of other patient characteristics.
only slightly better outcome predictions in terms of AUC [95%-CIs]
(AUC 0.654 [0.582, 0.726] for PtC vs. AUC 0.675 [0.602, 0.748] for
PtC + MRI(S), AUC 0.639 [0.565, 0.713] for PtC + MRI and AUC DISCUSSION
0.684 [0.612, 0.756] for the expert predictor set) and Brier score
(0.245 for PtC vs. 0.226 for PtC + MRI(S), 0.240 for PtC + MRI and In stroke patients with MCA-M1 occlusion referred to thrombec-
0.223 for the expert predictor set). Yet, no significant differences tomy within 6 h of symptom onset, functional outcome is variable.
between predictor sets were observed when adding imaging vari- In our cohort, favorable outcome at 3 months (mRS score 0–2) was
ables to patient characteristics (see Appendix S1C for a comparison reached in 54% of patients and was associated with a decrease in
of predictor sets). When we compared the RF with predictor set PtC ischemic core volume. In this homogeneous group of patients re-
to the THRIVE score, we observed similar results. However, in terms ceiving early (<6 h) EVT, adding imaging variables including ischemic
of overall prediction performance our proposed prediction mod- core volume and imaging mismatch to clinical predictors improved
els outperformed THRIVE score (Brier scores of 0.293 for THRIVE the prediction of a stroke patient's outcome only slightly. The driv-
vs. 0.245 for PtC). As expected, prediction was significantly better ing factor for functional outcome prediction was age. Considering
when recanalization success (TICI grade) and the early clinical out- the wide range of outcomes after stroke even in patients with similar
come measure NIHSS score after 24 h were added (For the best vascular occlusions, prognostic research is an evolving field. Several
model, the RF with the expert predictor set, we achieved an AUC prognostic scores have been established in the general stroke pop-
ulation [13–18], in stroke patients treated with IVT [19,20], and in
1·0 patients eligible for endovascular treatment (e.g., THRIVE, HIAT,
●
●
●
●
● ●
● ●
●
●
●●
●
MR PREDICTS) [9,12,21]. Some of these scores were validated and
● ● ● ● ●
compared recently in a large dataset including hemorrhagic and

●
● ●● ● ● ●
● ● ●
● ●● ●● ●
● ● ●● ●
● ●●
● ● ● ● ● ● ●
● ● ● ● ●
●
●● ● ● ●● ●● ●
● ● ●● ●
●
ischemic stroke (n = 10,777) from the Virtual International Stroke

●● ● ● ● ● ● ● ●
0·8
● ●
● ● ●
Probability for favorable outcome
● ● ● ● ●
●●
● ●
●
● ● ● ● ●● ●
● ●
● ●●●
Trials Archive (VISTA). However, even when the highly heteroge-

● ● ● ● ●
●
●
●
neous VISTA patient cohort with stroke in any vascular territory

●
0·6 independent of vessel occlusion status was used (thus enhancing ac-
curate prediction), the resulting AUCs for outcome prediction were
only between 0.6 and 0.79 [22], similar to our findings.
0·4 In our models for outcome prediction, we incorporated patient
● ●
●
●
●
●
● baseline variables and MRI estimates of the infarct core and hy-
●
poperfused area from the stroke hemisphere as well as whole-brain

● ● ● ●
●
● ●●
● ●
● ●
● ●
● ●
● ● ●
0·2
● ● ●●
MRI readouts. AUCs for different prediction models using varying

● ● ● ●
● ● ●
● ●● ●● ●● ●
● ● ●
● ● ●
● ● ● ● ● ●●
●
● ● ● ●
● ● ● ● ●●
● ●
combinations of potential predictors were between 0.6 and 0.7, and

● ● ● ● ● ●
● ● ●● ●
●● ●
●● ●
●
● ● ● ●● ● ●
● ●● ●
●
similar to the THRIVE score [23]. In terms of overall prediction per-

0·0 formance, our models outperformed THRIVE score, as indicated by
20 40 60 80 100
Age (y) a smaller Brier score. The robustness of our findings is supported
by similar results for different prediction models, predictor sets and
F I G U R E 5 Partial dependence plots of the random forest model perfusion variables.
in the predictor set PtC + MRI (with regional cerebral blood volume To derive our prediction models, we applied machine-learn-
as imaging feature). Each curve represents the change in outcome ing methods, which have recently succeeded the development of
prediction for one patient, when changing the variable of interest
novel outcome prediction tools in other fields of cardiovascular
(x-axis) and holding the remaining covariables constant. The red
disease [24,25]. Expectations are high that these highly efficient
dots highlight the actual observed values. The thick, coloured line
marks the average dependence curve algorithms are capable of outperforming classic score-based pre-
diction tools in medicine [24,26]. While machine-learning was
|
1242 HAMANN et al.
applied to stroke outcome prediction analyses, so far, none of AU T H O R C O N T R I B U T I O N S

these studies have incorporated diffusion- or perfusion-weighted J.H. prepared the first draft of the manuscript and performed pa-
imaging variables [27–29]. tient clinical and image data analysis. L.H. did the statistical analyses,
Older age (>78 years) was an important determinant of outcome data visualization and contributed in designing the statistical analy-
in our study. Although age is known to be a critical factor for re- sis plan and in writing the manuscript. C.D.W. contributed to image
covery after stroke [30], the abrupt change in the probability of a data analysis. M.P. assisted with image processing. A.B. contributed
good outcome observed in the present study has not been described to planning and conduction of image data analysis. T.D., L.P., J.K.,
before. U.F., J.G., M.A. and R.W. participated in patient enrolment, data col-
A limitation of this study is that it had a retrospective, single-cen- lection, and data analysis. A.R.L. and C.S. contributed to data anal-
ter cohort design, with data collected over 5 years (2012–2017), a ysis and interpretation. B.S. designed the statistical plan and data
period during which thrombectomy methods have evolved. We se- analysis. S.W. conceived the study, contributed to data analysis and
lected only patients deemed to be good thrombectomy candidates, drafting of the manuscript. All authors critically reviewed the report,
which resulted in a homogenous cohort of patients, but limits con- and approved the final version.
clusions about treatment effects or extended time windows (>6 h) of
treatment. The outcomes observed in the present cohort ere very DATA AVA I L A B I L I T Y S TAT E M E N T
similar to data from randomized trials pooled by the HERMES collab- The data that support the findings of this study are available from
oration [2]. However, since we analyzed data from treated patients the corresponding author upon reasonable request. Code for statis-
only, we cannot make inferences about treatment effect (such as tical analysis and visualization is made available on Github (https://
EVT vs. no EVT). github.com/liherz/stroke_outcome_predic tion_ML).
In conclusion, for patients with LVO stroke referred to throm-
bectomy within 6 h of symptom onset, functional disability at ORCID
3 months after EVT can be predicted moderately well using pa- Susanne Wegener https://orcid.org/0000-0003-4369-7023
tient baseline variables. Increasing age had the strongest predic-
tive ability for unfavorable outcome in these patients. ROI-derived REFERENCES
imaging variables such as infarct core, area of hypoperfusion or 1. Phipps MS, Cronin CA. Management of acute ischemic stroke. BMJ.
imaging mismatch indicated an enhancement in outcome predic- 2020;368:l6983.
2. Goyal M, Menon BK, van Zwam WH, et al. Endovascular throm-
tion in these LVO stroke patients treated early, however, no sig-
bectomy after large-vessel ischaemic stroke: a meta-analysis
nificant improvement was observed when we added the imaging of individual patient data from five randomised trials. Lancet.
information. Our results support current treatment guidelines for 2016;387:1723-1731.
thrombectomy within early time windows, which maintain that 3. Campbell BCV, Donnan GA, Lees KR, et al. Endovascular stent
thrombectomy: the new standard of care for large vessel ischaemic
large core and lack of imaging mismatch should not be used to
stroke. Lancet Neurol. 2015;14:846-854.
exclude patients from treatment because their recovery may be 4. Campbell BC, Mitchell PJ, Investigators E-I. Endovascular therapy
better than anticipated. It is likely that MRI may contain additional for ischemic stroke. N Engl J Med. 2015;372:2365-2366.
valuable information for outcome prediction in LVO stroke pa- 5. Albers GW, Marks MP, Kemp S, et al. Thrombectomy for stroke at
6 to 16 hours with selection by perfusion imaging. N Engl J Med.
tients, since the available ROI-based imaging variables might not
2018;378:708-718.
capture all the relevant image information. Novel approaches such 6. Campbell BCV, Majoie C, Albers GW, et al. Penumbral imaging and
as deep-learning-based methods promise to outperform standard functional outcome in patients with anterior circulation ischaemic
image analysis techniques in a wide range of applications [31]; stroke treated with endovascular thrombectomy versus medical
therefore, we plan to perform a follow-up study using deep-learn- therapy: a meta-analysis of individual patient-level data. Lancet
Neurol. 2019;18:46-55.
ing approaches to further improve functional outcome prediction
7. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis
in LVO stroke patients after early EVT. and prognostic research: what, why, and how? BMJ. 2009;338:b375.
8. Team RC. R: A language and environment for statistical computing.
AC K N OW L E D G M E N T S Vienna, Austria: R Foundation for Statistical Computing; 2014.
9. Venema E, Mulder M, Roozenbeek B, et al. Selection of patients for
None.
intra-arterial treatment for acute ischaemic stroke: Development
and validation of a clinical decision tool in two randomised trials.
C O N FL I C T O F I N T E R E S T BMJ. 2017;357:j1710.
J.H., L.H., C.D.W., T.D., A.B., M.P., L.P., U.F., C.S., A.R.L., J.G., M.A., 10. Corso G, Bottacchi E, Tosi P, et al. Outcome predictors in first-ever
ischemic stroke patients: a population-based study. Int Sch Res
B.S. and R.W. report no conflicts of interest. J.K. has received travel
Notices. 2014;2014:904647.
grants from Pfizer and Stryker and academic grants from the SAMW/ 11. Friedman JH, Tibshirani R, Hastie T. The elements of statistical
Bangerter Foundation and the Swiss Stroke Society. S.W. has re- learning. New York, NY: Springer, 2008.
ceived academic grants from the Swiss National Science Foundation, 12. Flint AC, Cullen SP, Faigeles BS, Rao VA. Predicting long-term out-
come after endovascular stroke treatment: the totaled health risks in
the UZH (Clinical Research Priority Program Stroke), the Swiss Heart
vascular events score. AJNR Am J Neuroradiol. 2010;31:1192-1196.
foundation and the Olga Mayenfish foundation.
|
13. Ntaios G, Faouzi M, Ferrari J, Lang W, Vemmos K, Michel P. An in- 25. Al'Aref SJ, Anchouche K, Singh G, et al. Clinical applications of ma-
teger-based score to predict functional outcome in acute ischemic chine learning in cardiovascular disease and its relevance to cardiac
stroke: the astral score. Neurology. 2012;78:1916-1922. imaging. Eur Heart J. 2019;40:1975-1986.
14. Reid JM, Dai D, Delmonte S, Counsell C, Phillips SJ, MacLeod 26. Patel UK, Anwar A, Saleem S, et al. Artificial intelligence as an
MJ. Simple prediction scores predict good and devastating out- emerging technology in the current care of neurological disorders.
comes after stroke more accurately than physicians. Age Ageing. J Neurol. 2019. https://doi.org/10.1007/s00415-019-09518-3.
2017;46:421-426. 27. van Os HJA, Ramos LA, Hilbert A, et al. Predicting outcome of en-
15. O'Donnell MJ, Fang J, D'Uva C, et al. The plan score: a bedside pre- dovascular treatment for acute ischemic stroke: potential value of
diction rule for death and severe disability following acute ischemic machine learning algorithms. Front Neurol. 2018;9:784.
stroke. Arch Intern Med. 2012;172:1548-1556. 28. Asadi H, Dowling R, Yan B, Mitchell P. Machine learning for out-
16. Saposnik G, Kapral MK, Liu Y, et al. Iscore: a risk score to predict come prediction of acute ischemic stroke post intra-arterial ther-
death early after hospitalization for an acute ischemic stroke. apy. PLoS One. 2014;9:e88225.
Circulation. 2011;123:739-749. 29. Monteiro M, Fonseca AC, Freitas AT, et al. Using machine learning to
17. Konig IR, Ziegler A, Bluhmki E, et al. Predicting long-term outcome improve the prediction of functional outcome in ischemic stroke pa-
after acute ischemic stroke: a simple index works in patients from tients. IEEE/ACM Trans Comput Biol Bioinform. 2018;15:1953-1959.
controlled clinical trials. Stroke. 2008;39:1821-1826. 3 0. Lee WI, Mitchell P, Dowling R, Yan B. Clinical factors are significant
18. Kwok CS, Potter JF, Dalton G, et al. The soar stroke score pre- predictors of outcome post intra-arterial therapy for acute isch-
dicts inpatient and 7-day mortality in acute stroke. Stroke. aemic stroke: a review. J Neuroradiol. 2013;40:315-325.
2013;44:2010-2012. 31. Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3d
19. Strbian D, Meretoja A, Ahlhelm FJ, et al. Predicting outcome of iv cnn with fully connected crf for accurate brain lesion segmentation.
thrombolysis-treated ischemic stroke patients: the dragon score. Med Image Anal. 2017;36:61-78.
Neurology. 2012;78:427-432.
20. Saposnik G, Guzik AK, Reeves M, Ovbiagele B, Johnston SC.
Stroke prognostication using age and nih stroke scale: Span-100. S U P P O R T I N G I N FO R M AT I O N
Neurology. 2013;80:21-28. Additional supporting information may be found online in the
21. Hallevi H, Barreto AD, Liebeskind DS, et al. Identifying patients
Supporting Information section.
at high risk for poor outcome after intra-arterial therapy for acute
ischemic stroke. Stroke. 2009;40:1780-1785.
22. Quinn TJ, Singh S, Lees KR, Bath PM, Myint PK, Collaborators
V. Validating and comparing stroke prognosis scales. Neurology.
2017;89:997-1002. How to cite this article: Hamann J, Herzog L, Wehrli C, et al.
23. Kastrup A, Brunner F, Hildebrandt H, et al. Thrive score predicts Machine-learning-based outcome prediction in stroke patients
clinical and radiological outcome after endovascular therapy or
with middle cerebral artery-M1 occlusions and early
thrombolysis in patients with anterior circulation stroke in every-
day clinical practice. Eur J Neurol. 2017;24:1032-1039. thrombectomy. Eur J Neurol. 2021;28:1234–1243. https://doi.
24. Than MP, Pickering JW, Sandoval Y, et al. Machine learning to org/10.1111/ene.14651
predict the likelihood of acute myocardial infarction. Circulation.
2019;140:899–909.

Machine-Learning-Based Outcome Prediction in Stroke Patients With Middle Cerebral Artery-M1 Occlusions and Early Thrombectomy

Uploaded by

Copyright:

Available Formats

Machine-Learning-Based Outcome Prediction in Stroke Patients With Middle Cerebral Artery-M1 Occlusions and Early Thrombectomy

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Machine-Learning-Based Outcome Prediction in Stroke Patients With Middle Cerebral Artery-M1 Occlusions and Early Thrombectomy

Uploaded by

Copyright:

Available Formats

|

Received: 30 October 2020 Accepted: 16 November 2020

Machine-learning-based outcome prediction in stroke

Janne Hamann1 | Lisa Herzog1,2,3 | Carina Wehrli1,4 | Tomas Dobrocky5 |

Ischemic stroke treatment aims at recanalization of the occluded Patient data

All (n = 222) Favorable (n = 119) Unfavorable (n = 103) p

Age, years 73.54 (19.99) 68.80 (18.23) 79.20 (17.70) <0.001

All (n = 222) Favorable (n = 119) Unfavorable (n = 103) p

M1-Occlusions (Berne 2012-17)

Missing mRS at 3 months (n=27)

Initial Imaging n = 551

Multiple occlusion sites (n = 6)

DSA n = 222; preceding IVT n = 103

compared recently in a large dataset including hemorrhagic and

ischemic stroke (n = 10,777) from the Virtual International Stroke

Trials Archive (VISTA). However, even when the highly heteroge-

neous VISTA patient cohort with stroke in any vascular territory

poperfused area from the stroke hemisphere as well as whole-brain

MRI readouts. AUCs for different prediction models using varying

combinations of potential predictors were between 0.6 and 0.7, and

similar to the THRIVE score [23]. In terms of overall prediction per-

applied to stroke outcome prediction analyses, so far, none of AU T H O R C O N T R I B U T I O N S

You might also like