Machine learning identification of maternal inflammatory response and histologic choroamnionitis from placental membrane whole slide images

Abstract

Introduction: The placenta forms a critical barrier to infection through pregnancy, labor and, delivery. Inflammatory processes in the placenta have short-term, and long-term consequences for offspring health. Digital pathology and machine learning can play an important role in understanding placental inflammation, and there have been very few investigations into methods for predicting and understanding Maternal Inflammatory Response (MIR). This work intends to investigate the potential of using machine learning to understand MIR based on whole slide images (WSI), and establish early benchmarks.
Methods: To that end, use Multiple Instance Learning framework with 3 feature extractors: ImageNet-based EfficientNet-v2s, and 2 histopathology foundation models, UNI and Phikon to investigate predictability of MIR stage from histopathology WSIs. We also interpret predictions from these models using the learned attention maps from these models. We also use the MIL framework for predicting white blood cells count (WBC) and maximum fever temperature ($T_{max}$).
Results: Attention-based MIL models are able to classify MIR with a balanced accuracy of up to 88.5% with a Cohen’s Kappa ($\kappa$) of up to 0.772. Furthermore, we found that the pathology foundation models (UNI and Phikon) are both able to achieve higher performance with balanced accuracy of 87.2 % and 88.5 % respectively, and $\kappa$ of 0.751 and 0.772 respectively, compared to ImageNet-based feature extractor (EfficientNet-v2s) which achieves a balanced accuracy of 83.7% and $\kappa$ of 0.724. For WBC and $T_{max}$ prediction, we found mild correlation between actual values and those predicted from histopathology WSIs.
Discussion: We used MIL framework for predicting MIR stage from WSIs, and compared effectiveness of foundation models as feature extractors, with that of an ImageNet-based model. We further investigated model failure cases and found them to be either edge cases prone to interobserver variability, examples of pathologist’s overreach, or mislabeled due to processing errors.

A machine learning model can identify maternal inflammatory response in placental membranes from whole slide images with high accuracy.

Medical imaging foundation models are better feature extractors for MIR identification than model pretrained for ImageNet classification.

As an exploratory analysis, we found white blood cell count and maximum fever temperature to be mildly predictable from placental whole slide images.

1 Introduction

Maternal Inflammatory Response: Definitions and Pathogenesis

Acute Placental Inflammation (API) is categorized into Maternal Inflammatory Response (MIR) or Fetal Inflammatory Response (FIR) depending on the origin of the inflammatory cells. MIR is characterized by extravasation of maternal neutrophils with movement towards the chorionic layer, followed by moving across the amnion and into amniotic space. MIR is divided into 3 stages. These stages are defined according to the Amsterdam criteria [35]. Stage 1 corresponds to subchorionitis, where where maternal neutrophils are limited to the cellular chorion in the membranes and subchorionic fibrin in the chorionic plate. Stage 2, chorioamnionitis is characterized by neutrophilic migration into the fibrous chorion and amniotic connective tissue. Stage 3, chorioamnionitis with amnion necrosis, is characterized by presence of neutrophil karyorrhectic debris, asbcess formation, and thickened basement membranes. MIR is classically seen in the presence of ascending infection by Group B. Streptococcus, E. coli and other pathogenic bacteria [39, 41]. However, organisms are not always identified and some authorities argue that MIR may be the result of sterile inflammation. [51, 42, 21, 23, 38, 32]

Immediate and long-term implications of MIR

MIR is associated with several adverse health outcomes. Bacteria causing MIR may enter the maternal or fetal circulation, causing puerperal fever or early onset neonatal sepsis, respectively. When MIR co-occurs with FIR, there is an increased risk of neonatal death [25, 27]. MIR2 is a risk factor for recurrent wheeze [22, 30], asthma [22, 16, 12], and chronic lung disease [12]. MIR has also been associated with lower mental development index [50], and increased risk of autism spectrum disorder [46]. In a small case-control study, severe MIR has also been linked to cerebral palsy albeit indirectly [37].

Challenges in analysing MIR

The stages of MIR are broadly defined with significant variability in how different pathologists classify these patterns and the cut-off between different stages [17, 36]. In a study by Redline et al. [36], showed a moderate kappa agreement of $0.58$ for the diagnosis of no MIR (MIR0) vs. any MIR (MIR1-3). An older study using a previous definition of histologic chorioamnionitis showed a kappa of 0.72 [43]. A study using secondary review of generalist pathologist diagnoses by an expert placental pathologist showed a kappa for any MIR of 0.6 [13]. This variability creates challenges for analysis. This may explain the relatively loose association between histologic chorioamnionitis and clinical chorioamnionitis or markers of maternal systemic inflammation [34, 32, 23, 28].

Machine Learning and Multiple Instance Learning

Machine Learning along with digital pathology allows investigations to be conducted with larger datasets [5, 10, 44]. Consequently, there have been several studies conducted in recent years utilizing machine learning to build tools for pathological diagnoses, and analyse digital pathology datasets. These include both local-level tasks like nuclei segmentation, and global-level tasks prostate cancer detection [45].

Whole slide images are up to 100,000 x 60,000 pixels – too large for efficient computation. The earliest studies in computational pathology randomly selected patches from WSI in the hope that they would carry diagnostic material from the slide. Then convolutional neural networks (ResNet, EfficientNet) or attention-based models (ViT) were finetuned on the resultant dataset [24]. Later, multiple instance learning (MIL) based methods were proposed, that take all the patches into account, and learn to ascribe importance to patches. The importance scores can be calculated using probability of belonging to the positive class [4], as a proxy for selecting highly important patches. Another approach was proposed in [26], where attention mechanism was used to force the model to ascribe importance to patches during classification. These attention scores were calculated based on features extracted using some pretrained model e.g., ResNet50, instead of using raw patches directly. Recently, several groups have released pathology foundation models. Unlike ResNet or EfficientNet, these models use transformer architecture and have been trained on tens of thousands to millions of slides [15, 6, 53]. These models show superior performance on benchmarks. However, the training sets lack placental slides and benchmarks are nearly all on detection, classification, and mutation identification of neoplasia.

Recently, several studies have been conducted employing machine learning and MIL for predicting and analyzing placenta [29]. Chen et al. [7] used CNNs to design a system for morphological characterization, and clinically meaningful feature analysis of placentas from photos. Zhang et al. [52] propose a Cycle-GAN along with attention module and saliency constraint to enable cross-domain image segmentation by translating target domain placenta pictures (from 1 hospital) into source domain (a different hospital), and adapting a pretrained segmentation model to segment them. Clymer et al. [9] designed multiresolution CNNs to classify desidual vasculopathy in placental membranes. Mobadersany et al. [31] used MIL framwork to improve prediction of gestational age from placental WSI. Andreasen et al. [3] proposed deep learning method for placenta segmentation from obstetric ultrasound. Goldstein et al. [18] used the MIL framework for classification of placental villous infarction, perivillous fibrin deposition, and intervillous thrombus. Patnaik et al. [33] used a pretrained ResNet-18 for feature extraction from placental histopathology images, and classify maternal vascular malperfusion. Irmakci et al. [19] investigated the challenges posed to Machine Learning models by presence of tissue contaminants e.g., floaters in WSIs. Studies have also been conducted to classify cell and regions in the placental disc [14, 49]

Recently, Chou et al. [8] used machine learning and other quantitative techniques to characterize maternal inflammatory response (MIR) in placental membranes.

In this study, we investigate MIR stage prediction from WSI.

2 Methods

Dataset: Patients and Placentas

The study was approved by our Institutional Review Board as STU00214052 and operated under waiver of consent. Inclusion criteria were singleton placentas examined at our institution between 2010 and 2024. We selected cases with and without MIR for retrieval and scanning. Slides were digitized on a Leica GT450 scanner with a 40× objective magnification (0.263 $\mu m$ per pixel). Patient demographic, laboratory, and placental pathology information were extracted from our electronic data warehouse. Slides were linked to pathology reports via optical character recognition of the labels with human review. MIR stage involving the membranes was identified from reports using regular expressions [48]. Unfortunately, our site does not grade MIR. Highest fever ($T_{max}$) was defined as the highest maternal temperature recorded in the 72 hours before delivery. White blood cell count (WBC) was defined as the highest maternal white blood cell count in samples collected in the 72 hours before delivery.

Data processing for MIR prediction

We simplified the MIR stage classification problem by combining data from MIR0 and MIR1 into a one class, and MIR2 and MIR3 into another. Patches of size $224\times 224$ with no overlap were extracted at 20x from whole slide images of placental membranes from each patient. We used three different feature extractors. EfficientNetV2S [47] was trained using the ImageNet dataset of everyday objects (non-medical) [11]. Phikon [15] and UNI [6] are pathology foundation models, trained on diverse datasets of histology images, though not including placenta. These features were stored in ’tfrecords’ files for the aggregation and classification step in MIL pipeline.

Dataset was split into training, validation and testing splits, where $20\%$ of the data (678 samples) were held-out as the test set, and the training (2436 samples) and validation set (271 samples) were used for training and hyperparameter optimization respectively.

Data processing for white blood cell count and fever prediction

For white blood cell count and $T_{max}$ prediction, we used UNI and Phikon for extracting features which were then aggregated using learned attention scores, and used by the regression network for prediction. The data split between training, validation, and test was created in a similar manner as for MIR prediction. However, there are fewer data point available with WBC (1320 training, 142 validation, and 370 test samples) and $T_{max}$ (2256 training, 251 validation, and 632 test samples) information.

Machine Learning Models

We used an attention-based multiple instance learning architecture similar to [26]. The architecture splits information processing into two stages- First stage involves patch-level feature extraction using a pretrained network. The pretrained network filters information from patches, that is available during stage 2 i.e., aggregation for the final prediction. Thus, the representations learnt by feature extractor model can play a crucial role in the final predictions. Stage 2 involves aggregation of patch-level feature to make the final prediction. The architecture uses attention to weigh different patch features. These attention weights are themselves learned during training.

Figure 1: (A) Model architecture for MIR stage classification. Each MIR class has a corresponding set of attention weights for the patch embeddings [26]. The aggregated embeddings for each class are processed by fully connected classification layers followed by a softmax layer to convert to class probabilities. The class with highest probability is the model prediction. Attention maps are visualized by coloring each patch based on corresponding attention weight. (B) Model architecture for white blood cell count (WBC) and maternal highest temperature ($T_{max}$) prediction

Foundation models for feature extraction:

As a first step for the MIL pipeline, we extracted patch-level features from each slide using different feature extractors. We used EfficientNet-v2s [47], trained on ImageNet dataset (a non-medical dataset) [11], which served as our model performance baseline. We investigated and compared features from two pathology foundation models- 1) Phikon [15], and 2) UNI [6], against the baseline.

Loss functions:

For MIR stage prediction we used hinge loss. We also investigated effect of class weighting along with these loss functions. For white blood cell count prediction and fever prediction, we used mean squared error (mse), and weighted mean squared error (wmse). For weighted mean squared error calculation, we first split the ground truth values into $10$ bins, then assigned weights to these bins which are inversely proportional to the number of samples in the corresponding bin. The mean squared error corresponding to each sample is then weighted according to the bin its ground truth belongs to.

Evaluation Metrics

For MIR prediction, we used area under the receiver operator characteristic curve (auroc), Balanced Accuracy, Mathew’s Correlation Coefficient (mcc), and Cohen’s Kappa ($\kappa$) to evaluation the classification task. To evaluate white blood cell count prediction and fever prediction, we used mean squared error (MSE), mean absolute error (MAE), and $R^{2}$-score.

3 Results

Table 1 summarizes the information about patients. Patients with MIR0,1 and MIR2,3 were of similar maternal age at delivery, and had a similar gestational age, although with a higher variation. Patients with MIR2,3 were more likely to be nulliparous, and less likely to have diabetes and hypertension in pregnancy, than patients with MIR0,1.

Table 1: The median (IQR) values for maternal age and gestational age are shown below, along with the count (percentage) for parous, diabetes, and hypertension. p-values are reported from Mann-Whitney U rank test for Maternal and Gestational Age, and Chi-squared test for Parous, Diabetes, and Hyptension.

value	MIR 0,1 (n = 2786)	MIR 2, 3 (n = 599)	p_mir01_mir23
Maternal Age (yrs.)	33 (30 - 36)	32 (28 - 36)	6.74E-07
Gestational Age (wks.)	37 (34 - 39)	37 (27 - 39)	0.000275
Parous	1352 (0.49)	221 (0.38)	6.15E-05
Diabetes in Pregnancy	333 (0.12)	56 (0.09)	0.088
Hypertension in Pregnancy	764 (0.27)	80 (0.13)	3.97E-10

MIR stage prediction: Balanced accuracy, MCC, Cohen’s kappa

Table 2 shows auroc, balanced accuracy, Mathew’s correlation coefficient (mcc), and Cohen’s kappa ($\kappa$) values for EfficientNet, Phikon, and UNI features. These models achieved a balanced accuracy and Cohen’s Kappa ($\kappa$) of $>83.7\%$, and $>0.724$ respectively.

Table 2: Comparison of performance between different feature extractors in terms of auroc, balanced accuracy, mcc, and Cohen’s Kappa, specificity, and sensitivity. Values reported are for the test set (N=676).

	AUROC	Balanced Acc.	MCC	$\kappa$	Specificity	Sensitivity
EfficientNet	0.837	83.7 %	0.729	0.724	0.97	0.70
Phikon	0.885	88.5 %	0.772	0.772	0.96	0.81
UNI	0.872	87.2 %	0.751	0.751	0.96	0.79

Figure 2: Test set confusion matrices for EfficientNet (top), UNI (bottom-left), Phikon (bottom-right).

Comparison between feature extractors

Table 2 shows a comparison between a feature extractor trained on ImageNet (EfficientNet-v2s), and 2 pathology foundation models (UNI and Phikon). EfficientNet is prone to making more false negative predictions compared to UNI and Phikon, and is worse at identifying MIR2,3. We found that both the foundation models perform significantly better than EfficientNet in terms of balanced accuracy. In addition, we found that Phikon is marginally better than UNI.

Visualizing slide-level and patch-level features

Figure 3 a the t-SNE visualization of slide-level features from the test set, aggregated by the pooling layer of the Phikon MIL model. We see slides being separated into two major clusters. Data points with MIR0 and MIR1 are shown in dark blue and light blue respectively. MIR2 and MIR3 shown in red, and light red respectively. The aggregated embeddings show definite, but imperfect separation between the MIR0,1 and MIR2,3 slides. The embeddings also show a gradient with MIR0 slides (dark blue) farthest to the left and MIR3 slides (bright red) mostly to the right. MIR1 and MIR2 are placed in between, with MIR1 being closer to MIR0 than MIR3, while MIR2 is closer to MIR3.

Figure 4 shows t-SNE visualization for top-1 attention patches from all the wsi. The weighted average featuer vectors encode information relevant to the MIR stage diagnosis, as reflected by separate blue and red clusters for MIR0,1 and MIR2,3 respectively.

Figure 3: t-SNE space of aggregated features from Phikon in the test set. Colors represent the true class. MIR0 is shown in dark blue, MIR1 in light blue, MIR2 in light red, and MIR3 in dark red.

Figure 4: t-SNE embeddings of top-1 attended patches for attention branch 1 (See model architecture in Fig. 1), from each slide in the test set. Image patches corresponding to randomly sampled patches are shown with patching outlines highlighting the original diagnosis (MIR Stage) of corresponding WSIs. Blue patches are largely normal placenta or some neutrophil infiltration. The red patches show stroma with greater neutrophil infiltration. All the embeddings are shown in the inset.

Investigating model failure cases: False positives and negatives.

To understand how the model functions or malfunctions, we investigated a few cases where the model gives false-positive or false-negative results. Specifically, we looked at the extreme false positive (rightmost blue points in Figure 3), and extreme false negative (leftmost red points in Figure 3). The attention heatmap, and top-10 attention patches for a false positive and false negative are shown in Fig. 5 and 6 respectively.

Figure 5: An example of a false positive. (A) WSI, (B) Attention Heatmap, (C) Top-10 attention patches from attention branch 1 (See model architecture in Fig. 1). This is a case with severe MIR1 with decidual necrosis, but does not meet criteria for MIR2.

Figure 6: An example of false negative. (A) WSI, (B) Attention Heatmap, (C) Top-10 attention patches from attention branch 1 (See model architecture in Fig. 1). Review of high attention patches showed amnion with activated-appearing macrophages. Manual review of the whole slide showed scant inflammation, which may not be classified as MIR2 by all observers.

White blood cell count prediction

Table 3 shows performance of UNI and Phikon-based regression networks for white blood cell prediction. UNI performs slightly better than Phikon in terms of RMSE, MAE, and $R^{2}$-score values. However, both the models show weak correlation between predicted and actual values of WBC.

Figure 7: UNI predicts white blood cell count (WBC) with an $R\sup{2}$ of 0.071. Phikon predicts WBC with a R2 of 0.043 (See Table 3)

Maternal highest temperature prediction

Table 3 shows performance of UNI and Phikon-based regression networks for fever prediction ($T_{max}$). UNI performs slightly better than Phikon in terms of RMSE, MAE, and $R^{2}$-score. Both the models show mild correlation between predicted and actual values of $T_{max}$.

Figure 8: UNI predicts maximum fever temperature ($T_{max}$) with a R2 of 0.396 and rmse of 1.021. Phikon predicts $T_{max}$ with a R2 of 0.291 and rmse of 1.125. (See Table 3)

Table 3: Performance for white blood cell count (WBC) prediction and fever ($T_{max}$) prediction using UNI features

value	Model	RMSE	MAE	$R^{2}$	slope
WBC	UNI	5.370	4.045	0.071	0.123
	Phikon	7.744	6.392	0.043	0.186
$T_{max}$	UNI	1.021	0.856	0.132	0.396
	Phikon	1.125	0.999	0.123	0.291

4 Discussion

Automated analysis of MIR

We investigated the feasibility of predicting MIR stages (MIR0,1 and MIR2,3) from whole slide images. We found that attention-based MIL models are able to classify MIR with a balanced accuracy of up to 88.5% with a Cohen’s $\kappa$ of up to 0.772. Furthermore, we found that the pathology foundation models (UNI and Phikon) are both able to achieve higher performance with balanced accuracy of 87.2 % and 88.5 % respectively, and Cohen’s Kappa $(\kappa)$ of 0.751 and 0.772 respectively, compared to ImageNet-based feature extractor (EfficientNet-v2s). This is despite the fact that both the pathology foundation models have not been pretrained on any placental data.

We also investigated prediction of white blood cell count (WBC), and fever ($T_{max}$) from WSI. We found that the regression model with the UNI feature extractor, shows a moderate correlation between the predicted and actual $T_{max}$, indicating a moderate ability to predict fever from histopathological images. Prediction of white blood cell count shows a much weaker correlation.

Comparison between feature extractors In this study, we found that feature from histology foundation models- UNI, and Phikon, outperform ImageNet features for MIR stage prediction, despite the foundation models not being pretrained on any placenta data. In addition, we found Phikon to be slightly better than UNI.

Interobserver variability Of the cases misclassified by our model, many showed borderline findings between MIR1 and MIR2. This highlights the challenges of interobserver variability in building and assessing models. In a different context, we found that training on noisy labels from multiple observers can yield expert level performance [2]. Future studies could show similar improvement.

Clinical chorioamnionitis is defined by maternal fever, fetal or maternal tachycardia, uterine tenderness, and foul discharge, and is often accompanied by elevated WBC count [1, 40, 20]. As an exploratory study, we tested whether we could estimate $T_{max}$ or WBC using placental histology. We found weak, but statistically significant correlations.

Strengths and limitations:

One of the major strengths of this study is that the dataset the large (N = 3,385) real-world dataset. Further, we have investigated the use of medical imaging foundation models (Phikon and UNI) as feature extractors in the MIL pipeline, and demonstrated that they perform better than using features extracted from model pretrained on images from everyday objects i.e., ImageNet (EfficientNet-v2s), for MIR classification, even though no placental images were used to train Phikon and UNI. We also showed that Phikon and UNI both achieve similar performance for the MIR prediction task. Thus highlighting the need for finetuning on placenta data for these models to extract finer details from placenta images, and achieve further performance gains. We investigated model’s ability to predict fever from histopathological images, and found moderate correlation between predicted and actual $T_{max}$.

One of the weaknesses of this study is that the dataset was sourced from a single site. The models might not generalize to datasets sources from other sites. Further limitations include that other maternal or fetal signs of inflammation were not considered. The rarity of neonatal sepsis poses challenges to predict it, using our data. Also, due to the nature of our dataset, long-term neonatal outcomes could not be investigated.

Contributions

Conception of the work: J.A.G., L.A.D.C., A.S.; Patient selection/scanning: J.A.G.; Technique and tool development: A.S., R.N., L.A.D.C., M.A.; Data preparation: A.S.; Experiments: A.S.; Analysis: A.S., J.A.G.; Manuscript: A.S.; Editing: A.S., J.A.G., L.A.D.C.

Declaration of competing interest

The authors have no relevant disclosures