-
PLUTO: Pathology-Universal Transformer
Authors:
Dinkar Juyal,
Harshith Padigela,
Chintan Shah,
Daniel Shenker,
Natalia Harguindeguy,
Yi Liu,
Blake Martin,
Yibo Zhang,
Michael Nercessian,
Miles Markey,
Isaac Finberg,
Kelsey Luu,
Daniel Borders,
Syed Ashar Javed,
Emma Krause,
Raymond Biju,
Aashish Sood,
Allen Ma,
Jackson Nyman,
John Shamshoian,
Guillaume Chhor,
Darpan Sanghavi,
Marc Thibault,
Limin Yu,
Fedaa Najdawi
, et al. (8 additional authors not shown)
Abstract:
Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this wor…
▽ More
Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Improved statistical benchmarking of digital pathology models using pairwise frames evaluation
Authors:
Ylaine Gerardin,
John Shamshoian,
Judy Shen,
Nhat Le,
Jamie Prezioso,
John Abel,
Isaac Finberg,
Daniel Borders,
Raymond Biju,
Michael Nercessian,
Vaed Prasad,
Joseph Lee,
Spencer Wyman,
Sid Gupta,
Abigail Emerson,
Bahar Rahsepar,
Darpan Sanghavi,
Ryan Leung,
Limin Yu,
Archit Khosla,
Amaro Taylor-Weiner
Abstract:
Nested pairwise frames is a method for relative benchmarking of cell or tissue digital pathology models against manual pathologist annotations on a set of sampled patches. At a high level, the method compares agreement between a candidate model and pathologist annotations with agreement among pathologists' annotations. This evaluation framework addresses fundamental issues of data size and annotat…
▽ More
Nested pairwise frames is a method for relative benchmarking of cell or tissue digital pathology models against manual pathologist annotations on a set of sampled patches. At a high level, the method compares agreement between a candidate model and pathologist annotations with agreement among pathologists' annotations. This evaluation framework addresses fundamental issues of data size and annotator variability in using manual pathologist annotations as a source of ground truth for model validation. We implemented nested pairwise frames evaluation for tissue classification, cell classification, and cell count prediction tasks and show results for cell and tissue models deployed on an H&E-stained melanoma dataset.
△ Less
Submitted 7 June, 2023;
originally announced June 2023.
-
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images
Authors:
Faisal Mahmood,
Daniel Borders,
Richard Chen,
Gregory N. McKay,
Kevan J. Salimian,
Alexander Baras,
Nicholas J. Durr
Abstract:
Nuclei segmentation is a fundamental task that is critical for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Conventional vision-based methods for nuclei segmentation struggle in challenging cases and deep learning approaches have proven to be more robust and generalizable. However, CNNs require large amounts of lab…
▽ More
Nuclei segmentation is a fundamental task that is critical for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Conventional vision-based methods for nuclei segmentation struggle in challenging cases and deep learning approaches have proven to be more robust and generalizable. However, CNNs require large amounts of labeled histopathology data. Moreover, conventional CNN-based approaches lack structured prediction capabilities which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher order consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.
△ Less
Submitted 19 October, 2018; v1 submitted 29 September, 2018;
originally announced October 2018.