-
Synthetic Privileged Information Enhances Medical Image Representation Learning
Authors:
Lucas Farndale,
Chris Walsh,
Robert Insall,
Ke Yuan
Abstract:
Multimodal self-supervised representation learning has consistently proven to be a highly effective method in medical image analysis, offering strong task performance and producing biologically informed insights. However, these methods heavily rely on large, paired datasets, which is prohibitive for their use in scenarios where paired data does not exist, or there is only a small amount available.…
▽ More
Multimodal self-supervised representation learning has consistently proven to be a highly effective method in medical image analysis, offering strong task performance and producing biologically informed insights. However, these methods heavily rely on large, paired datasets, which is prohibitive for their use in scenarios where paired data does not exist, or there is only a small amount available. In contrast, image generation methods can work well on very small datasets, and can find mappings between unpaired datasets, meaning an effectively unlimited amount of paired synthetic data can be generated. In this work, we demonstrate that representation learning can be significantly improved by synthetically generating paired information, both compared to training on either single-modality (up to 4.4x error reduction) or authentic multi-modal paired datasets (up to 5.6x error reduction).
△ Less
Submitted 8 March, 2024;
originally announced March 2024.
-
Ensuring accurate stain reproduction in deep generative networks for virtual immunohistochemistry
Authors:
Christopher D. Walsh,
Joanne Edwards,
Robert H. Insall
Abstract:
Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have sh…
▽ More
Immunohistochemistry is a valuable diagnostic tool for cancer pathology. However, it requires specialist labs and equipment, is time-intensive, and is difficult to reproduce. Consequently, a long term aim is to provide a digital method of recreating physical immunohistochemical stains. Generative Adversarial Networks have become exceedingly advanced at mapping one image type to another and have shown promise at inferring immunostains from haematoxylin and eosin. However, they have a substantial weakness when used with pathology images as they can fabricate structures that are not present in the original data. CycleGANs can mitigate invented tissue structures in pathology image mapping but have a related disposition to generate areas of inaccurate staining. In this paper, we describe a modification to the loss function of a CycleGAN to improve its mapping ability for pathology images by enforcing realistic stain replication while retaining tissue structure. Our approach improves upon others by considering structure and staining during model training. We evaluated our network using the Fréchet Inception distance, coupled with a new technique that we propose to appraise the accuracy of virtual immunohistochemistry. This assesses the overlap between each stain component in the inferred and ground truth images through colour deconvolution, thresholding and the Sorensen-Dice coefficient. Our modified loss function resulted in a Dice coefficient for the virtual stain of 0.78 compared with the real AE1/AE3 slide. This was superior to the unaltered CycleGAN's score of 0.74. Additionally, our loss function improved the Fréchet Inception distance for the reconstruction to 74.54 from 76.47. We, therefore, describe an advance in virtual restaining that can extend to other immunostains and tumour types and deliver reproducible, fast and readily accessible immunohistochemistry worldwide.
△ Less
Submitted 14 April, 2022;
originally announced April 2022.
-
Real-time shape approximation and 5-D fingerprinting of single proteins
Authors:
Erik C. Yusko,
Brandon R. Bruhn,
Olivia Eggenberger,
Jared Houghtaling,
Ryan C. Rollings,
Nathan C. Walsh,
Santoshi Nandivada,
Mariya Pindrus,
Adam R. Hall,
David Sept,
Jiali Li,
Devendra S. Kalonia,
Michael Mayer
Abstract:
This work exploits the zeptoliter sensing volume of electrolyte-filled nanopores to determine, simultaneously and in real time, the approximate shape, volume, charge, rotational diffusion coefficient, and dipole moment of individual proteins. We have developed the theory for a quantitative understanding and analysis of modulations in ionic current that arise from rotational dynamics of single prot…
▽ More
This work exploits the zeptoliter sensing volume of electrolyte-filled nanopores to determine, simultaneously and in real time, the approximate shape, volume, charge, rotational diffusion coefficient, and dipole moment of individual proteins. We have developed the theory for a quantitative understanding and analysis of modulations in ionic current that arise from rotational dynamics of single proteins as they move through the electric field inside a nanopore. The resulting multi-parametric information raises the possibility to characterize, identify, and quantify individual proteins and protein complexes in a mixture. This approach interrogates single proteins in solution and determines parameters such as the approximate shape and dipole moment, which are excellent protein descriptors and cannot be obtained otherwise from single protein molecules in solution. Taken together, this five-dimensional characterization of biomolecules at the single particle level has the potential for instantaneous protein identification, quantification, and possibly sorting with implications for structural biology, proteomics, biomarker detection, and routine protein analysis.
△ Less
Submitted 29 August, 2015;
originally announced October 2015.
