Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders... more Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders result in excess growth of trabecular bone and collagen fibers that replace hematopoietic cells, resulting in abnormal bone marrow function. Treatments using imatinib and JAK2 pathway inhibitors can be effective on osteosclerosis and fibrosis, therefore accurate grading is critical for tracking treatment effectiveness. Current grading standards use a four-class system based on analysis of biopsies stained with three histological stains: hematoxylin and eosin (H&E), Masson’s trichrome, and reticulin. However, conventional grading can be subjective and imprecise, impacting the effectiveness of treatment. In this paper, we demonstrate that mid-infrared spectroscopic imaging may serve as a quantitative diagnostic tool for quantitatively tracking disease progression and response to treatment. The proposed approach is label-free and provides automated quantitative analysis of osteosclerosis a...
Infrared spectroscopy combined with deep learning provide an automated and quantitative alternati... more Infrared spectroscopy combined with deep learning provide an automated and quantitative alternative to traditional histological examination.
Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosi... more Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation ...
Infrared (IR) spectroscopic microscopes provide the potential for label-free quantitative molecul... more Infrared (IR) spectroscopic microscopes provide the potential for label-free quantitative molecular imaging of biological samples, which can be used to aid in histology, forensics, and pharmaceutical analysis. Most IR imaging systems use broadband illumination combined with a spectrometer to separate the signal into spectral components. This technique is currently too slow for many biomedical applications such as clinical diagnosis, primarily due to the availability of bright mid-infrared sources and sensitive MCT detectors. There has been a recent push to increase throughput using coherent light sources, such as synchrotron radiation and quantum cascade lasers. While these sources provide a significant increase in intensity, the coherence introduces fringing artifacts in the final image. We demonstrate that applying time-delayed integration in one dimension can dramatically reduce fringing artifacts with minimal alterations to the standard infrared imaging pipeline. The proposed te...
There has recently been significant interest within the vibrational spectroscopy community to app... more There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit d...
There has recently been significant interest within the vibrational spectroscopy community to app... more There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit d...
Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and de... more Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and densely packed. This becomes particularly challenging for extremely large images, since manual intervention and processing time can make segmentation intractable. In this paper, we present an efficient and highly parallel formulation for symmetric three-dimensional (3D) contour evolution that extends previous work on fast two-dimensional active contours. We provide a formulation for optimization on 3D images, as well as a strategy for accelerating computation on consumer graphics hardware. The proposed software takes advantage of Monte-Carlo sampling schemes in order to speed up convergence and reduce thread divergence. Experimental results show that this method provides superior performance for large 2D and 3D cell segmentation tasks when compared to existing methods on large 3D brain images.
Fourier transform infrared (FTIR) spectroscopy enables label-free molecular identification and qu... more Fourier transform infrared (FTIR) spectroscopy enables label-free molecular identification and quantification of biological specimens. The resolution of diffraction limited FTIR imaging is poor due to the long optical wavelengths (2.5{\mu}m to 12.5{\mu}m)used and this is particularly limiting in biomedical imaging. Photothermal imaging overcomes this diffraction limit by using a multimodal pump/probe approach. However, these measurements require approximately 1 s per spectrum, making them impractical for large samples. This paper introduces an adaptive compressive sampling technique to dramatically reduce hyperspectral data acquisition time by utilizing both spectral and spatial sparsity. This method identifies the most informative spatial and spectral features and integrates a fast tensor completion algorithm to reconstruct megapixel-scale images and demonstrates speed advantages over FTIR imaging
Although image restoration methods based on spectral filtering techniques are very efficient, the... more Although image restoration methods based on spectral filtering techniques are very efficient, they can be applied only to problems with fairly simple spatially invariant blurring operators. Iterative methods, however, are much more flexible; they can be very efficient for spatially invariant as well as spatially variant blurs, they can incorporate a variety of regularization techniques and boundary conditions, and they can more easily incorporate additional constraints, such as nonnegativity. This chapter describes a variety of iterative methods used in image restoration, with a particular emphasis on efficiency, convergence behavior, and implementation. Discussion
2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2018
Histological stains, such as hemotaxylin and eosin (H&E), are commonly used to label tissue in cl... more Histological stains, such as hemotaxylin and eosin (H&E), are commonly used to label tissue in clinical biopsies. However, these labels modify the tissue chemistry, making it difficult to use for further downstream analysis. Fourier transform infrared spectroscopy (FTIR) has shown promising results for characterizing disease-relevant tissues without chemical labels or dyes. However, tissue classification requires human annotation, which is difficult and tedious to acquire for complex samples. In addition, the results of a molecular analysis must be presented in a way that facilitates diagnosis for a trained pathologist.One proposed approach is digital staining, which uses machine learning to map an infrared spectroscopic image to the image that would be ideally produced with a chemical stain. While these methods produce promising results, the resolution is significantly lower than traditional histology. We demonstrate that high-resolution mappings can be obtained using FTIR imaging ...
Deep neural networks have emerged as a set of robust machine learning tools for computer vision. ... more Deep neural networks have emerged as a set of robust machine learning tools for computer vision. The suitability of convolutional and recurrent neural networks, along with their variants, is well documented for color image analysis. However, remote sensing and biomedical imaging often rely on hyperspectral images containing more than three channels for pixel-level characterization. Deep learning can facilitate image analysis in multi-channel images; however, network architecture and design choices must be tailored to the unique characteristics of this data. In this two-part series, we review convolution and recurrent neural networks as applied to hyperspectral imagery. Part I focuses on the algorithms and techniques, while Part II focuses on application-specific design choices and real-world remote sensing and biomedical test cases. These chapters also survey recent advances and future directions for deep learning with hyperspectral images.
Deep neural networks are emerging as a popular choice for hyperspectral image analysis—compared w... more Deep neural networks are emerging as a popular choice for hyperspectral image analysis—compared with other machine learning approaches, they are more effective for a variety of applications in hyperspectral imaging. Part I (Chap. 3) introduces the fundamentals of deep learning algorithms and techniques deployed with hyperspectral images. In this chapter (Part II), we focus on application-specific nuances and design choices with respect to deploying such networks for robust analysis of hyperspectral images. We provide quantitative and qualitative results with a variety of deep learning architectures, and compare their performance to baseline state-of-the-art methods for both remote sensing and biomedical image analysis tasks. In addition to surveying recent developments in these areas, our goal in these two chapters is to provide guidance on how to utilize such algorithms for multichannel optical imagery. With that goal, we also provide code and example datasets used in this chapter.
Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders... more Osteosclerosis and myefibrosis are complications of myeloproliferative neoplasms. These disorders result in excess growth of trabecular bone and collagen fibers that replace hematopoietic cells, resulting in abnormal bone marrow function. Treatments using imatinib and JAK2 pathway inhibitors can be effective on osteosclerosis and fibrosis, therefore accurate grading is critical for tracking treatment effectiveness. Current grading standards use a four-class system based on analysis of biopsies stained with three histological stains: hematoxylin and eosin (H&E), Masson’s trichrome, and reticulin. However, conventional grading can be subjective and imprecise, impacting the effectiveness of treatment. In this paper, we demonstrate that mid-infrared spectroscopic imaging may serve as a quantitative diagnostic tool for quantitatively tracking disease progression and response to treatment. The proposed approach is label-free and provides automated quantitative analysis of osteosclerosis a...
Infrared spectroscopy combined with deep learning provide an automated and quantitative alternati... more Infrared spectroscopy combined with deep learning provide an automated and quantitative alternative to traditional histological examination.
Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosi... more Histological stains, such as hematoxylin and eosin (H&E), are routinely used in clinical diagnosis and research. While these labels offer a high degree of specificity, throughput is limited by the need for multiple samples. Traditional histology stains, such as immunohistochemical labels, also rely only on protein expression and cannot quantify small molecules and metabolites that may aid in diagnosis. Finally, chemical stains and dyes permanently alter the tissue, making downstream analysis impossible. Fourier transform infrared (FT-IR) spectroscopic imaging has shown promise for label-free characterization of important tissue phenotypes and can bypass the need for many chemical labels. Fourier transform infrared classification commonly leverages supervised learning, requiring human annotation that is tedious and prone to errors. One alternative is digital staining, which leverages machine learning to map IR spectra to a corresponding chemical stain. This replaces human annotation ...
Infrared (IR) spectroscopic microscopes provide the potential for label-free quantitative molecul... more Infrared (IR) spectroscopic microscopes provide the potential for label-free quantitative molecular imaging of biological samples, which can be used to aid in histology, forensics, and pharmaceutical analysis. Most IR imaging systems use broadband illumination combined with a spectrometer to separate the signal into spectral components. This technique is currently too slow for many biomedical applications such as clinical diagnosis, primarily due to the availability of bright mid-infrared sources and sensitive MCT detectors. There has been a recent push to increase throughput using coherent light sources, such as synchrotron radiation and quantum cascade lasers. While these sources provide a significant increase in intensity, the coherence introduces fringing artifacts in the final image. We demonstrate that applying time-delayed integration in one dimension can dramatically reduce fringing artifacts with minimal alterations to the standard infrared imaging pipeline. The proposed te...
There has recently been significant interest within the vibrational spectroscopy community to app... more There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit d...
There has recently been significant interest within the vibrational spectroscopy community to app... more There has recently been significant interest within the vibrational spectroscopy community to apply quantitative spectroscopic imaging techniques to histology and clinical diagnosis. However, many of the proposed methods require collecting spectroscopic images that have a similar region size and resolution to the corresponding histological images. Since spectroscopic images contain significantly more spectral samples than traditional histology, the resulting data sets can approach hundreds of gigabytes to terabytes in size. This makes them difficult to store and process, and the tools available to researchers for handling large spectroscopic data sets are limited. Fundamental mathematical tools, such as MATLAB, Octave, and SciPy, are extremely powerful but require that the data be stored in fast memory. This memory limitation becomes impractical for even modestly sized histological images, which can be hundreds of gigabytes in size. In this paper, we propose an open-source toolkit d...
Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and de... more Cell segmentation in microscopy is a challenging problem, since cells are often asymmetric and densely packed. This becomes particularly challenging for extremely large images, since manual intervention and processing time can make segmentation intractable. In this paper, we present an efficient and highly parallel formulation for symmetric three-dimensional (3D) contour evolution that extends previous work on fast two-dimensional active contours. We provide a formulation for optimization on 3D images, as well as a strategy for accelerating computation on consumer graphics hardware. The proposed software takes advantage of Monte-Carlo sampling schemes in order to speed up convergence and reduce thread divergence. Experimental results show that this method provides superior performance for large 2D and 3D cell segmentation tasks when compared to existing methods on large 3D brain images.
Fourier transform infrared (FTIR) spectroscopy enables label-free molecular identification and qu... more Fourier transform infrared (FTIR) spectroscopy enables label-free molecular identification and quantification of biological specimens. The resolution of diffraction limited FTIR imaging is poor due to the long optical wavelengths (2.5{\mu}m to 12.5{\mu}m)used and this is particularly limiting in biomedical imaging. Photothermal imaging overcomes this diffraction limit by using a multimodal pump/probe approach. However, these measurements require approximately 1 s per spectrum, making them impractical for large samples. This paper introduces an adaptive compressive sampling technique to dramatically reduce hyperspectral data acquisition time by utilizing both spectral and spatial sparsity. This method identifies the most informative spatial and spectral features and integrates a fast tensor completion algorithm to reconstruct megapixel-scale images and demonstrates speed advantages over FTIR imaging
Although image restoration methods based on spectral filtering techniques are very efficient, the... more Although image restoration methods based on spectral filtering techniques are very efficient, they can be applied only to problems with fairly simple spatially invariant blurring operators. Iterative methods, however, are much more flexible; they can be very efficient for spatially invariant as well as spatially variant blurs, they can incorporate a variety of regularization techniques and boundary conditions, and they can more easily incorporate additional constraints, such as nonnegativity. This chapter describes a variety of iterative methods used in image restoration, with a particular emphasis on efficiency, convergence behavior, and implementation. Discussion
2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 2018
Histological stains, such as hemotaxylin and eosin (H&E), are commonly used to label tissue in cl... more Histological stains, such as hemotaxylin and eosin (H&E), are commonly used to label tissue in clinical biopsies. However, these labels modify the tissue chemistry, making it difficult to use for further downstream analysis. Fourier transform infrared spectroscopy (FTIR) has shown promising results for characterizing disease-relevant tissues without chemical labels or dyes. However, tissue classification requires human annotation, which is difficult and tedious to acquire for complex samples. In addition, the results of a molecular analysis must be presented in a way that facilitates diagnosis for a trained pathologist.One proposed approach is digital staining, which uses machine learning to map an infrared spectroscopic image to the image that would be ideally produced with a chemical stain. While these methods produce promising results, the resolution is significantly lower than traditional histology. We demonstrate that high-resolution mappings can be obtained using FTIR imaging ...
Deep neural networks have emerged as a set of robust machine learning tools for computer vision. ... more Deep neural networks have emerged as a set of robust machine learning tools for computer vision. The suitability of convolutional and recurrent neural networks, along with their variants, is well documented for color image analysis. However, remote sensing and biomedical imaging often rely on hyperspectral images containing more than three channels for pixel-level characterization. Deep learning can facilitate image analysis in multi-channel images; however, network architecture and design choices must be tailored to the unique characteristics of this data. In this two-part series, we review convolution and recurrent neural networks as applied to hyperspectral imagery. Part I focuses on the algorithms and techniques, while Part II focuses on application-specific design choices and real-world remote sensing and biomedical test cases. These chapters also survey recent advances and future directions for deep learning with hyperspectral images.
Deep neural networks are emerging as a popular choice for hyperspectral image analysis—compared w... more Deep neural networks are emerging as a popular choice for hyperspectral image analysis—compared with other machine learning approaches, they are more effective for a variety of applications in hyperspectral imaging. Part I (Chap. 3) introduces the fundamentals of deep learning algorithms and techniques deployed with hyperspectral images. In this chapter (Part II), we focus on application-specific nuances and design choices with respect to deploying such networks for robust analysis of hyperspectral images. We provide quantitative and qualitative results with a variety of deep learning architectures, and compare their performance to baseline state-of-the-art methods for both remote sensing and biomedical image analysis tasks. In addition to surveying recent developments in these areas, our goal in these two chapters is to provide guidance on how to utilize such algorithms for multichannel optical imagery. With that goal, we also provide code and example datasets used in this chapter.
Uploads
Papers by Sebastian Berisha