We seek the development and evaluation of a fast, accurate, and consistent method for general-pur... more We seek the development and evaluation of a fast, accurate, and consistent method for general-purpose segmentation, based on interactive machine learning (IML). To validate our method, we identified retrospective cohorts of 20 brain, 50 breast, and 50 lung cancer patients, as well as 20 spleen scans, with corresponding ground truth annotations. Utilizing very brief user training annotations and the adaptive geodesic distance transform, an ensemble of SVMs is trained, providing a patient-specific model applied to the whole image. Two experts segmented each cohort twice with our method and twice manually. The IML method was faster than manual annotation by 53.1% on average. We found significant (p < 0.001) overlap difference for spleen (DiceIML/DiceManual = 0.91/0.87), breast tumors (DiceIML/DiceManual = 0.84/0.82), and lung nodules (DiceIML/DiceManual = 0.78/0.83). For intra-rater consistency, a significant (p = 0.003) difference was found for spleen (DiceIML/DiceManual = 0.91/0.8...
Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported resu... more Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported results from either private institutional data or publicly available datasets. However, current public datasets are limited in terms of: a) number of subjects, b) lack of consistent acquisition protocol, c) data quality, or d) accompanying clinical, demographic, and molecular information. Toward alleviating these limitations, we contribute the “University of Pennsylvania Glioblastoma Imaging, Genomics, and Radiomics” (UPenn-GBM) dataset, which describes the currently largest publicly available comprehensive collection of 630 patients diagnosed withde novoglioblastoma. The UPenn-GBM dataset includes (a) advanced multi-parametric magnetic resonance imaging scans acquired during routine clinical practice, at the University of Pennsylvania Health System, (b) accompanying clinical, demographic, and molecular information, (d) perfusion and diffusion derivative volumes, (e) computationally-derived ...
Background Gliomas represent a biologically heterogeneous group of primary brain tumors with unco... more Background Gliomas represent a biologically heterogeneous group of primary brain tumors with uncontrolled cellular proliferation and diffuse infiltration that renders them almost incurable, thereby leading to a grim prognosis. Recent comprehensive genomic profiling has greatly elucidated the molecular hallmarks of gliomas, including the mutations in isocitrate dehydrogenase 1 and 2 (IDH1 and IDH2), loss of chromosomes 1p and 19q (1p/19q), and epidermal growth factor receptor variant III (EGFRvIII). Detection of these molecular alterations is based on ex vivo analysis of surgically resected tissue specimen that sometimes is not adequate for testing and/or does not capture the spatial tumor heterogeneity of the neoplasm. Methods We developed a method for noninvasive detection of radiogenomic markers of IDH both in lower-grade gliomas (WHO grade II and III tumors) and glioblastoma (WHO grade IV), 1p/19q in IDH-mutant lower-grade gliomas, and EGFRvIII in glioblastoma. Preoperative MRIs ...
The management of prostate cancer (PCa) is dependent on biomarkers of biological aggression. This... more The management of prostate cancer (PCa) is dependent on biomarkers of biological aggression. This includes an invasive biopsy to facilitate a histopathological assessment of the tumor’s grade. This review explores the technical processes of applying magnetic resonance imaging based radiomic models to the evaluation of PCa. By exploring how a deep radiomics approach further optimizes the prediction of a PCa’s grade group, it will be clear how this integration of artificial intelligence mitigates existing major technological challenges faced by a traditional radiomic model: image acquisition, small data sets, image processing, labeling/segmentation, informative features, predicting molecular features and incorporating predictive models. Other potential impacts of artificial intelligence on the personalized treatment of PCa will also be discussed. The role of deep radiomics analysis-a deep texture analysis, which extracts features from convolutional neural networks layers, will be high...
Cancer pathology reflects disease progression (or regression) and associated molecular characteri... more Cancer pathology reflects disease progression (or regression) and associated molecular characteristics, and provides rich phenotypic information that is predictive of cancer grade and has potential implications in treatment planning and prognosis. According to the remarkable performance of computational approaches in the digital pathology domain, we hypothesized that machine learning can help to distinguish low-grade gliomas (LGG) from high-grade gliomas (HGG) by exploiting the rich phenotypic information that reflects the microvascular proliferation level, mitotic activity, presence of necrosis, and nuclear atypia present in digital pathology images. A set of 735 whole-slide digital pathology images of glioma patients (median age: 49.65 years, male: 427, female: 308, median survival: 761.26 days) were obtained from TCGA. Sub-images that contained a viable tumor area, showing sufficient histologic characteristics, and that did not have any staining artifact were extracted. Several c...
White matter lesions (WML) are common in a variety of brain pathologies, including ischemia affec... more White matter lesions (WML) are common in a variety of brain pathologies, including ischemia affecting blood vessels deeper inside the brain’s white matter, and show an abnormal signal in T1-weighted and FLAIR images. The emergence of personalized medicine requires quantification and analysis of differential characteristics of WML across different brain regions. Manual segmentation and analysis of WMLs is laborious and time-consuming; therefore, automated methods providing robust, reproducible, and fast WML segmentation and analysis are highly desirable. In this study, we tackled the segmentation problem as a voxel-based classification problem. We developed an ensemble of different classification models, including six models of support vector machine, trained on handcrafted and transfer learning features, and five models of Residual neural network, trained on varying window sizes. The output of these models was combined through majority-voting. A series of image processing operations...
BACKGROUND Microscopic features of brain tumors, such as tumor cell morphology, type/degree of mi... more BACKGROUND Microscopic features of brain tumors, such as tumor cell morphology, type/degree of microvascular hyperplasia, mitotic activity, and extent of zonal/geographic necrosis, among many others, are measurable and reflect underlying molecular markers that are predictive of patient prognosis, signifying that quantitative analysis may provide insight into disease mechanics. We developed a computational method to predict overall-survival and molecular markers for brain tumors using deep learning on whole-slide digital images (WSDI). METHODS The WSDI were acquired from TCGA for 663 patients [IDH:333 wildtype, 330 mutants, 1p/19q:201 non-codeleted, 129 codeleted]. A set of 100 region-of-interest each comprising 1024x1024 that contained viable tumor with descriptive histologic characteristics and that were free of artifacts were extracted. A modified version of ResNet with architecture of 50 convolutional layers was used. The network was optimized using stochastic gradient decent opt...
Distinguishing benign from malignant disease is a primary challenge for colon histopathologists. ... more Distinguishing benign from malignant disease is a primary challenge for colon histopathologists. Current clinical methods rely on qualitative visual analysis of features such as glandular architecture and size that exist on a continuum from benign to malignant. Consequently, discordance between histopathologists is common. To provide more reliable analysis of colon specimens, we propose an end-to-end computational pathology pipeline that encompasses gland segmentation, cancer detection, and then further breaking down the malignant samples into different cancer grades. We propose a multi-step gland segmentation method, which models tissue components as ellipsoids. For cancer detection/grading, we encode cellular morphology, spatial architectural patterns of glands, and texture by extracting multi-scale features: (i) Gland-based: extracted from individual glands, (ii) local-patch-based: computed from randomly-selected image patches, and (iii) image-based: extracted from images, and em...
The remarkable heterogeneity of glioblastoma, across patients and over time, is one of the main c... more The remarkable heterogeneity of glioblastoma, across patients and over time, is one of the main challenges in precision diagnostics and treatment planning. Non-invasive in vivo characterization of this heterogeneity using imaging could assist in understanding disease subtypes, as well as in risk-stratification and treatment planning of glioblastoma. The current study leveraged advanced imaging analytics and radiomic approaches applied to multi-parametric MRI of de novo glioblastoma patients (n = 208 discovery, n = 53 replication), and discovered three distinct and reproducible imaging subtypes of glioblastoma, with differential clinical outcome and underlying molecular characteristics, including isocitrate dehydrogenase-1 (IDH1), O-methylguanine-DNA methyltransferase, epidermal growth factor receptor variant III (EGFRvIII), and transcriptomic subtype composition. The subtypes provided risk-stratification substantially beyond that provided by WHO classifications. Within IDH1-wildtype...
Journal of medical imaging (Bellingham, Wash.), 2018
Standard surgical resection of glioblastoma, mainly guided by the enhancement on postcontrast T1-... more Standard surgical resection of glioblastoma, mainly guided by the enhancement on postcontrast T1-weighted magnetic resonance imaging (MRI), disregards infiltrating tumor within the peritumoral edema region (ED). Subsequent radiotherapy typically delivers uniform radiation to peritumoral FLAIR-hyperintense regions, without attempting to target areas likely to be infiltrated more heavily. Noninvasivedelineation of the areas of tumor infiltration and prediction of early recurrence in peritumoral ED could assist in targeted intensification of local therapies, thereby potentially delaying recurrence and prolonging survival. This paper presents a method for estimating peritumoral edema infiltration using radiomic signatures determined via machine learning methods, and tests it on 90 patients withglioblastoma. The generalizability of the proposed predictive model was evaluated via cross-validation in a discovery cohort ([Formula: see text]) and was subsequently evaluated in a replication c...
Journal of medical imaging (Bellingham, Wash.), 2018
The growth of multiparametric imaging protocols has paved the way for quantitative imaging phenot... more The growth of multiparametric imaging protocols has paved the way for quantitative imaging phenotypes that predict treatment response and clinical outcome, reflect underlying cancer molecular characteristics and spatiotemporal heterogeneity, and can guide personalized treatment planning. This growth has underlined the need for efficient quantitative analytics to derive high-dimensional imaging signatures of diagnostic and predictive value in this emerging era of integrated precision diagnostics. This paper presents cancer imaging phenomics toolkit (CaPTk), a new and dynamically growing software platform for analysis of radiographic images of cancer, currently focusing on brain, breast, and lung cancer. CaPTk leverages the value of quantitative imaging analytics along with machine learning to derive phenotypic imaging signatures, based on two-level functionality. First, image analysis algorithms are used to extract comprehensive panels of diverse and complementary features, such as m...
Brainlesion : glioma, multiple sclerosis, stroke and traumatic brain injuries : second International Workshop, BrainLes 2016, with the challenges on BRATS, ISLES and mTOP 2016, held in conjunction with MICCAI 2016, Athens, Greece, Octob..., 2016
We present an approach for segmenting both low- and high-grade gliomas in multimodal magnetic res... more We present an approach for segmenting both low- and high-grade gliomas in multimodal magnetic resonance imaging volumes. The proposed framework is an extension of our previous work [6,7], with an additional component for segmenting post-operative scans. The proposed approach is based on a hybrid generative-discriminative model. Firstly, a generative model based on a joint segmentation-registration framework is used to segment the brain scans into cancerous and healthy tissues. Secondly, a gradient boosting classification scheme is used to refine tumor segmentation based on information from multiple patients. We evaluated our approach in 218 cases during the training phase of the BRAin Tumor Segmentation (BRATS) 2016 challenge and report promising results. During the testing phase, the proposed approach was ranked among the top performing methods, after being additionally evaluated in 191 unseen cases.
We present an approach for segmenting low- and high-grade gliomas in multimodal magnetic resonanc... more We present an approach for segmenting low- and high-grade gliomas in multimodal magnetic resonance imaging volumes. The proposed approach is based on a hybrid generative-discriminative model. Firstly, a generative approach based on an Expectation-Maximization framework that incorporates a glioma growth model is used to segment the brain scans into tumor, as well as healthy tissue labels. Secondly, a gradient boosting multi-class classification scheme is used to refine tumor labels based on information from multiple patients. Lastly, a probabilistic Bayesian strategy is employed to further refine and finalize the tumor segmentation based on patient-specific intensity statistics from the multiple modalities. We evaluated our approach in 186 cases during the training phase of the BRAin Tumor Segmentation (BRATS) 2015 challenge and report promising results. During the testing phase, the algorithm was additionally evaluated in 53 unseen cases, achieving the best performance among the competing methods.
We seek the development and evaluation of a fast, accurate, and consistent method for general-pur... more We seek the development and evaluation of a fast, accurate, and consistent method for general-purpose segmentation, based on interactive machine learning (IML). To validate our method, we identified retrospective cohorts of 20 brain, 50 breast, and 50 lung cancer patients, as well as 20 spleen scans, with corresponding ground truth annotations. Utilizing very brief user training annotations and the adaptive geodesic distance transform, an ensemble of SVMs is trained, providing a patient-specific model applied to the whole image. Two experts segmented each cohort twice with our method and twice manually. The IML method was faster than manual annotation by 53.1% on average. We found significant (p < 0.001) overlap difference for spleen (DiceIML/DiceManual = 0.91/0.87), breast tumors (DiceIML/DiceManual = 0.84/0.82), and lung nodules (DiceIML/DiceManual = 0.78/0.83). For intra-rater consistency, a significant (p = 0.003) difference was found for spleen (DiceIML/DiceManual = 0.91/0.8...
Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported resu... more Glioblastoma is the most common aggressive adult brain tumor. Numerous studies have reported results from either private institutional data or publicly available datasets. However, current public datasets are limited in terms of: a) number of subjects, b) lack of consistent acquisition protocol, c) data quality, or d) accompanying clinical, demographic, and molecular information. Toward alleviating these limitations, we contribute the “University of Pennsylvania Glioblastoma Imaging, Genomics, and Radiomics” (UPenn-GBM) dataset, which describes the currently largest publicly available comprehensive collection of 630 patients diagnosed withde novoglioblastoma. The UPenn-GBM dataset includes (a) advanced multi-parametric magnetic resonance imaging scans acquired during routine clinical practice, at the University of Pennsylvania Health System, (b) accompanying clinical, demographic, and molecular information, (d) perfusion and diffusion derivative volumes, (e) computationally-derived ...
Background Gliomas represent a biologically heterogeneous group of primary brain tumors with unco... more Background Gliomas represent a biologically heterogeneous group of primary brain tumors with uncontrolled cellular proliferation and diffuse infiltration that renders them almost incurable, thereby leading to a grim prognosis. Recent comprehensive genomic profiling has greatly elucidated the molecular hallmarks of gliomas, including the mutations in isocitrate dehydrogenase 1 and 2 (IDH1 and IDH2), loss of chromosomes 1p and 19q (1p/19q), and epidermal growth factor receptor variant III (EGFRvIII). Detection of these molecular alterations is based on ex vivo analysis of surgically resected tissue specimen that sometimes is not adequate for testing and/or does not capture the spatial tumor heterogeneity of the neoplasm. Methods We developed a method for noninvasive detection of radiogenomic markers of IDH both in lower-grade gliomas (WHO grade II and III tumors) and glioblastoma (WHO grade IV), 1p/19q in IDH-mutant lower-grade gliomas, and EGFRvIII in glioblastoma. Preoperative MRIs ...
The management of prostate cancer (PCa) is dependent on biomarkers of biological aggression. This... more The management of prostate cancer (PCa) is dependent on biomarkers of biological aggression. This includes an invasive biopsy to facilitate a histopathological assessment of the tumor’s grade. This review explores the technical processes of applying magnetic resonance imaging based radiomic models to the evaluation of PCa. By exploring how a deep radiomics approach further optimizes the prediction of a PCa’s grade group, it will be clear how this integration of artificial intelligence mitigates existing major technological challenges faced by a traditional radiomic model: image acquisition, small data sets, image processing, labeling/segmentation, informative features, predicting molecular features and incorporating predictive models. Other potential impacts of artificial intelligence on the personalized treatment of PCa will also be discussed. The role of deep radiomics analysis-a deep texture analysis, which extracts features from convolutional neural networks layers, will be high...
Cancer pathology reflects disease progression (or regression) and associated molecular characteri... more Cancer pathology reflects disease progression (or regression) and associated molecular characteristics, and provides rich phenotypic information that is predictive of cancer grade and has potential implications in treatment planning and prognosis. According to the remarkable performance of computational approaches in the digital pathology domain, we hypothesized that machine learning can help to distinguish low-grade gliomas (LGG) from high-grade gliomas (HGG) by exploiting the rich phenotypic information that reflects the microvascular proliferation level, mitotic activity, presence of necrosis, and nuclear atypia present in digital pathology images. A set of 735 whole-slide digital pathology images of glioma patients (median age: 49.65 years, male: 427, female: 308, median survival: 761.26 days) were obtained from TCGA. Sub-images that contained a viable tumor area, showing sufficient histologic characteristics, and that did not have any staining artifact were extracted. Several c...
White matter lesions (WML) are common in a variety of brain pathologies, including ischemia affec... more White matter lesions (WML) are common in a variety of brain pathologies, including ischemia affecting blood vessels deeper inside the brain’s white matter, and show an abnormal signal in T1-weighted and FLAIR images. The emergence of personalized medicine requires quantification and analysis of differential characteristics of WML across different brain regions. Manual segmentation and analysis of WMLs is laborious and time-consuming; therefore, automated methods providing robust, reproducible, and fast WML segmentation and analysis are highly desirable. In this study, we tackled the segmentation problem as a voxel-based classification problem. We developed an ensemble of different classification models, including six models of support vector machine, trained on handcrafted and transfer learning features, and five models of Residual neural network, trained on varying window sizes. The output of these models was combined through majority-voting. A series of image processing operations...
BACKGROUND Microscopic features of brain tumors, such as tumor cell morphology, type/degree of mi... more BACKGROUND Microscopic features of brain tumors, such as tumor cell morphology, type/degree of microvascular hyperplasia, mitotic activity, and extent of zonal/geographic necrosis, among many others, are measurable and reflect underlying molecular markers that are predictive of patient prognosis, signifying that quantitative analysis may provide insight into disease mechanics. We developed a computational method to predict overall-survival and molecular markers for brain tumors using deep learning on whole-slide digital images (WSDI). METHODS The WSDI were acquired from TCGA for 663 patients [IDH:333 wildtype, 330 mutants, 1p/19q:201 non-codeleted, 129 codeleted]. A set of 100 region-of-interest each comprising 1024x1024 that contained viable tumor with descriptive histologic characteristics and that were free of artifacts were extracted. A modified version of ResNet with architecture of 50 convolutional layers was used. The network was optimized using stochastic gradient decent opt...
Distinguishing benign from malignant disease is a primary challenge for colon histopathologists. ... more Distinguishing benign from malignant disease is a primary challenge for colon histopathologists. Current clinical methods rely on qualitative visual analysis of features such as glandular architecture and size that exist on a continuum from benign to malignant. Consequently, discordance between histopathologists is common. To provide more reliable analysis of colon specimens, we propose an end-to-end computational pathology pipeline that encompasses gland segmentation, cancer detection, and then further breaking down the malignant samples into different cancer grades. We propose a multi-step gland segmentation method, which models tissue components as ellipsoids. For cancer detection/grading, we encode cellular morphology, spatial architectural patterns of glands, and texture by extracting multi-scale features: (i) Gland-based: extracted from individual glands, (ii) local-patch-based: computed from randomly-selected image patches, and (iii) image-based: extracted from images, and em...
The remarkable heterogeneity of glioblastoma, across patients and over time, is one of the main c... more The remarkable heterogeneity of glioblastoma, across patients and over time, is one of the main challenges in precision diagnostics and treatment planning. Non-invasive in vivo characterization of this heterogeneity using imaging could assist in understanding disease subtypes, as well as in risk-stratification and treatment planning of glioblastoma. The current study leveraged advanced imaging analytics and radiomic approaches applied to multi-parametric MRI of de novo glioblastoma patients (n = 208 discovery, n = 53 replication), and discovered three distinct and reproducible imaging subtypes of glioblastoma, with differential clinical outcome and underlying molecular characteristics, including isocitrate dehydrogenase-1 (IDH1), O-methylguanine-DNA methyltransferase, epidermal growth factor receptor variant III (EGFRvIII), and transcriptomic subtype composition. The subtypes provided risk-stratification substantially beyond that provided by WHO classifications. Within IDH1-wildtype...
Journal of medical imaging (Bellingham, Wash.), 2018
Standard surgical resection of glioblastoma, mainly guided by the enhancement on postcontrast T1-... more Standard surgical resection of glioblastoma, mainly guided by the enhancement on postcontrast T1-weighted magnetic resonance imaging (MRI), disregards infiltrating tumor within the peritumoral edema region (ED). Subsequent radiotherapy typically delivers uniform radiation to peritumoral FLAIR-hyperintense regions, without attempting to target areas likely to be infiltrated more heavily. Noninvasivedelineation of the areas of tumor infiltration and prediction of early recurrence in peritumoral ED could assist in targeted intensification of local therapies, thereby potentially delaying recurrence and prolonging survival. This paper presents a method for estimating peritumoral edema infiltration using radiomic signatures determined via machine learning methods, and tests it on 90 patients withglioblastoma. The generalizability of the proposed predictive model was evaluated via cross-validation in a discovery cohort ([Formula: see text]) and was subsequently evaluated in a replication c...
Journal of medical imaging (Bellingham, Wash.), 2018
The growth of multiparametric imaging protocols has paved the way for quantitative imaging phenot... more The growth of multiparametric imaging protocols has paved the way for quantitative imaging phenotypes that predict treatment response and clinical outcome, reflect underlying cancer molecular characteristics and spatiotemporal heterogeneity, and can guide personalized treatment planning. This growth has underlined the need for efficient quantitative analytics to derive high-dimensional imaging signatures of diagnostic and predictive value in this emerging era of integrated precision diagnostics. This paper presents cancer imaging phenomics toolkit (CaPTk), a new and dynamically growing software platform for analysis of radiographic images of cancer, currently focusing on brain, breast, and lung cancer. CaPTk leverages the value of quantitative imaging analytics along with machine learning to derive phenotypic imaging signatures, based on two-level functionality. First, image analysis algorithms are used to extract comprehensive panels of diverse and complementary features, such as m...
Brainlesion : glioma, multiple sclerosis, stroke and traumatic brain injuries : second International Workshop, BrainLes 2016, with the challenges on BRATS, ISLES and mTOP 2016, held in conjunction with MICCAI 2016, Athens, Greece, Octob..., 2016
We present an approach for segmenting both low- and high-grade gliomas in multimodal magnetic res... more We present an approach for segmenting both low- and high-grade gliomas in multimodal magnetic resonance imaging volumes. The proposed framework is an extension of our previous work [6,7], with an additional component for segmenting post-operative scans. The proposed approach is based on a hybrid generative-discriminative model. Firstly, a generative model based on a joint segmentation-registration framework is used to segment the brain scans into cancerous and healthy tissues. Secondly, a gradient boosting classification scheme is used to refine tumor segmentation based on information from multiple patients. We evaluated our approach in 218 cases during the training phase of the BRAin Tumor Segmentation (BRATS) 2016 challenge and report promising results. During the testing phase, the proposed approach was ranked among the top performing methods, after being additionally evaluated in 191 unseen cases.
We present an approach for segmenting low- and high-grade gliomas in multimodal magnetic resonanc... more We present an approach for segmenting low- and high-grade gliomas in multimodal magnetic resonance imaging volumes. The proposed approach is based on a hybrid generative-discriminative model. Firstly, a generative approach based on an Expectation-Maximization framework that incorporates a glioma growth model is used to segment the brain scans into tumor, as well as healthy tissue labels. Secondly, a gradient boosting multi-class classification scheme is used to refine tumor labels based on information from multiple patients. Lastly, a probabilistic Bayesian strategy is employed to further refine and finalize the tumor segmentation based on patient-specific intensity statistics from the multiple modalities. We evaluated our approach in 186 cases during the training phase of the BRAin Tumor Segmentation (BRATS) 2015 challenge and report promising results. During the testing phase, the algorithm was additionally evaluated in 53 unseen cases, achieving the best performance among the competing methods.
Uploads
Papers by Saima Rathore