Ravi Samala

Transfer learning in deep convolutional neural networks (DCNNs) is an important step in its application to medical imaging tasks. We propose a multi-task transfer learning DCNN with the aim of translating the 'knowledge' learned from non-medical images to medical diagnostic tasks through supervised training and increasing the generalization capabilities of DCNNs by simultaneously learning auxiliary tasks. We studied this approach in an important application: classification of malignant and benign breast masses. With Institutional Review Board (IRB) approval, digitized screen-film mammograms (SFMs) and digital mammograms (DMs) were collected from our patient files and additional SFMs were obtained from the Digital Database for Screening Mammography. The data set consisted of 2242 views with 2454 masses (1057 malignant, 1397 benign). In single-task transfer learning, the DCNN was trained and tested on SFMs. In multi-task transfer learning, SFMs and DMs were used to train the DCNN, which was then tested on SFMs. N-fold cross-validation with the training set was used for training and parameter optimization. On the independent test set, the multi-task transfer learning DCNN was found to have significantly (p = 0.007) higher performance compared to the single-task transfer learning DCNN. This study demonstrates that multi-task transfer learning may be an effective approach for training DCNN in medical imaging applications when training samples from a single modality are limited.

Purpose: The authors are developing a computerized system for bladder segmentation in CT urogra-phy (CTU) as a critical component for computer-aided detection of bladder cancer. Methods: A deep-learning convolutional neural network (DL-CNN) was trained to distinguish between the inside and the outside of the bladder using 160 000 regions of interest (ROI) from CTU images. The trained DL-CNN was used to estimate the likelihood of an ROI being inside the bladder for ROIs centered at each voxel in a CTU case, resulting in a likelihood map. Thresholding and hole-filling were applied to the map to generate the initial contour for the bladder, which was then refined by 3D and 2D level sets. The segmentation performance was evaluated using 173 cases: 81 cases in the training set (42 lesions, 21 wall thickenings, and 18 normal bladders) and 92 cases in the test set (43 lesions, 36 wall thickenings, and 13 normal bladders). The computerized segmentation accuracy using the DL likelihood map was compared to that using a likelihood map generated by Haar features and a random forest classifier, and that using our previous conjoint level set analysis and segmentation system (CLASS) without using a likelihood map. All methods were evaluated relative to the 3D hand-segmented reference contours. Results: With DL-CNN-based likelihood map and level sets, the average volume intersection ratio, average percent volume error, average absolute volume error, average minimum distance, and the Jaccard index for the test set were 81.9% ± 12.1%, 10.2% ± 16.2%, 14.0% ± 13.0%, 3.6 ± 2.0 mm, and 76.2% ± 11.8%, respectively. With the Haar-feature-based likelihood map and level sets, the corresponding values were 74.3% ± 12.7%, 13.0% ± 22.3%, 20.5% ± 15.7%, 5.7 ± 2.6 mm, and 66.7% ± 12.6%, respectively. With our previous CLASS with local contour refinement (LCR) method, the corresponding values were 78.0% ± 14.7%, 16.5% ± 16.8%, 18.2% ± 15.0%, 3.8 ± 2.3 mm, and 73.9% ± 13.5%, respectively. Conclusions: The authors demonstrated that the DL-CNN can overcome the strong boundary between two regions that have large difference in gray levels and provides a seamless mask to guide level set segmentation, which has been a problem for many gradient-based segmentation methods. Compared to our previous CLASS with LCR method, which required two user inputs to initialize the segmentation, DL-CNN with level sets achieved better segmentation performance while using a single user input. Compared to the Haar-feature-based likelihood map, the DL-CNN-based likelihood map could guide the level sets to achieve better segmentation. The results demonstrate the feasibility of our new approach of using DL-CNN in combination with level sets for segmentation of the bladder.

Purpose: Develop a computer-aided detection (CAD) system for masses in digital breast tomosyn-thesis (DBT) volume using a deep convolutional neural network (DCNN) with transfer learning from mammograms. Methods: A data set containing 2282 digitized film and digital mammograms and 324 DBT volumes were collected with IRB approval. The mass of interest on the images was marked by an experienced breast radiologist as reference standard. The data set was partitioned into a training set (2282 mammograms with 2461 masses and 230 DBT views with 228 masses) and an independent test set (94 DBT views with 89 masses). For DCNN training, the region of interest (ROI) containing the mass (true positive) was extracted from each image. False positive (FP) ROIs were identified at prescreening by their previously developed CAD systems. After data augmentation, a total of 45 072 mammographic ROIs and 37 450 DBT ROIs were obtained. Data normalization and reduction of non-uniformity in the ROIs across heterogeneous data was achieved using a background correction method applied to each ROI. A DCNN with four convolutional layers and three fully connected (FC) layers was first trained on the mammography data. Jittering and dropout techniques were used to reduce overfitting. After training with the mammographic ROIs, all weights in the first three convolutional layers were frozen, and only the last convolution layer and the FC layers were randomly initialized again and trained using the DBT training ROIs. The authors compared the performances of two CAD systems for mass detection in DBT: one used the DCNN-based approach and the other used their previously developed feature-based approach for FP reduction. The prescreening stage was identical in both systems, passing the same set of mass candidates to the FP reduction stage. For the feature-based CAD system, 3D clustering and active contour method was used for segmentation; morphological, gray level, and texture features were extracted and merged with a linear discriminant classifier to score the detected masses. For the DCNN-based CAD system, ROIs from five consecutive slices centered at each candidate were passed through the trained DCNN and a mass likelihood score was generated. The performances of the CAD systems were evaluated using free-response ROC curves and the performance difference was analyzed using a non-parametric method. Results: Before transfer learning, the DCNN trained only on mammograms with an AUC of 0.99 classified DBT masses with an AUC of 0.81 in the DBT training set. After transfer learning with DBT, the AUC improved to 0.90. For breast-based CAD detection in the test set, the sensitivity for the feature-based and the DCNN-based CAD systems was 83% and 91%, respectively, at 1 FP/DBT volume. The difference between the performances for the two systems was statistically significant (p-value < 0.05). Conclusions: The image patterns learned from the mammograms were transferred to the mass detection on DBT slices through the DCNN. This study demonstrated that large data sets collected from mammography are useful for developing new CAD systems for DBT, alleviating the problem and effort of collecting entirely new large data sets for the new modality. C 2016 American Association of Physicists in Medicine. [http://dx.doi.org/10.1118/1.4967345]

With IRB approval, digital breast tomosynthesis (DBT) images of human subjects were collected using a GE GEN2 DBT prototype system. Corresponding digital mammograms (DMs) of the same subjects were collected retrospectively from patient files. The data set contained a total of 237 views of DBT and equal number of DM views from 120 human subjects, each included 163 views with microcalcification clusters (MCs) and 74 views without MCs. The data set was separated into training and independent test sets. The pre-processing, object prescreening and segmentation, false positive reduction and clustering strategies for MC detection by three computer-aided detection (CADe) systems designed for DM, DBT, and a planar projection image generated from DBT were analyzed. Receiver operating characteristic (ROC) curves based on features extracted from microcalcifications and free-response ROC (FROC) curves based on scores from MCs were used to quantify the performance of the systems. Jackknife FROC (JAFROC) and non-parametric analysis methods were used to determine the statistical difference between the FROC curves. The difference between the CAD DM and CAD DBT systems when the false positive rate was estimated from cases without MCs did not reach statistical significance. The study indicates that the large search space in DBT may not be a limiting factor for CADe to achieve similar performance as that observed in DM.

ABSTRACT PURPOSE Both mass and microcalcifications (MCs) are important signs for breast cancer. Most regularization methods depend on local gradient and may treat the ill-defined margins or subtle spiculations of masses and subtle MCs as noise because of their small gradient. In this study, we developed a new bilateral-filter regularization method utilizing the multiscale structure (MBiF) of the image to enhance subtle edges and MCs in DBT. METHOD AND MATERIALS DBT data were first decomposed into frequency bands using the Laplacian Pyramid Decomposition. The mass margins and spiculations fall into the low-frequency (LF) bands while MCs, edges and noise fall into the high-frequency (HF) bands. Bilateral filtering with domain and range filters optimized for subtle signals was applied to the HF bands to exploit simultaneously the spatial and gray level information for enhancing the signals while suppressing noise. The image was reconstructed from the Laplacian pyramid after bilateral regularization in selected frequency bands. With IRB approval, DBT of subjects with MCs or masses was acquired with a GE prototype DBT system. DBT was reconstructed with SART regularized by MBiF and the total p-norm variation (TpV) method. The full width at half maximum (FWHM) of the gray-level profile of MCs or spiculations and the contrast-to-noise ratio (CNR) of MCs were used as sharpness and contrast measures to compare the two methods and without regularization (NR). RESULTS MBiF reduced spurious noise and contouring artifacts compared to TpV, thus preserving the image quality of the structured background. MBiF achieved 100-200% higher CNR for large MCs and 50-100% higher CNR for subtle MCs than NR, and 50-100% higher CNR than TpV for all MCs. The FWHMs of MCs were comparable among different methods. For spiculated masses, TpV blurred the spiculations and margins while MBiF preserved the edge sharpness and enhanced the mass margins. The FWHM of subtle spiculations by TpV was 25% larger than those by MBiF and NR. CONCLUSION The new MBiF method enhanced the CNR of MCs and preserved the sharpness of MCs and spiculated masses. MBiF provided better image quality of the structured background and was superior to TpV and NR for both MCs and masses. CLINICAL RELEVANCE/APPLICATION A regularized DBT reconstruction method that reduces noise and enhances the visibility of both MCs and masses may improve breast cancer detection and patient care without increasing dose.

ABSTRACT We are developing a CAD system to assist radiologists in detecting microcalcification clusters (MCs) in digital breast tomosynthesis (DBT). In this study, we investigated the feasibility of using as input to the CAD system an enhanced DBT volume that was reconstructed with the iterative simultaneous algebraic reconstruction technique (SART) regularized by a new multiscale bilateral filtering (MBiF) method. The MBiF method utilizes the multiscale structures of the breast to selectively enhance MCs and preserve mass spiculations while smoothing noise in the DBT images. The CAD system first extracted the enhancement-modulated calcification response (EMCR) in the DBT volume. Detection of the seed points for MCs and individual calcifications were guided by the EMCR. MC candidates were formed by dynamic clustering. FPs were further reduced by analysis of the feature characteristics of the MCs. With IRB approval, two-view DBT of 91 subjects with biopsy-proven MCs were collected. Seventy-eight views from 39 subjects with MCs were used for training and the remaining 52 cases were used for independent testing. For view-based detection, a sensitivity of 85% was achieved at 3.23 FPs/volume. For case-based detection, the same sensitivity was obtained at 1.63 FPs/volume. The results indicate that the new MBiF method is useful in improving the detection accuracy of clustered microcalcifications. An effective CAD system for microcalcification detection in DBT has the potential to eliminate the need for additional mammograms, thereby reducing patient dose and reading time.

ABSTRACT Texture features of histopathological images of lung carcinoma have been evaluated using gray level co-occurrence matrices and multiwavelets. The investigation is done from a pathological perspective resulting in optimum subset of features for classification.

This paper describes a new approach to detection of microcalcification clusters (MCs) in digital breast tomosynthesis (DBT) via its planar projection (PPJ) image.  With IRB approval, two-view (cranio-caudal and mediolateral oblique views) DBTs of human subject breasts were obtained with a GE GEN2 prototype DBT system that acquires 21 projection angles spanning 600 in 30 increments. A data set of 307 volumes (154 human subjects) was divided by case into independent training (127 with MCs) and test sets (104 with MCs and 76 free of MCs). Simultaneous algebraic reconstruction technique with multiscale bilateral filtering (MSBF) regularization was used to enhance microcalcifications and suppress noise. During the MSBF regularized reconstruction, the DBT volume was separated into high frequency (HF) and low frequency components representing microcalcifications and larger structures. At the final iteration, maximum intensity projection was applied to the regularized HF volume to generate a PPJ image that contained MCs with increased contrast-to-noise ratio (CNR) and reduced search space. High CNR objects in the PPJ image were extracted and labeled as microcalcification candidates. Convolution neural network (CNN) trained to recognize the image pattern of microcalcifications was used to classify the candidates into true calcifications and tissue structures and artifacts. The remaining microcalcification candidates were grouped into MCs by dynamic conditional clustering based on adaptive CNR threshold and radial distance criteria. False positive (FP) clusters were further reduced using the number of candidates in a cluster, CNR and size of microcalcification candidates. At 85% sensitivity an FP rate of 0.71 and 0.54 was achieved for view- and case-based sensitivity, respectively, compared to 2.16 and 0.85 achieved in DBT. The improvement was significant (p-value = 0.003) by JAFROC analysis.

Digital breast tomosynthesis (DBT) has the potential to replace digital mammography (DM) for breast cancer screening. An effective computer-aided detection (CAD) system for microcalcification clusters (MCs) on DBT will facilitate the transition. In this study, we collected a data set with corresponding DBT and DM for the same breasts. DBT was acquired with IRB approval and informed consent using a GE GEN2 DBT prototype system. The DM acquired with a GE Essential system for the patient’s clinical care was collected retrospectively from patient files. DM-based CAD (CADDM) and DBT-based CAD (CADDBT) were previously developed by our group. The major differences between the CAD systems include: (a) CADDBT uses two parallel processes whereas CADDM uses a single process for enhancing MCs and removing the structured background, (b) CADDBT has additional processing steps to reduce the false positives (FPs), including ranking of candidates of cluster seeds and cluster members and the use of adaptive CNR and size thresholds at clustering and FP reduction, (c) CADDM uses convolution neural network (CNN) and linear discriminant analysis (LDA) to differentiate true microcalcifications from FPs based on their morphological and CNN features. The performance difference is assessed by FROC analysis using test set (100 views with MCs and 74 views without MCs) independent of their respective training sets. At sensitivities of 70% and 80%, CADDBT achieved FP rates of 0.78 and 1.57 per view compared to 0.66 and 2.10 per image for the CADDM. JAFROC showed no significant difference between MC detection on DM and DBT by the two CAD systems.

Purpose: Detection of subtle microcalcifications in digital breast tomosynthesis (DBT) is a challenging task because of the large, noisy DBT volume. It is important to enhance the contrast-to-noise ratio (CNR) of microcalcifications in DBT reconstruction. Most regularization methods depend on local gradient and may treat the ill-defined margins or subtle spiculations of masses and subtle microcalcifications as noise because of their small gradient. The authors developed a new multiscale bilateral filtering (MSBF) regularization method for the simultaneous algebraic reconstruction technique (SART) to improve the CNR of microcalcifications without compromising the quality of masses.

Methods: The MSBF exploits a multiscale structure of DBT images to suppress noise and selectively enhance high frequency structures. At the end of each SART iteration, every DBT slice is decomposed into several frequency bands via Laplacian pyramid decomposition. No regularization is applied to the low frequency bands so that subtle edges of masses and structured background are preserved. Bilateral filtering is applied to the high frequency bands to enhance microcalcifications while suppressing noise. The regularized DBT images are used for updating in the next SART iteration. The new MSBF method was compared with the nonconvex total p-variation (TpV) method for noise regularization with SART. A GE GEN2 prototype DBT system was used for acquisition of projections at 21 angles in 3° increments over a ±30° range. The reconstruction image quality with no regularization (NR) and that with the two regularization methods were compared using the DBT scans of a heterogeneous breast phantom and several human subjects with masses and microcalcifications. The CNR and the full width at half maximum (FWHM) of the line profiles of microcalcifications and across the spiculations within their in-focus DBT slices were used as image quality measures.

Results: The MSBF method reduced contouring artifacts and enhanced the CNR of microcalcifications compared to the TpV method, thus preserving the image quality of the structured background. The MSBF method achieved the highest CNR of microcalcifications among the three methods. The FWHM of the microcalcifications and mass spiculations resulting from the MSBF method was comparable to that without regularization, and superior to that of the TpV method.

Conclusions: The SART regularized by the multiscale bilateral filtering method enhanced the CNR of microcalcifications and preserved the sharpness of microcalcifications and spiculated masses. The MSBF method provided better image quality of the structured background and was superior to TpV and NR for enhancing microcalcifications while preserving the appearance of mass margins.

To investigate the feasibility of a new two-dimensional (2D) multichannel response (MCR) analysis approach for the detection of clustered microcalcifications (MCs) in digital breast tomosynthesis (DBT). With IRB approval and informed consent, a data set of two-view DBTs from 42 breasts containing biopsy-proven MC clusters was collected in this study. The authors developed a 2D approach for MC detection using projection view (PV) images rather than the reconstructed three-dimensional (3D) DBT volume. Signal-to-noise ratio (SNR) enhancement processing was first applied to each PV to enhance the potential MCs. The locations of MC candidates were then identified with iterative thresholding. The individual MCs were decomposed with Hermite-Gaussian (HG) and Laguerre-Gaussian (LG) basis functions and the channelized Hotelling model was trained to produce the MCRs for each MC on the 2D images. The MCRs from the PVs were fused in 3D by a coincidence counting method that backprojects the MC candidates on the PVs and traces the coincidence of their ray paths in 3D. The 3D MCR was used to differentiate the true MCs from false positives (FPs). Finally a dynamic clustering method was used to identify the potential MC clusters in the DBT volume based on the fact that true MCs of clinical significance appear in clusters. Using two-fold cross validation, the performance of the 3D MCR for classification of true and false MCs was estimated by the area under the receiver operating characteristic (ROC) curve and the overall performance of the MCR approach for detection of clustered MCs was assessed by free response receiver operating characteristic (FROC) analysis. When the HG basis function was used for MCR analysis, the detection of MC cluster achieved case-based test sensitivities of 80% and 90% at the average FP rates of 0.65 and 1.55 FPs per DBT volume, respectively. With LG basis function, the average FP rates were 0.62 and 1.57 per DBT volume at the same sensitivity levels. The difference in the two sets of basis functions for detection of MCs did not show statistical significance. The authors&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; experimental results indicate that the MCR approach is promising for the detection of MCs on PV images. The HG or LG basis functions are both effective in characterizing the signal response of MCs using the channelized Hotelling model. The coincidence counting method for fusion of the 2D MCR in 3D is an important step for FP reduction. Further study is underway to improve the MCR approach for microcalcification detection in DBT.

Breast tomosynthesis is an emerging state-of-the-art three-dimensional (3D) imaging technology that demonstrates significant early promise in screening and diagnosing breast cancer. However, this kind of image has significant out-of-plane artifacts due to its limited tomography nature, which affects the image quality and further would interrupt interpretation. In this paper, we develop a robust deblurring method to remove or suppress blurry artifacts by applying three-dimensional (3D) nonlinear anisotropic diffusion filtering method. Differential equation of 3D anisotropic diffusion filtering is discretized using explicit and implicit numerical methods, respectively, combined by first (fixed grey value) and second (adiabatic) boundary conditions under ten nearest neighbor grids configuration of finite difference scheme. The discretized diffusion equation is applied in the breast volume reconstructed from the entire tomosynthetic images of breast. The proposed diffusion filtering method is evaluated qualitatively and quantitatively on clinical tomosynthesis images. Results indicate that the proposed diffusion filtering method is very powerful in suppressing the blurry artifacts, and the results also indicate that implicit numerical algorithm with fixed value boundary condition has better performance in enhancing the contrast of tomosynthesis image, demonstrating the effectiveness of the proposed filtering method in deblurring the out-of-plane artifacts.

Mammography reading by radiologists and breast tissue image interpretation by pathologists often leads to high False Positive (FP) Rates. Similarly, current Computer Aided Diagnosis (CADx) methods tend to concentrate more on sensitivity, thus increasing the FP rates. A novel method is introduced here which employs similarity based method to decrease the FP rate in the diagnosis of microcalcifications. This method employs the Principal Component Analysis (PCA) and the similarity metrics in order to achieve the proposed goal. The training and testing set is divided into generalized (Normal and Abnormal) and more specific (Abnormal, Normal, Benign) classes. The performance of this method as a standalone classification system is evaluated in both the cases (general and specific). In another approach the probability of each case belonging to a particular class is calculated. If the probabilities are too close to classify, the augmented CADx system can be instructed to have a detailed analysis of such cases. In case of normal cases with high probability, no further processing is necessary, thus reducing the computation time. Hence, this novel method can be employed in cascade with CADx to reduce the FP rate and also avoid unnecessary computational time. Using this methodology, a false positive rate of 8% and 11% is achieved for mammography and cellular images respectively.

An approach for optimum selection of lung nodule image characteristics in the feature domain is presented. This was applied to the classification module in the CAD system with data that was extracted from 42 ROI's of the 38 cases with an effective diameter of 3 to 8.5mm. 11 fundamental features were computed on the basis of dimensionality and image characteristics. The relation between the represented features of the 4 radiologists and the computed features was mapped using non-parametric correlation coefficients, multiple regression analysis and principle component analysis (PCA). Malignant and benign modules were classified based on the artificial neural network (ANN) to confirm the hypothesis from the mapping analysis. From the computed features and the radiologist's annotations, correlation coefficients between 0.2693 and 0.5178 were obtained. A combination of analyses namely regression, PCA, correlation and ANN were used to select optimum features. This resulted in F-test values of 0.821 and 0.643 for malignant and benign nodules respectively. The study of the relationship between the features and the weightage towards each of the representative classes resulted in optimum feature input for a CAD system. A composite analysis derived from correlation, PCA, multiple regression and the classification algorithm, collectively termed as the knowledge base, was used arrive at an "optimum" set of lung nodule features.

A novel approach to feature optimization for classification of lung carcinoma using tissue images is presented. The methodology uses a combination of three characteristics of computational features: F-measure, which is a representation of each feature towards classification, inter-correlation between features and pathology based information. The metadata provided from pathological parameters is used for mapping between computational features and biological information. Multiple regression analysis maps each category of features based on how pathology information is correlated with the size and location of cancer. Relatively the computational features represented the tumor size better than the location of the cancer. Based on the three criteria associated with the features, three sets of feature subsets with individual validation are evaluated to select the optimum feature subset. Based on the results from the three stages, the knowledgebase produces the best subset of features. An improvement of 5.5% was observed for normal Vs all abnormal cases with Az value of 0.731 and 74/114 correctly classified. The best Az value of 0.804 with 66/84 correct classification and improvement of 21.6% was observed for normal Vs adenocarcinoma.

The primary objective of this paper is to illustrate the applicability of reconstruction methods to objects of various geometrical shapes. Reconstruction methods are compared by evaluating the efficiency in reconstruction of individual geometric shapes. Signal difference noise ratio, artifact spread function; object extension and artifact extension are taken into consideration to determine the efficiency of the reconstruction methods. The reliability of reconstruction methods is compared qualitatively as well as quantitatively by graphically plotting the cumulative average of individual parameters against corresponding reconstruction methods. All the four parameters are found to be in good agreement with one another. Seven different filters namely Shift add filter, filtered back projection, enhanced shift add filter, enhanced filtered back projection, enhanced 2 shift add filter, enhanced 2 filtered back projection filter, enhanced 3 filtered back projection filter are compared in this paper for the quality of reconstruction and their reliability. Of all the methods, enhanced 3 filtered back projection proved to be the best one. Filtered back projection filter is found to have a clear edge over the shift add filter in viewing cylindrical as well as spherical objects. Besides, shift and add filter fails to trace minute spherical objects. Also, the histograms of average image contrast and average number of slices of both cylinder and the sphere clearly drive home the point. The more the number of slices, the greater is the image resolution. Likewise, the more the signal difference noise ratio, the better the image quality.