A New Collaborative Classification Process for Microcalcification Detection Based on Graphs and Knowledge Propagation

First Detector: Morphological-based Detector

The first detector is based on an unsupervised detection technique using morphological operations and the structural similarity index (SSIM) [30]. The use of mathematical morphology makes it easy to deal with the issue of low contrast between MCs and their surrounding pixels. The key features of this detector concern:

• The use of various structuring elements to reduce the sensitivity to the low and various contrast between MCs and their surrounding pixels.

• The generation of a suspicion map using structural similarity indices.

• The automatic estimation of threshold values locally determined from the SP1 superpixels using a dispersion analysis of both grayscale and suspicion maps to generate a thematic map.

Second Detector: Conditional Region Growing Detector

The conditional region growing (CRG) technique is based on the standard region growing [31] algorithm. Its main idea is to integrate prior knowledge-based criteria to control the growing process and correctly delineate MCs starting from selected seed points. These latter are selected based on the analysis of SP1 superpixels and a regional maxima detection. The criteria used to control the growing process are derived from the MC descriptions arising from radiologists and can be divided into two categories. The first one analyzes the neighborhood searching size. The second one exploits the gradient information and the shape evolution of the segmented region within the growing process. The key feature of this detector is to analyze below each individual MC to estimate the adequate criteria for an accurate delineation and not to use the same parameters for all of them. The SP2 superpixels are used by this detector to select the initial seed points for growing process purposes. However, the SP1 superpixels are used to estimate an intensity-based criterion as well as to select the set of candidate pixels for a possible evolution starting from a seed point.

Third Detector: PFCM-based Detector

The third detector is based on the possibilistic fuzzy C-means (PFCM) [32] algorithm. PFCM is an iterative unsupervised clustering algorithm that combines the advantages of both fuzzy C-means (FCM) [33] and possibilistic C-means (PCM) [34] algorithms. Indeed, it generates a fuzzy partition and associates to each pixel a membership and a typicality value. Membership values indicate the degrees to which pixels belong to each cluster. Typicality values indicate the degree of compatibility that a pixel has with respect to the cluster to which it belongs. The use of both membership and typicality values allows PFCM to solve the noise sensitivity issue of the FCM and to avoid the coincident cluster issue of the PCM [35].

MCs in mammographic images acquire the characteristics of the breast tissues on which they are superimposed. However, they appear with higher intensity values compared to them. Thus, they present low typicality degrees with respect to the cluster that represents their superimposed breast tissues. In order to identify these atypical pixels, the baseline technique [15] was based on PFCM algorithm and a static threshold value to segment the region of interest selected by the radiologists and where all presented MCs belong to. The change we have made, in this technique, consists of applying the PFCM algorithm to each SP1 superpixel in the entire mammographic image and to automatically estimate a threshold value per superpixel instead of using a single static value. Indeed, a mammographic image presents various breast tissues and the appearance of MCs is not restricted to only one breast tissue type. Thus, the use of a static threshold value makes the reliability of the results sensitive to the homogeneity of the superimposition tissues. Given that each SP1 superpixel refers to a homogeneous region and, thus, to a unique breast tissue type, the segmentation result will have a semantic signification and a local analysis will be able to estimate an accurate threshold value. Used threshold values are estimated on the basis of an intensity distribution analysis, inside each superpixel using John Wilder Tukey’s criterion [36].

Fourth Detector: Band-pass Fourier Filtering Detector

The last detector is an edge finder technique based on a Butterworth Band-Pass (BBP) filter in the Fourier domain. The BBP filter has the ability to properly control the frequencies by assigning the accurate low and high cutoff frequencies as well as the slope rate [29]. Thus, by applying the BBP filter to a mammographic image, fibroglandular tissue pixels are considered as background pixels and will be then removed. Also, MC edges will be considered foreground pixels and will be enhanced. To further improve the contrast of detected edges and reduce background noise, the resulting edge image is enhanced by applying a median filter and a gamma correction. The final image segmentation results from applying morphological operations.

Analyzing the descriptions of the four used detectors, we can notice that there is an implicit relationship between the thematic and suspicion maps generated by each of them where each of them can be derived from the other. The conversion, we propose, from one map to another is based on prior knowledge (Fig. 3). It is to analyze the gray-level distribution of the connected pixels in the thematic map and the suspicion degree distributions in the SP1 superpixels from the suspicion map.

From the thematic map, the obtained suspicious areas are selected and the gray-level distribution of the pixels in each of them is extracted. These distributions are shown to correspond to symmetrical Gaussian distributions with respect to regional maxima pixels. The suspicion degree values are estimated by transforming these distributions to probabilities based on the z-score table.

To transform a suspicion map into a thematic map, we start by projecting the SP1 superpixel boundaries on this latter. With such projection, the suspicion map is divided into local regions with homogeneous grayscale values. Adjacent pixels, which will be considered as potential MC pixels, are the outlier pixels in a considered superpixel. Therefore, we propose to estimate an adaptive threshold value for each superpixel based on modelling outlier pixels in it. In our study, the threshold value (for which a pixel P is considered a potential MC pixel on a given superpixel) is based on John Wilder Tukey’s criterion [36]. This latter is based on constructing boxplot on a given dataset.

The results of each of the used detectors are sensitive to the MC characteristic diversities. They usually generate false detections (FP, FN). Hence, combining these detectors is required to exchange their own knowledge and to reduce the overall FN rate.

Step 1: Generation of Initial Candidate Object Lists

The transformation of the thematic maps into lists of candidate objects is based on the SP2 map that associates a label to each superpixel. The number of different labels in this map is equal to that of the superpixels it contains. The proposed transformation is to project the SP2 map onto the different generated thematic maps. To accurately outline the identified objects starting from the suspicious areas, we propose to analyze the regions they occupy compared to those of the SP2 superpixels to which they belong. Figure 5 represents three samples of suspicious areas (white regions with red contour) in a thematic map (left image in each line). It illustrates the two possible situations when comparing a suspicious area to the SP2 superpixels (dashed blue contours):

• The suspicious area intersects only one superpixel (first line).

• The suspicious area intersects two (second line) or more (third line) superpixels.

White regions with red contours, in this figure, represent samples of suspicious areas in a thematic map (left image in each line).

For the first situation, we consider the suspicious area as a single candidate object. Its contours are those of this area while they do not extend those of the SP2 superpixel to which it belongs. For the second situation, we divide the suspicious area into different objects (two objects for the second line and nine objects for the third line). The intersections of the SP2 superpixel contours with the suspicious area define those associated to the identified objects. For instance, the second line in Fig. 5, the suspicious area (white region with red contour) intersects two different superpixels $S{P}_i$ and $S{P}_j$. The contour dividing these two superpixels (blue dashed contour) is the one that defines the outlines of the two identified objects (regions with green and yellow contours in the right image).

This configuration gives the opportunity to deal with all the issues of selecting the best contours of an object starting from a suspicious area. It also normalizes the object identification process for all the detectors and ensures that all the considered objects comply with or at least show very close shapes and sizes to those of MCs. Thus, the largest object we accept corresponds to an SP2 superpixel (ex. $O_3$, $O_5$ and $O_7$ from the third line in Fig. 5).

An object $O_i$ will be identified by its gravity center and a label inherent from that of the SP2 superpixel to which it belongs. Once these objects are identified, some characteristics are computed from the suspicion degree as well as the grayscale (mammographic image) maps and will be used in the collaborative process.

Step 2: Unification

The global list of candidate objects is the union of the lists identified from the different thematic maps arising from each detector. The objects with the same SP2 label will be considered a unique object. Their characterization results from projecting the SP2 superpixel map to the union thematic map. With this configuration, the same object can be differently characterized if it is not uniformly detected by the different detectors. Also, an object that was not detected by a given detector will inherit its properties from those associated in the global list. To resume, this list is a kind of a unifying result representing all the other ones.

Step 3: Refinement and Initial Classification

Once the global list of candidate objects is generated, we look back at the ones initially generated by the first step in order to:

• Display the identified objects from a same suspicious area relative to each list.

• Add the objects that do not appear in a list (not detected by a given detector).

• Associate a class label to each object.

In this work, we define three different class labels (Fig. 6), for the candidate objects, based on their occurrence among the detector results:

• The “absolute certainty” class label ($C_{\mathit{AC}}$) which is assigned to the candidate objects detected by at least $M-m+1$ detectors. M refers to the number of used detectors and m (integer with $m<M$) refers to the quality parameter based on which the decision about a candidate object can be considered certain. In this work, M is equal to 4 and m is equal to 1.

• The “partial certainty” class label ($C_{\mathit{PC}}$) which is assigned to the candidate objects detected by [$m+1$, ..., $M-m$] detectors.

• The “uncertainty” class label ($C_U$) which is assigned to the candidate objects detected by [1, ..., m] detectors.

Such labelling follows the ordinary reasoning. Indeed, it affirms that a decision coming with detectors’ agreement is a reliable decision ($m=1$).

Estimation of Confidence Degrees

This step consists on observing the objects’ labels in each SP1 superpixel and to compare their characteristics. The main idea is to estimate a confidence degree per detector and SP1 superpixel and to update the labels associated to the objects in order to reduce the disagreements they present between the detectors. The estimation of the confidence degrees per SP1 superpixel enhances the interest of the semantic information it produces for a better decision-making. Indeed, these regions present a grayscale homogeneity which refers to a semantic homogeneity in the image (the same breast tissue type pixels appear with similar gray-level values on the mammographic image). And therefore, MCs that appear in the same superpixel usually present similar characteristics, at least those detected by the same detector (Fig. 9).

The starting point for the confidence degree estimations is to compare the pairs of candidate object lists for each SP1 superpixel relatively to two detectors ($D_i$ and $D_j$). The similarity matrix ($S_k$) of the $k^{\mathit{th}}$ superpixel Eq. (7) we define evaluates the agreement between each $D_i$ and $D_j$ detector and is expressed as the proportion of their common detected objects (Os).

where

Once the similarity is calculated, we define the conflict degree ($C_{\mathit{ij}}^k$), which evaluates the disagreement between the $D_i$ and $D_j$ detectors Eq. (9), as well as the maximum conflict $C{C}^k$ Eq. (10) for the $k^{\mathit{th}}$ SP1 superpixel and thus to initialize the corresponding confidence degree $W_i^k$ Eq. (11) for each detector $D_i$. The maximum conflict degree $C{C}_i^k$ matches for a given detector $D_i$ its least correspondent detector $D_j$ in terms of common detected objects and will be used in the re-labeling process.

The estimated confidence degree plays an important role in selecting the superpixels to treat as well as fusing the final results.

Re-labelling/Classification of Candidate Objects

The iterative re-labeling step is applied for each SP1 superpixel and each $D_i$ detector. It aims to improve the quality of the results of each detector in order to reduce the conflict it presents with the others and, thus, to have objects with identical labels. It starts by selecting the SP1 superpixels ($S{P}_{\mathit{Cand}}^i$) with objects belonging to the $C_{\mathit{AC}}$ or $C_{\mathit{PC}}$ classes Eq. (12).

It is based on the fact that superpixels with objects presenting an absolute or a partial agreement by the detectors provide relevant information for better decision-making. It consists in updating the initial obtained graphs $G_{\mathit{Init}}^{i,k}$ generated on the $S{P}_k$ superpixels from $S{P}_{\mathit{Cand}}^i$ with applying the intra- and inter-superpixel analyses. These analyses calculate the similarities between the candidate objects ($Os{C}_i$), to be linked, and the nodes of the graph to be refined. These similarities are the weights of the new links added between the selected candidates and the old graph nodes.

(a) Intra-superpixel Analysis

Figure 10 illustrates an example of the intra-superpixel refinement.

A candidate object in an $S{P}_k$ superpixel, associated to the CU class, is added to the initial graph ($G_{\mathit{Init}}^{i,k}$) if and only if it satisfies the following condition:

where:

• $Sim{L}^{i,k}$ refers to the local similarity between the objects $O{I}_n$ and $O{I}_c$

• $Sim{G}^k$ refers to the normalized Euclidean distance between the $O{I}_{n1}$ and $O{I}_{n2}$ objects from the list of nodes of the $G_{i,k}^{\mathit{Init}}$ graph Eq. (14).

(b) Inter-superpixel Analysis

The inter-superpixel refinement (Fig. 11) is applied to the $G_L^{i,k}$ graph result from the local refinement. A new node (candidate object) which belongs to the immediate neighbors ($V(S{P}_k)$) of the $S{P}_k$ superpixel and associated to the CU class is accepted to be added to the $G_L^{i,k}$ graph only and only if it fulfills the following similarity criterion defined as the average of the normalized geometrical ($Sim{G}^k$) and feature ($Sim{F}^k$) similarities Eq. (15):

where “Feat(OI)” is the feature’s value of the candidate object OI. In this work, we adopted the standard gray-level deviation of the pixels constituting an object is considered as the feature.

Considering the new retained $G_{\mathit{Cont}}^{i,k}$ graph, with the tth iteration (t>1), the local and contextual refinements are applied to the one retained with the previous iteration ($t-1$) and not the $G_{\mathit{Init}}^{i,k}$ graphs of the t=1 iteration.

Overall Assessment

The true positive rates Eq. (17) resulting from applying the proposed collaborative process are presented in Fig. 13. Obtained measures prove the reliability of the proposed approach to identify the MCs. Indeed, it is able to identify more than 70% (respectively 90%) of MCs for 58% (respectively 80%) of the analyzed images. It is important to note here that the average of MCs per image is around 125 MCs. On the other side, the MC detection rate is lower than 50% for less than 4% of the images.

Performance Evaluation Compared to the Used Detectors

In order to evaluate the robustness of the proposed approach, it is important to prove its reliability compared to the used detectors. Table  1 displays the TP average detection rates obtained for each detector ($D_i$, i=1...4). It shows that the highest value (i.e., 85%) is derived from the proposed approach.

Table  2 presents the percentage analysis of the improved TP rates by the proposed approach compared to each of the used detectors.

The results in Table  2 indicate that more than the half of each of the initial detector results are improved by the proposed collaborative process. It also shows that the obtained TP rates are improved for 44% of the cases compared to those obtained by the best detector that returns the most relevant results for a given image. Such results highlight also the importance of adopting a collaborative fusion process instead of a simple detector to detect MCs. Indeed, even though the third detector ($D_3$) shows a comparable average of TP rate to that of the proposed approach, we have improved its TP rates for 52% of the cases.

Performance Evaluation Compared to Other Fusion Operators

In this study, the standard union (U) and intersection (I) operators for comparisons are used. This choice can be justified in particular by the features of the detectors as well as the mammographic image diversities. Indeed, no prior information is made about the global accuracy or reliability of the detector decisions. It depends on many factors such as the breast density and the MC feature diversities. From another side, the reliability of the used detectors depends on their reasoning compared to the MC appearances. For instance, the third detector describes the MCs as a set of adjacent “outlier” pixels in an SP1 superpixel. Despite the fact that it presents a concurrent average of TP rates, it shows some weaknesses with regard to MCs with various contrasts [29].

Figure 14 presents the boxplots of the different TP rates result from the intersection operator applied to combine the initial detectors’ decisions. It is clear that the proposed approach has successfully reduced the initial disagreements between the detectors and has improved the reliability of the intersection operator. However, the first assessment of the same figure with Table  3 suggests that the union operator presents higher TP rates compared to the proposed approach. Its average value is about 92% compared to 85% for the proposed approach.

In contrast, according to the quantitative analysis displayed in Table 4, the conjunctive operator, we applied, is a real competitor and even more reliable than the disjunctive operator applied to the initial detector results for 48% of the cases.

In addition to be competitive to the union operator, the proposed approach is able to reduce the false positive rates by an average of 32% for all the images. Such reduction helps discarding false positive objects with similar MC characteristics and, thus, to improve the reliability of a possible classification step which will discriminate the real MCs from the false positive ones. They also prove that the proposed approach was able to achieve its main objective, namely reducing the FN detections and reaching a competitive TP rated compared to those of the union operator.