Journal of Digital Imaging
https://doi.org/10.1007/s10278-020-00391-5
ORIGINAL PAPER
A Novel Fusion‑Based Texture Descriptor to Improve the Detection
of Architectural Distortion in Digital Mammography
Osmando Pereira Junior1 · Helder Cesar Rodrigues Oliveira2
Marcelo Andrade da Costa Vieira2 · Adilson Gonzaga2
· Carolina Toledo Ferraz3
· José Hiroki Saito3
·
Received: 30 July 2019 / Revised: 12 August 2020 / Accepted: 22 September 2020
© Society for Imaging Informatics in Medicine 2020
Abstract
Architectural distortion (AD) is the earliest sign of breast cancer that can be detected on a mammogram, and it is usually
associated with malignant tumors. Breast cancer is one of the major causes of death among women, and the chance of cure
can increase significantly when detected early. Computer-aided detection (CAD) systems have been used in clinical practice to
assist radiologists with the task of detecting breast lesions. However, due to the complexity and subtlety of AD, its detection
is still a challenge, even with the assistance of CAD. Recently, the fusion of descriptors has become a trend for improving
the performance of computer vision algorithms. In this work, we evaluated some local texture descriptors and their possible
combinations, considering different fusion approaches, for application in CAD systems to improve AD detection. In addition, we present a novel fusion-based texture descriptor, the Completed Mean Local Mapped Pattern (CMLMP), which is
based on complementary information between three LMP operators (signal, magnitude and center) and the local differences
between pixel values and the mean value of a neighborhood. We compared the performance of the proposed descriptor with
two other well-known descriptors: the Completed Local Binary Pattern (CLBP) and the Completed Local Mapped Pattern
(CLMP), for the task of detecting AD in 350 digital mammography clinical images. The results showed that the descriptor
proposed in this work outperforms the others, for both individual and fused approaches. Moreover, the choice of the fusion
operator is crucial because it results in different detection performances.
Keywords Architectural distortion · Fusion-based texture descriptor · Digital mammography · Fusion operators · Local
texture descriptor
* Osmando Pereira Junior
osmando@iftm.edu.br
Helder Cesar Rodrigues Oliveira
heldercro@usp.br
Carolina Toledo Ferraz
caroltoledoferraz@gmail.com
José Hiroki Saito
saito@cc.faccamp.br
Marcelo Andrade da Costa Vieira
mvieira@sc.usp.br
Adilson Gonzaga
agonzaga@sc.usp.br
1
Federal Institute of Education Science and Technology
of Triângulo Mineiro (IFTM), Patrocínio, Minas Gerais,
Brazil
2
São Carlos School of Engineering, University of São Paulo
(EESC/USP), São Carlos, São Paulo, Brazil
3
University Center Campo Limpo Paulista (UNIFACCAMP),
Campo Limpo Paulista (SP), São Paulo, Brazil
Introduction
Breast cancer is one of the leading causes of death among
women [1]. Early detection leads to better treatment options,
increasing the chances of curing the disease [2, 3]. X-ray
mammography is the most commonly used screening tool
for the early detection of breast cancer [4, 5]. Among breast
lesions, architectural distortion (AD) is the earliest sign of
breast cancer that can be detected on a mammogram [6, 7]
and there is a considerable risk of malignant tumors associated with it. However, AD is difficult to detect by the radiologist, as it is a subtle contraction of the breast parenchyma
with no defined mass or visible border [8, 9]. AD detection
is highly associated with the experience of radiologists, and
several cases are unnoticed by them [7, 10]. Thus, it is the
most common finding of retrospective reviews of false-negative mammograms, where approximately 75% of the malignancy cases were associated with AD [7, 11]. Therefore,
13
Vol.:(0123456789)
Journal of Digital Imaging
accurate detection of AD in mammography screening is
essential for early diagnosis of breast cancer and may significantly increase the chances of cure [2, 3].
Computer-aided detection (CAD) systems have been
used in clinical practice to assist radiologists with the task
of detecting breast lesions in mammography [12, 13]. Given
the complexity and subtlety of AD, even with the support
of CAD systems, its detection is still a challenge [9, 14].
CAD uses several techniques of computer vision and artificial intelligence to automatically detect suspicious lesions
in mammographic images, where the use of texture descriptors is essential, mainly for the detection of architectural
distortion [15, 16]. A texture descriptor aims to capture
the general characteristics of patterns in a local context. In
recent decades, several methods of texture analysis have
appeared. Among them, the Local Binary Pattern (LBP) has
been shown to be an efficient solution for the description of
textures and has been used successfully in many computer
vision tasks [16, 17].
To provide new effective information to LBP code for
texture classification, Guo et al. [18] proposed a completed
modeling of the local binary pattern operator, called the
Completed Local Binary Pattern (CLBP). In addition to
the conventional LBP descriptor, represented by the signal
operator (CLBP_S), two other operators were defined: the
magnitude (CLBP_M) and the center (CLBP_C). The first
two operators, CLBP_S and CLBP_M, are complementary
components and perform local feature extraction. The last
one, CLBP_C, applies a global threshold over the image.
The three components are then combined using two different fusion rules: concatenation and joint combination. The
final CLBP descriptor obtained better accuracy than others
based on the LBP [18].
Another texture descriptor, the Local Mapped Pattern
(LMP) [19], was presented as a generalization of the LBP.
A smooth approximation to the step function by a sigmoidal function was implemented in the feature extraction rule,
which allowed the LMP descriptor to better capture subtle
differences between the pixel values of a texture. The LMPbased descriptors achieved promising results in computer
vision applications and outperformed LBP-based descriptors in several applications, such as object classification [20],
color texture classification under varying illumination [21],
classification of rotated textures [22], iris recognition under
varying pupil size [23] and detection of architectural distortion in digital mammography [24].
Vieira et al. [22] proposed the Completed Local Mapped
Pattern descriptor (CLMP) for the classification of textures at different orientations considering that LMP-based
descriptors outperformed those based on LBP. As in the
case of CLBP, local differences were decomposed into two
complementary components, the signal (CLMP_S) and the
magnitude (CLMP_M). In addition to these operators, the
13
CLMP_C was proposed with the goal of adding global texture information as for the CLBP_C operator. In the context
of texture analysis in different orientations, the descriptor
CLMP performed better than CLBP, with less processing
time.
A variation of the LMP descriptor, the Mean Local
Mapped Pattern (M-LMP), was introduced by Ferraz
et al. [20] for the classification of objects in images. When
extracting the M-LMP features, pixels from a circular neighborhood were compared with the average value of gray levels in that neighborhood. M-LMP performance was compared with two other texture descriptors: the CS-LBP [25]
and SIFT [26], and the M-LMP achieved greater accuracy
with less processing time for the classification task. In addition to texture descriptors, other approaches have used statistical methods [27, 28] and convolutional neural networks
(CNNs) [29, 30] for feature extraction.
Recently, the fusion of descriptors has become a trend
for improving the performance of computer vision algorithms [31]. Thus, in this paper, we propose a novel fusionbased texture descriptor, the Completed Mean Local Mapped
Pattern (CMLMP), for application in mammography CAD
systems to improve AD detection. The CMLMP is based on
complementary information between three LMP operators
(signal, magnitude and center) and local differences between
pixel values and the mean value of a neighborhood. Furthermore, we evaluated the performance of the proposed
descriptor with two other texture descriptors, CLBP [18] and
CLMP [22], and their possible combinations considering
different fusion approaches.
Related Works
One of the most widely used texture descriptors is the Local
Binary Pattern (LBP) [17]. Due to its efficiency, simplicity and easy implementation, many variations have been
proposed for different tasks [18, 32–36]. Another approach
commonly used to describe textures in images is the ”statistical approach”. For the classification task, the aim is to find
a probabilistic decision rule that associates the texture with
a specific class [37]. Some textural measures normally used
are: energy, contrast and angular momentum.
Over the last ten years, several methods have been proposed for computerized detection of AD in mammographic
images, where a wide range of different approaches have
been proposed. Most of them performed the analysis of
breast tissue by applying texture descriptors to images
of digitized mammograms (screen-film mammography),
extracted from the Mini Mammographic Image Analysis
Society Digital Mammogram Database (mini-MIAS) [38]
and/or Digital Database for Screening Mammography
(DDSM) [39]. The most relevant works addressing the
Journal of Digital Imaging
detection of architectural distortion in mammography are
summarized in Table 1, which are briefly described in this
section.
Nemoto et al. [40] proposed a complete pipeline for the
analysis of 25 Fuji computed radiography (FCR) mammograms. After a preprocessing step, they extracted nine features from the suspicious regions: six of them (four contrast
measurements, energy and skewness) were calculated over
the intensity image; two (skewness and second moment)
were calculated over the Laplacian filtered image; and the
other one refers to the distance between the suspicious
region and the skin. The authors used the Mahalanobis distance for classification and reported a sensitivity of 80.0%
for AD classification.
Jasionowska et al. [10] proposed an approach based on
images in the frequency domain for AD detection, where 13
features were extracted from each suspicious region. Some
of them (contrast, correlation, energy, entropy and homogeneity) were based on the Gray-Level Co-occurrence Matrix
(GLCM), seven were statistical measures (variance, standard
deviation, kurtosis, skewness, energy and entropy), and two
(slope and point of intersection with y-axis) were computed
using the approximation of angular spectrum. After classification with support vector machine (SVM), they achieved
a sensitivity of 68.0%.
Minavathi et al. [41] proposed an AD detection pipeline
where textural, gradient-based and intensity features were
extracted from digitized mammograms. The authors used
both SVM and multilayer perceptron (MLP) classifiers.
The best achieved sensitivity was 94.7%, and the specificity was 89.6%.
Rangayyan et al. [42] proposed 13 features that aim
to assess the specularity and angular dispersion of breast
lesion patterns. They used MLP for classification and
obtained a sensitivity of 80.0% with an area under the
receiver operating characteristic curve (AUC) of 75.0%.
Mohammadi et al. [43] used the local texture descriptor called Monogenic Binary Coding (MBC) for detecting architectural distortion in Regions of Interest (ROI)
of digitized mammograms. They used the DDSM dataset
and measured the performance in terms of accuracy (Acc),
which best result equals 91.2%.
Kamra et al. [28] presented an approach in which 284
textural features were extracted from three datasets of
digitized mammograms: DDSM, mini-MIAS and a private dataset. A set of 280 features were obtained from the
GLCM with different radii, two features based on fractal analysis, and two features extracted from the Fourier
power spectrum. They reduced the feature vector’s size by
applying a postprocessing step and followed the classification with an SVM classifier. The results were reported
according to the dataset: for the mini-MIAS, it reached
an accuracy of 95.3% , sensitivity of 85.7% , specificity
of 97.2% and AUC of 98.0% ; for the DDSM, accuracy
of 92.9% , sensitivity of 93.3% , specificity of 92.0% and
AUC of 95.0% ; and for the private dataset, accuracy of
88.0%, sensitivity of 87.5%, specificity of 88.2% and AUC
of 93.0%.
Table 1 Relevant works addressing the detection of architectural distortion in mammography
Author
Dataset
#Images
with AD
#Images
without AD
Classification Performance*
Nemoto et al. (2009) [40]
Jasionowska et al. (2010) [10]
private FCR dataset
DDSM
25
33
-
S = 80%
S = 68%
Minavathi et al. (2011) [41]
Minavathi et al. (2011) [41]
Rangayyan et al. (2013) [42]
Mohammadi et al. [43]
mini-MIAS
DDSM
private digitized dataset
DDSM
19
23
106
Kamra et al. (2015) [28]
DDSM
Kamra et al. (2015) [28]
Kamra et al. (2015) [28]
mini-MIAS
private digitized dataset
60
13
Casti et al. (2016) [44]
Oliveira et al. (2017) [24]
Liu et al. (2018) [27]
Liu et al. (2018) [27]
Oliveira et al. (2019) [30]
Akhtar et al. (2020) [55]
Costa et al. (2020) [54]
DDSM
private FFDM dataset
DDSM
mini-MIAS
private FFDM dataset
DDSM & mini-MIAS
private FFDM dataset
150∗∗
∗∗
23∗∗
122
100
63
19
175
64
140
258∗∗
152
97
52
150∗∗
25
36
50∗∗
785
175
69
140
S = 94.7%, Sp = 89.6%
S = 94.7%, Sp = 89.6%
S = 80%, AUC = 75.0%
Acc = 91.2%
Acc = 92.9%, S = 93.3%, Sp = 92.0%, AUC = 95%
Acc = 95.3%, S = 85.7%, Sp = 97.2%, AUC = 98%
Acc = 88.0%, S = 87.5%, Sp = 88.2%, AUC = 93.0%
S = 81.0%
Acc = 90%, AUC = 97.3%
S = 90%
S = 90%
Acc = 75.3%, AUC = 82.9%
S = 85.0%
Acc = 80.9%, AUC = 89.0%
*Accuracy (Acc), sensitivity (S), specificity (Sp) and area under receiver operating characteristic curve (AUC)
**These values refer to the number of ROIs, not to the number of mammograms
13
Journal of Digital Imaging
Casti et al. [44] proposed a complete pipeline to detect
suspicious lesions in digitized mammograms, where a set
of 7 differential features (area, compactness, elongation,
distance, directionality, weighted directionality and dispersion) were calculated for classification of suspicious spots.
They used a classifier based on Fisher’s linear discriminant
analysis and reported a sensitivity of 81.0% for the detection of AD in the DDSM dataset.
Using a modified version of the Twin Bounded Support Vector Machine (TBSVM) for the detection of regions
with AD on the DDSM and MIAS dataset, Liu et al. [27]
proposed a method where 241 features were extracted: 228
features from the GLCM; 11 related to the measurement of
spiculated patterns; and the entropies of Renyi and Tsallis.
After the feature selection and classification steps, they
reported a sensitivity of 90.0% for AD detection.
Oliveira et al. [24] proposed the use of texture descriptors developed to map local micropatterns (LBP and LMP)
in the task of detecting AD. They used a dataset composed of 100 digital mamography images, from which they
extracted, per image, one ROI with AD and another one
with normal tissue. The efficiency of the proposed MLP
classifier was compared to four GLCM measurements:
contrast, correlation, energy and homogeneity. The best
two results were obtained with the LMP and energy of
GLCM, where the classifier reached AUCs of 97.3% and
94.6%, respectively.
In addition to the handcrafted descriptors, another methodology that has been recently reported in the literature for
AD detection was the use of convolutional neural networks
(CNNs). CNNs have been the winning card in computer
vision achieving good performance in many tasks [45].
Recently, many network architectures have been developed,
including Resnet [46], XNOR-Net [47], Mobilenet [48],
PPFNet [49] and DEP [29]. Some researchers have used
CNNs to extract features from images [50, 51] and then have
applied these features as descriptors for classification tasks.
Beyond these methods, there is a popular unsupervised
learning neural network called autoencoders [52].
Autoencoders are a representation learning method that
can learn data and represent them in lower dimensions [53].
Oliveira et al. [30] used an autoencoder to improve the detection of AD in digital mammography. The authors used two
different datasets, one to train the autoencoder and the other
to test the proposed model. They reported an accuracy of
75.3% and an AUC of 82.9%.
Costa et al. [54] evaluated the detection of AD in digital mammography using well-known deep learning models,
such as VGG-16, using transfer learning approaches. The
proposal is to map the region containing AD by aggregating the regions of interest recovered by the sliding window.
Among the several ways to train the VGG-16 layers, the best
result reached an accuracy of 80.9% and AUC of 89.0%.
13
Even today, in the digital era, new methods for AD
detection have been proposed based on digitized mammograms [9]. The study performed by Akhtar et al. [55] proposed a new approach using an ensemble composed of AdaBoost and Random Forest classifiers. They used 133 images
with and without AD from the DDSM and mini-MIAS databases. The results, reported by means of sensitivity in an
FROC curve with thresholds of 90% and 85%, indicate over
16 false-positive ROIs per image.
Despite the popularity of deep learning methods, there are
some issues related to the use of such an approach, including
the need for computers with high-performance GPUs, the
large number of images required for network training, the
loss of information in pooling layers and the overfitting of
the model [45, 56]. Furthermore, Liu et al. [57] presented a
large-scale study in which they evaluated the performance
of 32 LBP-based descriptors and compared them to those
of eight deep convolutional networks applied over 30 texture datasets for different classification tasks. The authors
showed that some LBP-based descriptors overcame CNN
approaches, achieving the best overall performance.
The CLMP descriptor [22] and CLBP [18] present different information among their three operators, which allows
more discriminating features when combining them. The
M-LMP descriptor [20] presented better performance than
other LMP and LBP variations.
Material and Method
Although full-field digital mammography (FFDM) is the
preferred tool for breast cancer screening, most of the solutions proposed in the literature for the detection of architectural distortion use datasets of digitized mammograms. To
assess our proposed descriptor for the AD detection task, we
created a dataset composed exclusively of FFDM images.
Digital Mammography Dataset
The dataset used in this study consists of 350 clinical images.
These data were acquired at the Institute of Radiology of
São Paulo (InRAD) of the University of São Paulo Medical
School (FMUSP) in a Hologic Selenia DimensionsTM system in the FFDM mode. This work was approved under the
Institutional Review Board (IRB #1.581.220 ). The dataset
is composed of 175 examinations with AD and 175 without
any type of lesion, i.e., healthy mammograms. All AD cases
were confirmed and marked by a radiologist.
ROIs with a size of 256 × 256 were extracted from the
central point of each region previously marked by the radiologist. Figure 1 shows an example of a clinical image from
our dataset and a magnified region containing the architectural distortion.
Journal of Digital Imaging
Fig. 1 Example of a mammography with architectural distortion
marked by a red box. The region containing the architectural distortion has been magnified for better visualization
AD is not associated with masses and is well characterized based on its neighborhood. Therefore, to increase the
number of samples in the dataset, we extracted 50 ROIs
around the center of the lesion. During the ROI extraction,
one criterion was considered: the central point of the AD
must be inside the selected ROI.
For mammograms without anomalies, 50 ROIs were
randomly extracted from the breast tissue. Figure 2 shows
examples of ROIs extracted from mammograms with and
without AD, where each ROI belongs to a different mammogram. The final dataset was composed of 17,500 samples.
Each ROI was processed for contrast enhancement, and the
pixel values were rescaled to the range [0, 255].
Fig. 2 Examples of ROIs (a) with AD and (b) without AD. Each ROI
was extracted from different clinical images
Feature Extraction and Classification
Figure 3 shows the framework of our method for architectural distortion recognition. We applied the texture
descriptors CLBP [18], CLMP [22] and the CMLMP
(Completed Mean Local Mapped Pattern), proposed in
this paper, over the entire dataset. We considered each
pixel surrounded by a symmetrical circular neighborhood
of unitary radius and with eight neighbors. Three coded
images [58] are generated for each descriptor, one per
component, signal (S), magnitude (M) and center(C). The
1-D histograms of the coded images are used to build the
feature vectors DESC_S, DESC_M and DESC_C, where
DESC = {CLBP, CLMP, CMLMP}.
To take advantage of the complementary information [18, 22] among these different descriptor components,
we used the same methods presented in [18] (concatenation, joint and hybrid combination) to fuse either the
13
Journal of Digital Imaging
Fig. 3 Framework of the proposed AD detection method
individual feature vectors or the individual coded images,
taken two by two or all three components.
We applied the concatenation fusion operator (⊕) [59],
as shown in Fig. 4, over the individual feature vectors
DESC_S, DESC_M and DESC_C, and generated four concatenated feature vectors for each texture descriptor, which
were denoted DESC_S⊕DESC_M, DESC_S⊕DESC_C,
DESC_M⊕DESC_C and DESC_S⊕DESC_M⊕DESC_C.
The concatenation operator can be described as follows.
Given two feature vectors, v1 with B1 bins, and v2 with B2
13
bins, we considered the concatenation of feature vectors
given by Eq. 1:
v3 = v1 ⊕ v2 ,
(1)
where v3 is the fused vector resulting from the concatenation
of v1 and v2 , whose size is B3 = B1 + B2.
The length of the concatenated feature vector is equal
to the sum of lengths of respective individual feature vectors. The use of the operator ⊕, instead of the notation ‘ _ ’
Journal of Digital Imaging
Fig. 4 The concatenation fusion operation
Fig. 5 The joint fusion operation
presented in [18], allows a clear representation of the concatenated components, even for the concatenation between
different descriptors.
To combine the coded images, we applied the joint fusion
operator (│) over them, as shown in Fig. 5. A joint 2-D
histogram of the coded images taken two by two was generated. The jointly fused feature vectors DESC_S│DESC_M,
DESC_S│DESC_C and DESC_M│DESC_C were obtained
by vectorizing these joint 2-D histograms. The length of
the resulting fused feature vector is equal to the product
between the maximum feature values B of the respective
individual coded images. The jointly 3-D histogram of the
three coded images was built and then converted into a 1-D
histogram to generate the jointly fused feature vector DESC_
S│DESC_M│DESC_C, with length B = BS ⋅ BM ⋅ BC . As
discussed before for the concatenation operator, the use of
the operator│to represent the Joint fusion, instead of the
notation ‘/’, also presented in [18], allows a clear representation for the joint fusion applied over different descriptors.
Concerning the hybrid combination, we concatenated
the individual feature vectors with the jointly fused ones.
Three fused vectors were generated, which we denoted
as DESC_S⊕(DESC_M│DESC_C), DESC_M⊕(DESC_
S│DESC_C) and DESC_C⊕(DESC_S│DESC_M).
To classify an image as AD or non-AD, we used the
K-nearest neighbors (KNN) classifier with the leave-oneout cross-validation method. Two different experiments
were performed (see Fig. 6). In experiment #1, the target
dataset consists of all images in dataset with the exception
of the current image to be predicted, whereas in experiment #2, the target dataset consists of all images, with the
exception of the N samples of the same mammogram as
the current image to be predicted.
The query sample, i.e., the sample to be predicted, is
compared to all samples in the target dataset. We measured their similarity by calculating the Euclidean distance,
defined by Eq. (2), between the feature vector of the query
13
Journal of Digital Imaging
Fig. 6 Leave-one-out cross-validation method. Experiment #1 the target dataset consists of all images, with the exception of the current
ROI to be predicted. Experiment #2 the target dataset consists of all
sample ( 𝐐 ) and the feature vector of the ith target sample
( 𝐓i ), where B stands for the length of the feature vectors.
√
√ B
√∑
(2)
d(𝐐, 𝐓i ) = √ (Qb − Tib )2
b=1
The query sample is considered of the same class as the
majority number of neighbors among the K nearest targets.
In other words, if K2 + 1 of samples from the K nearest
neighbors were of class AD, then the query sample is classified as AD; otherwise, it is classified as non-AD.
In the first moment, we used K = 1 and performed the first
nearest neighbor classifier (1-NN). In the second moment,
we determined a value K that increases the number of hits
achieved in the classification.
The performance of the AD detection method is evaluated
in terms of sensitivity (S) and accuracy (Acc). The sensitivity,
defined by Eq. (3), is the ratio of AD samples correctly classified as AD; and the accuracy, defined by Eq. (4), is the ratio of
all samples correctly classified (AD and non-AD), where true
positive (TP) is the number of AD samples correctly classified as AD, false negative (FN) is the number of AD samples
wrongly classified as non-AD, true negative (TN) is the number of non-AD samples correctly classified as non-AD, and
false positive (FP) is the number of non-AD samples wrongly
classified as AD.
S=
TP
TP + FN
13
(3)
images, with the exception of the N ROIs of the same mammogram
as the current ROI to be predicted
Acc =
TP + TN
TP + FN + TN + FP
(4)
From the treatment point of view, the occurrence of FN is
worse than FP, since in FN cases, the cancer has time to
grow, hindering the treatment [2, 3]. Then, a given descriptor could be considered more appropriate than others to be
used in the task of detecting architectural distortion when it
reaches the highest sensitivity with high accuracy.
Completed Mean Local Mapped Pattern (CMLMP)
The proposed descriptor Completed Mean Local Mapped
Pattern (CMLMP) is based on CLMP [22] complementary information and on the M-LMP [20] extraction feature method. It is composed of three components: signal (CMLMP_S), magnitude (CMLMP_M) and center
(CMLMP_C). The signal component considers local grayscale differences in a circular neighborhood for each pixel in
the image; the magnitude component relates local grayscale
differences with global information; and the center component takes into account only global texture information.
Each one of them is characterized by a specific feature
extraction rule and generates an individual coded image,
with one feature (code) for each pixel in the image, with
the exception of the pixels on the edge of the image. The
MATLAB code implementation for CMLMP is available for
download at https://github.com/LAVI-USP/CMLMP.
Let p be a pixel in the (x, y) coordinates of an X × Y image,
with gray value g(x, y), surrounded by a symmetrical circular neighborhood of radius R ( R > 0) and with N equally
spaced neighbors, as shown in Fig. 7. If the coordinates of
Journal of Digital Imaging
1
f(gdiff )
a
b
0.5
Fig. 7 The circular neighborhood for R = 1 and N = 8
b
p are (0,0), then the coordinates of each neighbor pi |Ni=1 are
(−R ⋅ sin(2𝜋(i − 1)∕N), R ⋅ cos(2𝜋(i − 1)∕N)) , with gray
value gi. The gray value of neighbors that are not located in
the center of each pixel can be estimated by bilinear interpolation [22, 60].
The feature extraction rules for CMLMP_S, CMLMP_M
and CMLMP_C applied over p(x, y) are presented in Eq.
(5), where h = {S, M, C}; W is a weight vector for which
Wi = 2(i−1); BS, BM and BC , with B ∈ ℤ+, are the maximum
feature values in the respective coded image; and fh is the
specific sigmoidal mapping function, defined in Eq. (6).
� ∑N
�
f W
i=1 hi i
CMLMP_h(x, y) = ∑N
(Bh − 1) + 0.5
(5)
W
i=1 i
1
gdiff _h
fh =
−
1+e
𝛽h
(6)
The grayscale differences for the signal, magnitude and
center components, gdiff _S , gdiff _M and gdiff _C , are defined in
Eqs. (7), (8) and (9), respectively.
gdiff _S =gi − gmean
(7)
gdiff _M =|gi − gmean | − G
(8)
gdiff _C =g(x, y) − G
(9)
where gmean is the average gray value for the N neighbors,
given by Eq. (10), and G is the average gray value among all
pixels of the X × Y image, defined by Eq. (11).
gmean =
G=
N
1∑
g
N i=1 i
X Y
1 ∑∑
g
X ⋅ Y x=1 y=1 (x,y)
(10)
(11)
0
-255
0
>> a
255
gdiff
Fig. 8 The mapping function behavior for a range of 𝛽 values (modified from [21])
The sigmoidal function f maps the grayscale difference gdiff
to a correspondent pattern in the respective coded image. Its
curve behavior is defined by the curve slope parameter (𝛽 ),
where 𝛽 is a positive real number.
Similar to other LMP-based descriptors, the performance
of the proposed descriptor is affected by 𝛽 , which changes
the sensitivity of the method to the nuances in the neighborhood [21]. As shown in Fig. 8, small values of 𝛽 saturate
the mapping function to zero, for negative gdiff , or to one,
for positive gdiff . As 𝛽 increases, the number of different
values returned by the mapping function f (gdiff ) increases.
Large values of 𝛽 make the sigmoidal mapping curve flatter,
closer to a straight line parallel to the gdiff axis, for which the
mapping function maps the grayscale differences to values
closer to 0.5.
The curve slope parameter for each sigmoidal mapping
function, 𝛽S , 𝛽M and 𝛽C , and the maximum feature values BS ,
BM and BC , must be tuned to achieve a high performance in
the desired application. We simultaneously tuned the parameters 𝛽 and B for each component by using a genetic algorithm, with accuracy as the objective function [21].
After coded image generation, we can build different feature vectors by considering the three CMLMP coded images
as individual or as fused approaches. For the individual
approach, we use each component as an individual descriptor, where the histogram of the coded image generates the
respective feature vector with length BS , BM or BC.
In the fused approach, by combining the individual
feature vectors, we generate the concatenated feature vectors CMLMP_S⊕CMLMP_M, CMLMP_S⊕CMLMP_C,
CMLMP_M⊕CMLMP_C and CMLMP_S⊕CMLMP_M⊕
CMLMP_C. The length of the concatenated feature vector
is equal to the sum of the length of the individual feature
vectors.
13
Journal of Digital Imaging
By combining the coded images, we build the joint 2-D
and the joint 3-D histograms of coded images and generate
the jointly fused feature vectors CMLMP_S│CMLMP_M,
CMLMP_S│CMLMP_C, CMLMP_M│CMLMP_C and
CMLMP_S│CMLMP_M│CMLMP_C, with length
B equals to BS ⋅ BM , BS ⋅ BC , BM ⋅ BC and BS ⋅ BM ⋅ BC ,
respectively.
We also generate the three hybrid feature vectors
CMLMP_S⊕(CMLMP_M│CMLMP_C), CMLMP_
M⊕( C M L M P _ S │ C M L M P _ C ) a n d C M L M P _ C⊕
(CMLMP_S│CMLMP_M).
Joint ( ˇ , B) Parameter Tuning
Before applying the descriptors over the dataset images,
we tuned the pair ( 𝛽 , B) parameters for both CMLMP
and CLMP descriptors. We selected 4% of the images
(700/17500) and the following constraints were assumed:
𝛽 ∈ [10−2 , 40], B ∈ [2 , 256], 50 individuals as the population size of the genetic algorithm with a maximum of 40
generations. The tuned values are presented in Table 2. To
illustrate the joint influence of 𝛽 and B on the performance
of the descriptors, we show in Fig. 9 the accuracy behavior
for a range of 𝛽 in (0 , 6] and B in (0 , 64] for CMLMP. The
range of values used to plot the curves is less than that used
to tune the parameters. We noted that small variations in 𝛽S
or in BS abruptly change the accuracy value for CMLMP_S,
where high accuracy is over the region BS < 20 ∪ 𝛽S < 2.
The CMLMP magnitude and center components presented
a smoother surface for accuracy.
The influence of the optimized parameters on the mapping function for each CMLMP component is shown in
Fig. 10. When analyzing a large range of gdiff , the sigmoidal
mapping function behavior of CMLMP_S resembles that of
the Heaviside step function, which is a consequence of the
Results and Discussion
In this section we present the acquired results from tuning the 𝛽 and B parameters, the results from two classification experiments and finally the computational complexity for all individual and fused descriptors by means
of processing time.
(a) 0.85
(b) 0.75
0.8
0.7
Acc
Acc
0.75
0.7
0.65
0.6
0.65
0.6
60
6
40
B
0.55
60
20
B
2
0
6
40
4
4
20
2
0
0
(c)
0
0.8
0.75
Acc
0.7
0.65
0.6
0.55
60
6
40
B
4
20
2
0
0
Fig. 9 Classification accuracy achieved with the application of (a) CMLMP_S, (b) CMLMP_M and (c) CMLMP_C, for a range of 𝛽 and B
13
Journal of Digital Imaging
Table 2 Tuned parameters
Descriptor
𝛽S
𝛽M
𝛽C
BS
BM
BC
CMLMP
CLMP [22]
0.1516
20.4247
26.2579
41
6
8
3.9095
39.8446
-
31
2
-
∗
*Proposed method
f(gdiff )
(a) 1
0.5
CMLMP_S
CMLMP_M
CMLMP_C
0
-100
0
100
gdiff
f( gdiff )
(b) 1
0.5
Fig. 11 Examples of coded images of CMLMP, CLMP and CLBP
considering a ROI with AD
CMLMP_S
CMLMP_M
CMLMP_C
0
-2
-1
0
gdiff
1
2
Fig. 10 The sigmoidal mapping function behavior of CMLMP
descriptor for optimized 𝛽S , 𝛽M and 𝛽C for gdiff in a range of (a) [-100 ,
100] and (b) [-2 , 2]
𝛽S small value (𝛽S = 0.1516). Although it saturates fS (gdiff )
to 0 for gdiff ≤ −1 or to 1 for gdiff ≥ 1, it represents a smooth
mapping for gdiff in (−1 , 1). CMLMP_C and CMLMP_M
mapping functions have curve behaviors similar to each
other, but since 𝛽C > 𝛽M , fC (gdiff ) returns values closer to
0.5 than fM (gdiff ) to the same gdiff value.
The descriptors were applied over the images from the
digital mammograms dataset considering a symmetrical
circular neighborhood with R = 1 and N = 8 . Figure 11
shows the generated coded images for a ROI with architectural distortion. When comparing the S-coded images with
the M-coded images, we noted that the former has more
detailed texture information, while the latter retained more
the spatial patterns. The C-coded images are the result of an
intensity transformation in the spatial domain applied over
the original ROI. The CLMP_C and the CLBP_C coded
images resulted from a binarization of the original ROI
with the threshold equal to the average gray value from the
whole image. The CMLMP_C coded image resulted from an
intensity transformation applied over the original ROI with
a sigmoidal mapping function (𝛽C = 26.2579) followed by
a quantization with 3 bits (values in [0,7]), since BC = 8.
By means of this figure, we might clearly see the complementary information among the different components from
the same descriptor.
Experiment #1: Test Set Composed of All
Mammograms, Excluding the Current ROI to Predict
In experiment #1, for each realization of the leave-one-out
cross-validation method, only the current query sample
13
Journal of Digital Imaging
Table 3 Sensitivity and accuracy (%) for individual descriptors using
the methodology of experiment #1
Descriptor
B
S
Acc
CMLMP_S∗
41
99.5
91.5
CMLMP_M∗
6
CMLMP_C
CLMP_S [22]
CLMP_M [22]
CLMP_C [22],CLBP_C [18]
CLBP_Sriu2 [18]
CLBP_Mriu2 [18]
8
74.0
72.4
31
2
2
10
10
95.7
91.6
72.4
87.9
76.1
86.0
63.5
54.4
80.2
72.2
∗
91.2
83.4
Entries in boldface indicate the best results achieved
*Proposed methods
was excluded from the target dataset, which means that the
query sample was compared with the other N − 1 samples
from the same mammogram and with those of the other
mammograms during the K-NN classification procedure.
Thus, the descriptors are expected to achieve high performance in experiment #1.
For classification, we considered K = 1 (1-NN), i.e.,
the query sample is classified as being of the same class as
the nearest neighbor. The performance for the individual
descriptors applied over the dataset is presented in Table 3.
The highest values for accuracy and sensitivity are shown
in boldface.
The signal component of the CMLMP descriptor
resulted in the best performance. It reached a sensitivity
of 99.5%, which means that 8,706 of 8,750 AD samples
were correctly classified as AD. Additionally, the accuracy
was equal to 91.5%, i.e., 16,013 samples from 17,500 were
correctly classified as AD or as non-AD. The classifier
returned only 44 FN samples; however, 1,443 FP occurrences were produced. The CLMP_S descriptor achieved
the second highest values for sensitivity and accuracy, S
= 95.7%, Acc = 86.0% and returned 376 FN and 2,074 FP.
The signal components yielded sensitivity values
greater than those of magnitude for all descriptors, and
the center components resulted in the lowest values.
When comparing component by component, the CMLMP
descriptor presented better results than the CLMP for
all components, and CLMP outperformed the CLBP
descriptor.
The considered methodology for this experiment modeled the problem as an image retrieval where the individual
to be classified is known by the model system, which is not
in accordance with the proposed application. In real cases,
the new mammogram to be analyzed is unknown by the
model system. Experiment #2, presented in the next section, addresses this new formulation of the problem. Nevertheless, experiment #1 is still relevant for analyzing the
13
descriptor performance in situations whose objective is to
retrieve images of the same individuals.
Experiment #2: Test Set Composed of N‑1
Mammograms, Excluding All ROIs of the Current
Mammogram to Predict
In experiment #2, the query sample was compared only
with the samples belonging to the other mammograms. The
results for the individual descriptors, considering K = 1, are
shown in Table 4. We can see a performance reduction for
all descriptors. CMLMP_S reached the best accuracy, Acc
= 68.8%, with a small value for sensitivity, S = 57.8%. The
CLMP_M achieved the best sensitivity, S = 91.1%, with
accuracy equals to 63.2%. To increase the descriptor performance, we optimized the constant K in the KNN algorithm,
considered in the classification process. Figure 12 shows the
accuracy achieved by the individual descriptor components
for a range of K values, K ∈ [1 , 2, 000]. For most descriptors, the highest accuracy was obtained with K equals to 100
nearest neighbors.
We combined the different components from the
same descriptor by using concatenation and joint fusion
operators. The results for individual and fused descriptors are shown in Table 5. For individual descriptors, the
CMLMP_C outperformed all others, resulting in the highest sensitivity, S = 81.5%, with accuracy equals to 76.0%.
The highest accuracy was reached by the CMLMP_S
descriptor, Acc = 77.3%, with sensitivity equals to 78.4%.
The CMLMP_M component resulted in sensitivity very
similar to that of CMLMP_S, S = 78.3%, but with a
smaller accuracy, Acc = 70.8%, and was overcome by
CLMP_M. Although the CLMP_M descriptor also outperformed the CMLMP_S in sensitivity, resulting in an
increase of 3%, it presented 2.7% lower accuracy. Beyond
that, CLMP_M has a very small feature vector size, with
Table 4 Sensitivity and accuracy (%) for individual descriptors using
the methodology of experiment #2 and K = 1
Descriptor
B
S
Acc
CMLMP_S∗
41
57.8
6
60.6
68.8
8
66.2
68.3
31
2
2
60.9
91.1
72.0
67.3
63.2
54.1
10
10
56.6
63.0
63.7
65.2
CMLMP_M∗
CMLMP_C∗
CLMP_S [22]
CLMP_M [22]
CLMP_C [22],
CLBP_C [18]
CLBP_Sriu2 [18]
CLBP_Mriu2 [18]
Entries in boldface indicate the best results achieved
*Proposed methods
66.7
Journal of Digital Imaging
0.8
0.75
Acc
0.7
0.65
CMLMP_S
CMLMP_M
CMLMP_C
CLMP_S
CLMP_M
CLBP_S
CLBP_M
CLBP_C
0.6
0.55
0.5
1
10
20
50
100 200 500 800 1000 1200 1500 2000
K nearest neighbours
Fig. 12 Accuracy of individual descriptor components for a range of
K values
only 2 bins (B = 2). The CLMP_C, which is the same as
the CLBP_C, presented the smaller performance among
all descriptors evaluated, and the CLBP_M resulted in
a higher sensitivity and accuracy when compared with
CLBP_S, and outperformed CMLMP_M in 2.3% for accuracy, with a reduction in sensitivity of 1.6%.
The performance of the fused descriptors is related to the
fusion operator and to the descriptors to be combined. Comparing the joint and the concatenation operators, we can see
that the former resulted in high accuracy for eleven of twelve
combinations. Only in the case of CMLMP_S│CMLMP_C,
the classification presented lower accuracy than CMLMP_
S⊕CMLMP_C. Regarding the sensitivity, joint operators
achieved higher values for eight of twelve combinations.
Only in the cases of CLMP_S⊕CLMP_M, CLMP_M⊕
CLMP_C, CLMP_S⊕CLMP_M⊕CLMP_C and CLBP_M⊕
CLBP_C the concatenation operators outperformed the joint
ones. Despite the best results achieved by the joint operator,
its feature vector size is larger than the concatenated fused
vectors. Consequently, it requires a larger time to determine
the distance between the samples, which could be a limiting issue for some applications, and could define the choice
by either approach. The high performance for jointly fused
descriptors is achieved by combining the three descriptor
components. This relationship is not observed for concatenated fusion vectors.
We generated the hybrid fused feature vectors by first
applying the joint operator over two components of a
descriptor and then concatenating the resulting joint feature vector with the third component. Their vector sizes are
larger than those of concatenated fused feature descriptors
but smaller than those of jointly fused descriptors.
For CLBP and CLMP, the hybrid fused descriptors
presented higher performance than the two other fusion
approaches. For CMLMP, joint combination yields the highest values.
Despite the sensitivity values reached by CLBP_Mriu2⊕
(CLBP_Sriu2│CLBP_C), S = 79.5%, and by CLMP_M⊕
(CLMP_S│CLMP_C), S = 80.4%, the highest reached
sensitivity for the respective descriptors, these values are
smaller than that achieved by the CMLMP_C individual
descriptor.
The best performance was reached by the CMLMP_
S│CMLMP_M│CMLMP_C fused descriptor, whose
sensitivity is equal to 85.7% with an accuracy of 77.0%.
However, it presents a large feature vector size, B = 1,968,
which directly increases the processing time. The CMLMP_
S│CMLMP_M presented the highest accuracy, Acc =
77.8%, but a small sensitivity, S = 80.5%. The jointly fused
descriptor CMLMP_S│CMLMP_C, with small size, B =
328, resulted in a performance strictly near to the best one,
S = 85.6% and Acc = 76.9%.
Computational Complexity
All the results were obtained by computer simulations in
MATLAB® R2018a, using an operational system Ubuntu
16.04 LTS, with a chipset Intel® CoreTM i7-6700 CPU @
3.40GHz, 32 GB of RAM memory and a TITAN XP NVIDIA
GPU with 12GB of GDDR5X memory.
A comparison of the average processing time is presented
in Fig. 13, where t feat , measured in milliseconds, refers to
the time to generate the feature vectors, and t dist , in microseconds, is the time to determine the Euclidean distance
between the query and the target samples. The descriptors
are arranged in descending order of sensitivity.
Generating the feature vectors is more expensive than
determining the distances. The former is related to the complexity of the feature extraction rule and to the fusion operation, and the latter is a consequence of the feature vectors
length. Although we cannot relate the performance of the
descriptor with its size, the highest t feat and t dist values are
achieved by CMLMP_S│CMLMP_M│CMLMP_C, with
the highest sensitivity too, and the smallest t feat and t dist are
required for CLMP_C, from which the sensitivity and the
accuracy are the smallest.
Conclusions
In this paper, we evaluated several fusion-based local texture
descriptors, considering different fusion approaches, for AD
classification in 350 digital mammography clinical images.
Moreover, we introduced a new fused-based texture descriptor, the CMLMP, and compared its performance with two
13
M
L
S
C
C MP
M
(C ML _M
C LM
M M |
M P
L P
LM _M
C MP _S CM
P_
M
(C LM _M | CMLM
C
C
P
M
M P |
LM
(C LM _M CM LM _C
P_
M P | L P_
M
S C LM _S CM P C
M P |
_
C LM _S CM LM C)
M P | L P
L
_
C
M
C M S M P _C
M P
LM _M C LM _C
M P_ )
P_
M C LM M)
C
M P
LM
C LM _C
P_ CM
M P
M L
C LM _C
C
M
M P
LM
LM _C
(C P_
P
S
C _S C LM | CL P_
LB
L P C M C
P_ C MP _S ML P_
M LM _M | C MP M
L _
C CM (C P_M C MP M
LM
L
LM _C
P_ LM BP
)
S| P_ _S CL P_C
C S | C MP
LM
LB _
C P_ CM P_ C
LM M LM C
P_ | C P )
S| LM _M
C P
C
C LM _C
LM
M P
_
P
C _S CM LM C
LM
LM P_S
P_ C P
C
LB
S LM _M
|C P
P_ C
L _
S| LB
C P C MP M
LB _ L _
P_ S| C MP M
M L _S
C
| C BP
LM
LB _C
P
C
C
_
C P
L
M
LM
C BP | LB _C
L
C P_
LM S C BP _S CL P_M
_ | M
C P_ (C LBP S CL P_
LB C L _
B C
P_ ( M M CL P_
BP M
C CL P_
C
M M C
LB
(C P_ | C LB _M
P_
LB S L P_
S C P | C MP C
L _S L _
C
C BP | MP C)
LB
LB _ C _
P_
M L
C
S L P_M | BP M)
C _M
M
LB
(C P_
LB S C P_ )
LB C
P_
M CL P_
C
| C MP C
LB
LB _C
P_
C S C P_
LM
LB C
P_ C P_ )
C LB S
,C P
LB _C
P_
C
C
S|
P_
LM
LM
P_
M
C
M
C
t dist (us)
C
P_
S
P_
S|
C
M
L
C MP
M
(C ML _M
C LM
M M |
M P_
L P
LM M
C MP _S CM
P_
M
(C LM _M | CMLM
C
C
P
M
M P |
LM
(C LM _M CM LM _C
P_
M P | L P_
M
S C LM _S CM P C
M P |
_
C LM _S CM LM C)
M P | L P
L
_
C
M
C M S M P _C
M P
LM _M C LM _C
M P_ )
P_
M C LM M)
C
M P
LM
C LM _C
P_ CM
M P
M L
C LM _C
C
M
M P
LM
LM _C
(C P_
P
S
C _S C LM | CL P_
LB
L P C M C
P_ C MP _S ML P_
M LM _M | C MP M
L _
C CM (C P_M C MP M
LM
L
LM _C
P_ LM BP
)
S| P_ _S CL P_C
C S | C MP
LM
LB _
C P_ CM P_ C
LM M LM C
P_ | C P )
S| LM _M
C P
C
C LM _C
LM
M P
_
P
C _S CM LM C
LM
LM P_S
P_ C P
C
LB
S LM _M
|C P
P_ C
L _
S| LB
C P C MP M
LB _ L _
P_ S| C MP M
M L _S
C
| C BP
LM
LB _C
P
C
C
_
C P
L
M
LM
C BP | LB _C
L
C P_
LM S C BP _S CL P_M
_ | M
C P_ (C LBP S CL P_
LB C L _
B C
P_ ( M M CL P_
BP M
C CL P_
C
M M C
LB
(C P_ | C LB _M
P_
LB S L P_
S C P | C MP C
L _S L _
C
C BP | MP C)
LB
LB _ C _
P_
M L
C
S L P_M | BP M)
C _M
M
LB
(C P_
LB S C P_ )
LB C
P_
M CL P_
C
| C MP C
LB
LB _C
P_
C S C P_
LM
LB C
P_ C P_ )
C LB S
,C P
LB _C
P_
C
LM
LM
M
C
M
C
t feat (ms)
Journal of Digital Imaging
(a) 20
15
13
10
5
0
(b)14
12
10
8
6
4
2
0
Fig. 13 Processing time (a) for feature vector generation, in ms, and (b) for Euclidean distance determination, in 𝜇s, for all individual and fused
descriptors in descending order of sensitivity
other previously published descriptors, CLMP and CLBP,
and their possible combinations considering joint and concatenation fusion operators. The center component of the
proposed descriptor, CMLMP_C, achieved higher sensitivity
than all individual and fused approaches for CLBP and for
CLMP descriptors. Despite the smallest size of the concatenated fused feature vectors, the jointly fused descriptors
resulted in higher sensitivity and accuracy values for combining the CMLMP components.
In conclusion, the proposed descriptor outperforms the
others, for both individual and fused approaches, and the
fused approach improves the performance of the descriptor. Moreover, the choice of the fusion operator is crucial
because it results in different detection performances. In
future work, we intend to evaluate the fusion among different
descriptors, considering multiscale and multimodal analysis.
Journal of Digital Imaging
Table 5 Sensitivity (%),
accuracy (%) and average
processing times for individual
and fused descriptors using the
methodology of experiment #2
and K = 100
Descriptor
CMLMP_S
∗
CMLMP_M
CMLMP_C
∗
∗
CMLMP_S⊕CMLMP_M∗
CMLMP_S⊕CMLMP_C∗
CMLMP_M⊕CMLMP_C∗
CMLMP_S⊕CMLMP_M⊕CMLMP_C
∗
CMLMP_S │ CMLMP_M∗
CMLMP_S │ CMLMP_C∗
CMLMP_M │ CMLMP_C∗
CMLMP_S │ CMLMP_M │ CMLMP_C
∗
CMLMP_S⊕(CMLMP_M │ CMLMP_C)
∗
CMLMP_M⊕(CMLMP_S │ CMLMP_C)∗
CMLMP_C⊕(CMLMP_S │ CMLMP_M)∗
CLMP_S [22]
CLMP_M [22]
CLMP_C [22] , CLBP_C [18]
CLMP_S⊕CLMP_M [22]
CLMP_S⊕CLMP_C [22]
CLMP_M⊕CLMP_C [22]
CLMP_S⊕CLMP_M⊕CLMP_C [22]
CLMP_S │ CLMP_M [22]
CLMP_S │ CLMP_C [22]
CLMP_M │ CLMP_C [22]
CLMP_S │ CLMP_M │ CLMP_C [22]
CLMP_S⊕(CLMP_M │ CLMP_C) [22]
CLMP_M⊕(CLMP_S │ CLMP_C) [22]
CLMP_C⊕(CLMP_S │ CLMP_M) [22]
CLBP_Sriu2 [18]
CLBP_Mriu2 [18]
CLBP_Sriu2⊕CLBP_Mriu2 [18]
CLBP_Sriu2⊕CLBP_C [18]
CLBP_Mriu2⊕CLBP_C [18]
CLBP_Sriu2⊕CLBP_Mriu2⊕ CLBP_C [18]
CLBP_Sriu2 │ CLBP_Mriu2 [18]
CLBP_Sriu2 │ CLBP_C [18]
CLBP_Mriu2 │ CLBP_C [18]
CLBP_Sriu2 │ CLBP_Mriu2 │ CLBP_C [18]
CLBP_Sriu2⊕(CLBP_Mriu2 │ CLBP_C) [18]
CLBP_Mriu2⊕(CLBP_Sriu2 │ CLBP_C) [18]
CLBP_C⊕(CLBP_Sriu2 │ CLBP_Mriu2) [18]
Autoencoder∗∗ [30]
AlexNet
B
S
Acc
41
78.4
77.3
8.5
1.0
6
78.3
70.8
7.4
0.6
8
81.5
76.0
1.1
0.6
47
79.3
77.3
15.9
0.8
49
82.9
76.1
9.6
0.9
14
81.6
76.2
8.5
0.6
55
82.4
76.3
17.0
0.9
246
80.5
16.7
2.2
328
85.6
76.9
10.4
2.7
48
83.1
75.6
9.2
0.8
77.0
18.2
13.1
1,968
89
85.7
77.8
t feat(ms)
tdist(𝜇s)
85.1
76.1
17.7
1.1
334
83.1
76.8
17.8
2.8
254
83.0
76.1
17.8
2.2
31
2
2
33
33
4
35
62
62
4
124
35
64
64
10
10
20
12
12
22
100
20
20
200
30
30
102
4096
77.2
81.4
57.1
78.1
71.0
80.0
79.5
78.0
79.1
76.3
79.2
73.5
80.4
72.9
67.8
76.7
74.6
65.5
74.3
71.4
75.8
77.0
72.5
77.0
70.7
79.5
72.9
80.0
75.1
74.6
63.6
74.3
71.9
73.6
74.5
74.5
76.4
73.7
76.7
73.9
76.5
72.2
71.2
73.1
72.5
70.2
71.5
71.0
74.4
74.1
73.8
76.0
73.9
75.1
72.2
70.0
8.4
6.5
0.6
14.9
9.0
7.1
15.5
15.6
9.7
7.8
16.6
16.2
16.2
16.2
6.8
6.8
13.6
7.4
7.4
14.2
14.3
8.1
8.1
15.3
14.9
14.9
14.3
-
0.7
0.6
0.6
0.8
0.8
0.6
0.8
0.9
0.9
0.6
1.4
0.8
0.9
0.9
0.6
0.6
0.6
0.6
0.7
1.2
1.2
0.7
0.7
1.8
0.7
0.7
1.2
-
-
71.2
77.5
-
-
Entries in boldface indicate the best results achieved.
*Proposed methods
**The sensitivity (S) of the autoencoder was calculated exclusively for this paper
Acknowledgements This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil
(CAPES) - Finance Code 001 and by the São Paulo Research Foundation (FAPESP), grant #2015/20812-5. The content is solely the
responsibility of the authors and does not necessarily represent the
official views of the CAPES or FAPESP. The authors would like to
thank Rodrigo B. Vimieiro for his helpful advice and comments in this
13
Journal of Digital Imaging
paper. We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan XP GPU used for this research.
References
1. WHO, ”World health organization - breast cancer,” 2019.
[Online]. Available: http://www.who.int/
2. Elmore JG, Nakano CY, Koepsell TD, et al: International variation
in screening mamography interpretations in community-based
programs. J Natl Cancer Inst 95:(18)1384-1393, 2003
3. Veronesi U, Boyle P, Goldhirsch A, et al: Breast Cancer. The Lancet 365:1727–1741, 2005
4. Karellas A, and Vedantham S: Breast cancer imaging: a perspective for the next decade. Medical Physics 35:(11)4878–4897, 2008
5. Glynn CG, Farria DM, Monsees BS, et al: Effect of transition to digital mammography on clinical outcomes. Radiology
260(3)664–670, 2011
6. Gaur S, Dialani V, Slanetz PJ, et al: Architectural Distortion of the
Breast. American Journal of Roentgenology 201:662–670, 2013
7. Bahl M, Baker JA, Kinsey EN, et al: Architectural Distortion
on Mammography: Correlation With Pathologic Outcomes and
Predictors of Malignancy. American Journal of Roentgenology 205:(6)1339–1345, 2015
8. D’Orsi CJ, Ed.: ACR BI-RADS Atlas, Breast Imaging Reporting
and Data System, American College of Radiology, 2013
9. Suleiman WI, Mcentee MF, Lewis SJ, et al: In the Digital Era,
Architectural Distortion Remains a Challenging Radiological
Task. Clinical Radiology 71:(1)e35–e40, 2016
10. Jasionowska M, and Przelaskowski A: A two-step method for
detection of architectural distortions in mammograms. Information Technology in Biomedicine 69:73–84, 2010
11. Ray KM, Turner E, Sickles EA, et al: Suspicious Findings at Digital Breast Tomosynthesis Occult to Conventional Digital Mammography, Imaging Features and Pathology Findings. The Breast
Journal, 2015, pp 1–5
12. Dean JC and Ilvento CC: Improved Cancer Detection Using Computer-Aided Detection with Diagnostic and Screening Mammography - prospective study of 104 cancers. American Journal of
Roentgenology 187:20–28, 2006
13. Yang SK, Moon WK, Cho N, et al: Screening mammographydetected cancers: Sensitivity of a computer-aided detection system
applied to full-field digital mammograms. Radiology 244:(1)104–
111, 2007
14. Kohli A and Jha S: Why CAD Failed in Mammography. Journal
of the American College of Radiology 15:(3)535–537, 2018
15. Rangayyan RM, Banik S, and Desautels JEL: Computer-Aided
Detection of Architectural Distortion in Prior Mammograms of
Interval Cancer. Journal of Digital Imaging 23:(5)611–631, 2010
16. Oliveira HCR, Mencattini A, Casti P, et al: Reduction of false-positives in a CAD scheme for automated detection of architectural
distortion in digital mammography, in Medical Imaging 2018:
Computer-Aided Diagnosis, Mori K and Petrick N, Eds. 10575.
SPIE, 2018, pp 96
17. Ojala T, Pietikäinen M, and Harwood D: A comparative study of
texture measures with classification based on featured distributions. Pattern Recognition 29:(1)51–59, 1996
18. Guo Z, Zhang L, and Zhang D: A completed modeling of local
binary pattern operator for texture classification. IEEE Transactions on Image Processing 19:(6)1657–1663, 2010
19. Ferraz CT, Pereira Jr O, and Gonzaga A: Feature description based
on center-symmetric local mapped patterns, in Proceedings of the
29th Annual ACM Symposium on Applied Computing (SAC’14).
2014, pp 39–44
13
20. Ferraz CT, Pereira Jr O, Rosa MV, et al: Object recognition based
on bag of features and a new local pattern descriptor. International Journal of Pattern Recognition and Artificial Intelligence
28:(8)14550101–145501032, 2014
21. Negri T, Zhou F, Obradovic Z, et al: A robust descriptor for color
texture classification under varying illumination, in International
Conference on Computer Vision, Theory and Applications.
2017, pp 378–388
22. Vieira RT, Negri TT, and Gonzaga A: Improving the classification
of rotated images by adding the signal and magnitude information to a local texture descriptor. Multimedia Tools and Applications 77:(23)31041–31066, Dec 2018
23. de Souza JM and Gonzaga A: Human iris feature extraction under
pupil size variation using local texture descriptors. Multimidia
Tools and Applications, 2019, pp. 1–28
24. Oliveira HCR, Moraes DR, Reche GA, et al: A new texture
descriptor based on local micro-pattern for detection of architectural distortion in mammographic images, in Proc. SPIE Medical Imaging 2017: Computer-Aided Diagnosis, Armato SG and
Petrick NA Eds, 2017, 10134:101342U
25. Heikkilä M, Pietikäinen M, and Schmid C: Description of interest
regions with local binary patterns. Pattern Recognition 42:425–
436, 2009
26. Lowe DG: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60:(2)91–110,
2004
27. Liu X, Zhai L, Zhu T, et al: Multiple TBSVM-RFE for the detection of architectural distortion in mammographic images. Multimidia Tools and Applications 77:(12)15773–15802, 2018
28. Kamra A, Jain VK, Singh S, et al: Characterization of architectural
distortion in mammograms based on texture analysis using support vector machine classifier with clinical evaluation. Journal of
Digital Imaging 29:(1)104–114, 2016
29. Xue J, Zhang H, and Dana K: Deep texture manifold for ground
terrain recognition, in The IEEE Conference on Computer Vision
and Pattern Recognition (CVPR). June 2018
30. Oliveira HC, Melo CFE, Catani JH, et al: Exploratory learning
with convolutional autoencoder for discrimination of architectural
distortion in digital mammography, in Proc. of SPIE Medical
Imaging 2019: Computer-Aided Diagnosis, Hahn HK and Mori
K, Eds. SPIE, mar 2019, pp 8
31. Ionescu B, Benois-Pineau J, Piatrik T, et al: Fusion in Computer
Vision: Understanding Complex Visual Content. Springer, Berlin,
2014
32. Cheung Y and Deng J: Ultra local binary pattern for image texture analysis, in Proceedings 2014 IEEE International Conference
on Security, Pattern Analysis, and Cybernetics (SPAC). 2014, pp
290–293
33. Yuan J, Huang D, Zhu H, et al: Completed hybrid local binary
pattern for texture classification, in 2014 International Joint Conference on Neural Networks (IJCNN). 2014, pp 2050–2057
34. Pan Z, Fan H, and Zhang L: Texture classification using local pattern based on vector quantization. IEEE Transactions on Image
Processing 24:(12)5379–5388, 2015
35. Talab ARR and Shakoor MH: Fabric classification using new
mapping of local binary pattern, in 2018 International Conference on Intelligent Systems and Computer Vision (ISCV). 2018,
pp 1–4
36. Deng W, Hu J, and Guo J: Compressive binary patterns: Designing a robust binary face descriptor with random-field eigenfilters, IEEE Transactions on Pattern Analysis and Machine Intelligence 41:(3)758–767, 2019
37. Sonka M, Hlavac V, and Boyle R: Image Processing, Analysis,
and Machine Vision. Springer US, 1993
Journal of Digital Imaging
38. Suckling J, Parker J, Dance D, et al: The mammographic image
analysis society digital mammogram database, in Exerpta Medica.
International Congress Series 1069:375–378, 1994
39. Heath M, Bowyer K, Kopans D, et al: The digital database for
screening mammography, in Proceedings of the 5th international
workshop on digital mammography, 2000, pp 212–218
40. Nemoto M, Honmura S, Shimizu A, et al: A pilot study of architectural distortion detection in mammograms based on characteristics of line shadows. International Journal of Computer Assisted
Radiology and Surgery 4:(1)27–36, 2009
41. Minavathi, Murali S, and Dinesh MS: Model based approach for
Detection of Architectural Distortions and Spiculated Masses in
Mammograms. International Journal on Computer Science and
Engineering 3:(11)3534–3546, 2011
42. Rangayyan RM, Banik S, Chakraborty J, et al: Measures of divergence of oriented patterns for the detection of architectural distortion in prior mammograms. International Journal of Computer
Assisted Radiology and Surgery 8:(4)527–545, 2013
43. Mohammadi E, Fatemizadeh E, Sheikhzadeh H, et al: A textural
approach for recognizing architectural distortion in mammograms
in 2013 8th Iranian Conference on Machine Vision and Image
Processing (MVIP). 2013, pp 136–140
44. Casti P, Mencattini A, Salmeri M, et al: Contour-independent
detection and classification of mammographic lesions. Biomedical Signal Processing and Control 25:165–177, 2016
45. Krizhevsky A, Sutskever I, and Hinton GE: Imagenet classification with deep convolutional neural networks, in Proceedings of
the 25th International Conference on Neural Information Processing Systems Volume 1, ser. NIPS’12. Curran Associates Inc. USA,
2012, pp 1097–1105
46. He K, Zhang X, Ren S, et al: Deep residual learning for image
recognition, CoRR, vol. abs/1512.03385, 2015. [Online]. Available: http://arxiv.org/abs/1512.03385
47. Rastegari M, Ordonez V, Redmon J, et al: ”Xnor-net: Imagenet
classification using binary convolutional neural networks,” CoRR,
vol. abs/1603.05279, 2016. [Online]. Available:http://arxiv.org/
abs/1603.05279
48. Howard AG, Zhu M, Chen B, et al: ”Mobilenets: Efficient convolutional neural networks for mobile vision applications,” CoRR,
vol. abs/1704.04861, 2017. [Online]. Available: http://arxiv.org/
abs/1704.04861
49. Deng H, Birdal T, and Ilic S, Ppfnet: Global context aware local
features for robust 3d point matching, in The IEEE Conference on
Computer Vision and Pattern Recognition (CVPR). June 2018
50. Bianco S, Cusano C, Napoletano P, et al: Improving cnn-based
texture classification by color balancing. Journal of Imaging 3:3307, 2017
51. Na L, Xiankai L, Wan L, et al: Improving the separability of deep
features with discriminative convolution filters for RSI classification. International Journal of Geo-Information 7:(3)95, 2018
52. Yang Y, Feng C, Shen Y, et al: Foldingnet: Interpretable unsupervised learning on 3d point clouds, CoRR, vol. abs/1712.07262,
2017. [Online]. Available:http://arxiv.org/abs/1712.07262
53. Zhao C: An autoencoder-based image descriptor for image matching and retrieval. PhD Dissertation, Wright State University,
1467, 2016. Available: https://corescholar.librar ies.wright.edu/
etd_all/1467
54. Costa AC, Oliveira HCR, Borges LR, et al: Transfer learning in
deep convolutional neural networks for detection of architectural
distortion in digital mammography in 15th International Workshop on Breast Imaging (IWBI2020), H. Bosmans, N. Marshall,
and C. V. Ongeval, Eds., vol. 11513, International Society for
Optics and Photonics. SPIE, pp 170–177, 2020
55. Akhtar Y and Mukherjee DP: Context-based ensemble classification for the detection of architectural distortion in a digitised
mammogram. IET Image Processing 14:(4)603–614, 2020
56. Schmidhuber, J: Deep learning in neural networks: An overview, Neural Networks, vol. 61, pp. 85–117, 2015. [Online].
Available:http://www.scienc edire ct.com/scienc e/articl e/pii/S0893
608014002135
57. Liu L, Fieguth P, Guo Y, et al: Local binary features for texture
classification: Taxonomy and experimental study. Pattern Recognition 62:135–160, 2017
58. Chen C, Zhang B, Su H, et al: Land-use scene classification using
multi-scale completed local binary patterns. Signal, Image and
Video Processing (SIViP) 10:(4)745–752, 2016
59. Pereira Jr O, Ferraz CT, and Gonzaga A: Image correspondence
using a fusion of local region descriptors, in XIV Workshop de
Visão Computacional (WVC 2018), 2018.
60. Ojala T, Pietikäinen M, and Mäenpää T: Multiresolution grayscale and rotation invariant texture classification with local binary
patterns. IEEE Transactions on Pattern Analysis and Machine
Intelligence 24:(7)971–987, 2002
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
13