Texture Based Segmentation

Reyer Zwiggelaar1 and Erika R.E. Denton2

1
Department of Computer Science,
University of Wales,
Aberystwyth SY23 3DB, UK
rrz@aber.ac.uk
2
Norfolk and Norwich University Hospital,
Norwich NR4 7UY, UK

Abstract. The ability of human observers to discriminate between textures is re-

lated to the contrast between key structural elements and their repeating patterns.
Here we have developed an automatic texture classification approach based on
this principle. Local contrast information is modelled and a hybrid metric, based
on probability density distributions and transportation estimation, are used to
classify unseen samples. Quantitative and qualitative evaluation, based on mam-
mographic images and Wolfe classification, is presented and shows segmentation
results in line with the various classes.

1 Introduction
Texture is one of the least understood areas in computer vision. Although no generic
texture model has emerged so far a number of problem specific approaches have been
developed successfully [1,2,3,4]. More recently, approaches have been investigated
which aim to automatically determine a feature vector to be used for segmentation pur-
poses [5,6] or provide a more fundamental approach to texture segmentation [7,8,9].
The work described here can be seen as such a more generic approach towards tex-
ture modelling. The principles behind this modelling are based on the notion that human
observers are able to distinguish between textures if there is significant contrast differ-
ence between the main structural elements and the way those specific (sub-)structures
form a repeating pattern. To achieve this we have investigated the modelling of the dis-
tribution of texture structural elements within specific grey-level bands. Subsequently,
unseen texture regions can be compared with the developed models. The comparison
can be based on various distance metrics.
In this paper we consider the segmentation of texture information within mammo-
graphic images. Here the main aim is to distinguish between a number of textures that
appear in mammographic images (e.g. the various textures associated with Wolfe [10]
or Tabar [11] based risk assessment) and use the extracted information to obtain seg-
mentation of texture images. We have investigated the use of a Hybrid Metric which
can be regarded as the non-integer approximation of the transportation cost approach.
We provide both quantitative and qualitative assessment of the developed approach.
The layout of the paper is as follows. In Sec. 2 the local contrast based texture
segmentation approach is presented, which covers the extraction of the local contrast
information and the use of a novel Hybrid Metric to measure the similarity between

Susan M. Astley et al. (Eds.): IWDM 2006, LNCS 4046, pp. 433440, 2006.
c Springer-Verlag Berlin Heidelberg 2006
434 R. Zwiggelaar and E.R.E. Denton

distributions. In Sec. 3 quantitative and qualitative results based on real textures are
presented. The paper concludes with discussion and conclusions sections.

2 Methods
The aspects discussed in the section cover: a) the model describing the local contrast
structures, b) the way that these are used to provide texture models, and c) the approach
to evaluate the difference between local contrast structure models and new local areas
within unseen images.

2.1 Local Contrast Structures

One of the motivations behind this research is that texture recognition is driven by the
contrast between key structural elements. Images are decomposed into distinct grey-
level bands. The binary images, B(x, y), representing only distinct grey-level bands are
determined by

1 if low <= I(x, y) < high
B(x, y) = (1)
0 otherwise
where I(x, y) is a grey-level image, and low and high are low and high threshold
values.
It should be clear that the distinct structures that represent the textures are only
present in very specific grey-level bands and that a specific position within the image
can only become equal to 1 once if the high and low threshold values in Eq. 1 form a
non-overlapping series (as will be the case throughout the presented work).

2.2 Modelling
Modelling the repeating key (sub-)structures that are essential to describe textures can
be achieved by estimating local aspects using a set of binary images determined by
Eq. 1, where the set is based on a sequence of n (low , high ) values covering the full
range of grey-level values within the images and (high low ) is constant (e.g. a pos-
sible (low , high ) set would be {(0, 64), (64, 128), (128, 192), (192, 256)}, and some
binary images based on such sets can be found in Fig. 1). Once such a set of images has
been obtained a model of local structures needs to be obtained. To achieve this for a spe-
cific binary image in the set a region of interest (with size equal to (2w+1)(2w+1)) is
extracted at each position in B(x, y) with value equal to one. For each region of interest
the segment containing the central position is extracted using simple four-connectivity.
Using each position in the obtained segments provides a summation over B(x, y) re-
stricted by w. After normalisation with respect to total occurrence, this results in a
probability density distribution representing local structures within a specific grey-level
range (as specified by Eq. 1). Such a probability density is denoted as Pm (i, j), where
the subscript indicates the level in the set of binary images for which the probability
density is derived, m [1, n], n represents the number of grey-level bands, and (i, j)
covers the region of interest, i.e. i, j [w, w].
Subsequent to the modelling it becomes possible to determine if a new region of
interest extracted from an image that was not part of the modelling data belongs to the
 Texture Based Segmentation 435

Fig. 1. Local contrast structures, with (a) original images and the other images determined by
Eq. 1, where the (low , high ) values are (b): (64, 192), (c): (64, 128), (d): (128, 192), (e):
(64, 96), (f): (96, 128), (g): (128, 160), and (h): (160, 192)

modelled texture or not. The methodology is very similar to that described to obtain the
local contrast structures model, with the exception that in this case it is only based on
a single region of interest instead of all the relevant regions of interest within a whole
image. The new region of interest is only compared with the relevant model which
covers the grey-level value that occurs at the centre of the region of interest. Such a
single region of interest based texture description is denoted as Rm (i, j), where m
indicates the level in the set of grey-level values within which the grey-level value of
the centre position falls.

2.3 Hybrid Metric

The similarity between a new region of interest and the various local contrast tex-
ture models is determined by a Hybrid Metric, which is based on probability density
436 R. Zwiggelaar and E.R.E. Denton

modelling and the transportation metric [12,13,14]. It has been noted that the trans-
portation metric has distinct advantages [13], but at the same time some drawbacks
that ensure it is not generic enough to cover all applications [14]. Giannopoulos and
Veltkamp [14] introduced a proportional transportation distance to solve a number
of problems and provide a more generic distance metric which retained most of the
benefits of the classical transportation metric as described in previous work
[6,12,13,14,15,16,17].
The developed Hybrid Metric is to provide a weighted cost of the non-overlapping
regions of the local contrast structures model, Pm (i, j), and the region of interest,
Rm (i, j). This is given by


phm (x, y) = (Rm (i, j) min{Pm (i, j), Rm (i, j)})
(i,j) (s,t)
(Pm (s, t) min{Pm (s, t), Rm (s, t)}) (2)
1

k(i, j, s, t)

where min{., .} gives the minimum value of the two parameters and k(i, j, s, t) =
|i s| + |j t| is the cost for transportation between positions (i, j) and (s, t) in the
two images (or in general patterns) to be compared.
It should be noted that the distance underlying Eq. 2 is a semi-metric as it fulfills the
following properties: a) self-identity, b) positivity, and c) symmetry. It is likely to be a
full metric, but as to now we do not have a full proof that the triangle inequality holds.
However, this is only the case if the cost function k is a metric and the total quantity
at the source and destination are the same, which is similar to the restrictions for the
transportation distance being a metric.
An alternative way to regard this Hybrid Metric approach is as a non-integer approx-
imation of the transportation algorithm. Instead of distributing the sources as integer
quantities to the destinations, the Hybrid Metric approach distributes weighted values
from all sources to all destinations.

2.4 Texture Likelihood Estimation

The metric used to compare local contrast structure representations results in a similar-
ity estimation (see Eq. 2). The likelihood that pixels belong to a specific texture t are
determined by an odds-ratio:

phm (x, y)|nc =t

pthm (x, y) =  (3)
nc phm (x, y)

where nc indicates the number of texture models that are being considered. In the ex-
periments presented here combinations of four textures (representing the four Wolfe
classes) were considered and hence the value of nc {1, 2, 3, 4} .
 Texture Based Segmentation 437

3 Results
The evaluation involves the segmentation of images representing the various Wolfe
classes [10]. The database consists of sixty images. The images in the database were
all assigned a Wolfe classification by an expert radiologist and the distribution over
the four Wolfe classes are 0.25, 0.10, 0.55 and 0.10 for N1, P1, P2 and DY,
respectively.
An overview of example segmentation results can be found in Fig. 2.

Fig. 2. Example segmentation results, where from top to bottom the four Wolfe classes are
represented, with on the left the original mammograms and on the right the segmentation results

The segmentation results in Fig. 2 show a strong correlation between the various
texture regions in the mammographic images and the segmented areas in the resulting
images. However, there also seem to be some region boundary effects playing a sig-
nificant role, which warrant further investigation. In addition, it should be noted that
for most segmentation results the relative area is mainly occupied by two classes; there
438 R. Zwiggelaar and E.R.E. Denton

are a number of possible explanation for this, like the un-balanced distribution over the
four Wolfe classes, and this needs further investigation (it is interesting to note that for
BI-RADS breast density categories it is not uncommon to mainly use two out of the
four available classes [18,19]).

3.1 Mammographic Risk Assessment

The origins of breast density classification are the work of Wolfe [10], who showed the
relation between mammographic parenchymal patterns and the risk of developing breast
cancer, classifying the parenchymal patterns in four categories. Automatic assessment
of breast density/risk can be sub-divided into two groups, those that are just using grey-
level information [20] and those that incorporate texture information [21,22]. The de-
veloped approach falls into the latter category.
The segmentation results (see Fig. 2 for examples) are used to obtain the relative size
of the segmented regions for each class. This feature is used as our classification space
and the distribution over the Wolfe N1, P1, P2 and DY is represented in Fig. 3. This
shows a certain degree of clustering for mammograms belonging to the same Wolfe
class (represented in Fig. 3 by the four different markers), but at the same time there is
clear overlap between the classes and there are distinct outliers for some of the classes.

0.4 1
Relative Wolfe P1 Area

Relative Wolfe P2 Area

0.8
0.3
0.6
0.2
0.4
0.1
0.2

0 0
0 0.2 0.4 0.6 0.8 0 0.2 0.4 0.6 0.8
Relative Wolfe N1 Area Relative Wolfe N1 Area

Fig. 3. Distribution of relative Wolfe class areas within the mammographic images for the used
dataset, where on the left N1 versus P1 and on the right N1 versus P2 is shown. The four different
markers represent the ground truth Wolfe classifications.

The data as represented in Fig. 3 can be used as a 4D feature space. Based on a k near-
est neighbour classifier and a leave-one-woman-out methodology correct classification
results of up to 72% (with = 0.48) are obtained. However, the correct classification
tends to be based on only two of the Wolfe classes (those with a higher proportion in
the used dataset) and as such do not represent satisfactory results and indicate a need
for further investigation involving a larger dataset and a more even distribution of the
Wolfe classes (and alternative metrics like BIRADS) for the training data.

4 Discussion
The developed Hybrid Metric is a generic approach for the comparison of distributions
or patterns and as such will have a wide range of application areas. It would be of
 Texture Based Segmentation 439

interest to investigate the use of this metric in applications such as image retrieval,
image registration, pattern matching and general allocation cost estimation problems.
The developed approach to texture modelling might on the surface show similarities
to the local binary pattern (LBP) based texture analysis [9]. However, the main dif-
ferences are: a) grey-level bands are used to generate the binary images and hence the
models, b) modelling is based on the binary images and no histogram information is ex-
tracted to summarise the information, c) models at each grey-level band can be directly
compared, d) only one model per grey-level band exists, and e) the region of interest
tends to be an order of magnitude larger than typically used for LBP.
The developed approach does also show similarities to USAN [23]. The first step in
our process uses local binarised information based on grey-level bands. The grey-level
bands in SUSAN are taken as +/- around the central pixels grey-level value, whereas
here we use distinct bands independent of the grey-level value of the central pixel.
Further to this we would like to mention: a) SUSAN is measuring an area in the USAN
whilst here we use statistical modelling of the local area as a fundamental step, b)
the window size used here tends to be an order of magnitude larger than those used for
SUSAN, c) our approach does not only model the central region but all univalue regions
within the local area, and d) to our knowledge USAN or more advanced information like
used here has not been used in this fashion for texture segmentation.
To provide further evaluation on a full range of (w, n) values it might be essential
to develop a less algorithmic complex estimation of the transportation cost. The Hybrid
Metric based approach already shows significant improvements over the classical trans-
portation metric and this will be further investigated. Further might incorporate texture
clique aspects [24,25], where the central segment within the region of interest could be
sub-divided into a number of cliques.

5 Conclusions
We have investigated a novel texture segmentation methodology based on a concept
of local contrast structures. In the process we have developed a hybrid transporta-
tion/probability metric to compare distributions, which is a generic metric with potential
beyond the presented work. The evaluation on mammographic images shows overall
good segmentation results and limitations on the current results have been discussed.

References
1. M.W. Haralick. Statistical and structural approaches to texture. Proceedings of the IEEE,
67(5):786804, 1979.
2. R.W. Conners and C.A. Harlow. A theoretical comparison of texture algorithms. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 2(3):204222, 1980.
3. A.P. Pentland. Fractal-based description of natural scenes. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 6(6):661674, 1984.
4. T.R. Reed and J.M.H. Dubuf. A review of recent texture segmentation and feature-extraction
techniques. Computer Vision, Graphics and Image Processing, 57(3):359372, 1993.
5. C.C. Reyes-Aldasoro and A. Bhalerao. Volumetric texture description and discrimant feature
selection for MRI. Lecture Notes in Computer Science, 2732:282293, 2003.
440 R. Zwiggelaar and E.R.E. Denton

6. R. Zwiggelaar. Texture based segmentation: automatic selection of co-occurrence matrices.

In 17th IEEE International Conference on Pattern Recognition, pages 588591, 2004.
7. M. Varma and A. Zisserman. Classifying images of materials: achieving viewpoint and
illumination independence. Lecture Notes in Computer Science, 2352:255271, 2002.
8. M. Chantler and L. Van Gool. Special issue on texture analysis and synthesis. International
Journal of Computer Vision, 62:5, 2005.
9. M. Pietikainen. Image analysis with local binary patterns. Lecture Notes in Computer Sci-
ence, 3540:115118, 2005.
10. J.N Wolfe. Risk for breast cancer development determined by mammographic parenchymal
pattern. Cancer, 37(5):24862492, 1976.
11. I.T. Gram, E. Funkhouser, and L. Tabar. The tabar classification of mammographic parenchy-
mal patterns. European Journal of Radiology, 24(2):131136, 1997.
12. F.L. Hitchcock. The distribution of a product from several sources to numerous localities.
Journal of Mathematical Physics, 20:224230, 1941.
13. Y. Rubner, C. Tomasi, and L.J. Guibas. A metric for distributions with applications to image
databases. In 6th International Conference on Computer Vision, pages 5966, 1998.
14. P. Giannopoulos and R.C. Veltkamp. A pseudo-metric for weighted point sets. Lecture Notes
in Computer Science, 2352:715730, 2002.
15. A.S. Holmes, C.J. Rose, and C.J. Taylor. Measuring similarity between pixel signatures.
Image and Vision Computing, 20(5-6):331340, 2002.
16. S. Haker, L. Zhu, A. Tannenbaum, and S. Angenent. Optimal mass transport for registration
and warping. International Journal of Computer Vision, 60(3):225240, 2004.
17. F. Jing, M.J. Li, H.J. Zhang, and B. Zhang. An efficient and effective region-based image
retrieval framework. IEEE Transactions on Image Processing, 13(5):699709, 2004.
18. F.M. Hall. Mammographic density categories. American Journal of Radiology, 178:242
242, 2002.
19. L.A. Venta and R.E. Hendrick. Mammographic density categories - reply. American Journal
of Radiology, 178:242243, 2002.
20. N. Karssemeijer. Automated classification of parenchymal patterns in mammograms. Phys.
Med. Biol., 43:365378, 1998.
21. J.W. Byng, M.J. Yaffe, G.A. Lockwood, L.E. Little, D.L. Tritchler, and N.F. Boyd. Auto-
mated analysis of mammographic densities and breast carcinoma risk. Cancer, 80(1):6674,
1997.
22. R. Zwiggelaar and E.R.E. Denton. Optimal segmentation of mammographic images. In 7th
International Workshop on Digital Mammography, to be published, 2004.
23. S.M. Smith and J.M. Brady. Susan a new approach to low level image processing. Inter-
national Journal of Computer Vision, 23(1):4578, 1997.
24. S. Geman and D. Geman. Stochastic relaxation, Gibbs distribution, and the Bayesian restora-
tion of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6:721741,
1984.
25. G.L. Gimelfarb. Texture modeling by multiple pairwise pixel interactions. IEEE Transac-
tions on Pattern Analysis and Machine Intelligence, 18:11101114, 1996.