Painting Crack Elimination Using Viscous Morphological Reconstruction

Allan Hanbury, Paul Kammerer and Ernestine Zolda

Pattern Recognition and Image Processing Group
Vienna University of Technology
Favoritenstraße 9/1832, A-1040 Vienna, Austria
Hanbury, Paul, Zolda
@prip.tuwien.ac.at

Abstract During the aging of the paintings, climactic fluctuations

cause changes in the dimensionality of the panels. While
A method for the elimination of cracks in infrared re- younger pigment layers are elastic enough to follow con-
flectograms is presented. Infrared reflectograms show the tractions, a network of fine cracks (craquelé) may cover the
underdrawing – the basic concept of the artist drawn on the whole painting during the aging process. The pattern of the
ground layer – in ancient wood panel paintings. Caused by cracks is determined by the background used. In the case
different mechanical behaviour of the panel and the cover- of wooden panels, the cracks are primarily oriented perpen-
ing ground layer during the aging of the painting, cracks dicular to the grain [6]. Willingen et al. [3] have studied
appear. Cracks not only disturb the appearance of a paint- the appearance of cracks and determined features to clas-
ing, but are also a problem for segmenting the lines and sify different types of cracks. They differentiate between
strokes of the underdrawing. A method based on mathe- features of individual cracks (smooth, jagged, depth, thick-
matical morphology, namely morphological reconstruction ness etc.) and features of crack patterns (distance between
with viscosity, is used to eliminate the cracks while keeping cracks, type of junctions).
as much detail as possible in the brush strokes. A priori in- One major goal of the project is to identify the drawing
formation that the cracks are usually thinner than the brush tools used by the painter to create the underdrawing from
strokes and have a preferred orientation is taken into ac- the appearance of the strokes in the infrared reflectogram.
count. A step toward the identification is the segmentation of the
individual strokes. From the segmentation point of view,
cracks are treated as structural noise and will produce arte-
1. Introduction facts in the segmentation step. To overcome this problem,
our intention is to eliminate the cracks while keeping the
Computer aided analysis can be an important tool for the boundaries of the strokes as accurately as possible for fur-
examination of works of art. An interdisciplinary project1 ther analysis.
between the fields of art history and image analysis is de- A similar problem has been treated by Giakoumis and
veloping a system to investigate infrared images (infrared Pitas [4]. They use a three step process which first detects
reflectograms [2]) of medieval and Renaissance panel paint- the cracks using a top-hat operator, separates them from
ings with methods of digital image processing and analysis. brush strokes using colour information, and then fills them
The infrared reflectography technique allows the pene- in. In contrast to this work, we are working on greyscale im-
tration of the covering paint layers and the visualisation of ages (recorded in the infra-red region). We therefore have
the underdrawing, the basic concept of a painter. The im- no colour information for separating the cracks from the
age depicted in Figure 1 gives an example of an underdraw- brush strokes. The information we start with is that the
ing taken from a panel painting by an Austrian medieval cracks are usually thinner than the brush strokes, and that
painter. The sleeve region, covered by red pigments, could they have a favoured orientation. To take this information
be penetrated by a CCD-camera in the spectral range be- into account, we make use of a morphological opening with
tween 700nm – 900nm. It shows the formation of strokes a viscous reconstruction step, which detects the cracks and
that sketch the crinkles in the top layer of the painting. fills them in in one step. Abas and Martinez [1], on the other
 This work was supported by the Austrian Science Foundation (FWF) hand, are interested in the structure of the crack network.
under grants P15471-MAT, P14445-MAT and P14662-INF. They use a top-hat operator to extract the cracks, and then
1 http://www.prip.tuwien.ac.at/Research/Casandra/index.html extract descriptive information about the crack network so

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.
 is
Æ

´½µ


 Æ

´½µ



(1)
´½µ
where Æ

indicates a dilation of image by the struc-
turing element  . The reconstruction by dilation of mask

from marker , defined as the iteration of the geodesic di-
lation of marker with respect to mask
until stability, is
denoted
´µ



 Æ


(2)
´ µ ´·½µ
where  is chosen such that Æ


 Æ


. In
general, the structuring element  is chosen to be the unit
structuring element.
Viscous morphological reconstruction was introduced to
solve a problem similar to the one shown in Figure 2. Fig-
ure 2a shows an area marked out by black circles on a white
foreground. We wish to expand the marker shown in Fig-
ure 2b so that it fills out the region surrounded by the black
circles. Obviously, a standard morphological reconstruction
using Figure 2a as the mask and Figure 2b as the marker
would produce Figure 2a as the result. The notion of vis-
cosity was therefore introduced into the reconstruction pro-
cess.
In order to incorporate viscosity, one performs an open-
ing on the marker image after each geodesic dilation step.
The viscous geodesic dilation of marker with respect to
Figure 1. Infra-red image of medieval painting mask
is defined as
taken in the near IR-range. The black rectan-

´½µ ´½µ
gles show the subregions used in figures 4 






 

Æ


(3)
and 5.
where  is, as before, the structuring element used for the
geodesic dilation, and  is the structuring element used
as to classify it. for the opening. As in equation 2, viscous reconstruction by
In section 2, we present the theory behind the viscous dilation is defined as
morphological reconstruction operator. The results of this  ´ µ





 




(4)
operator applied to removing cracks from infra-red images
of paintings are given in section 3. ´µ ´·½µ
where  is chosen such that 




 




.
As a demonstration of viscous reconstruction, we recon-
2. Viscous Morphological Reconstruction struct the marker in Figure 2b inside the mask in Figure 2a,
using a disc of radius  for both structuring elements  and
Viscous morphological reconstruction was introduced by  . For values of    pixels, one obtains the initial mask
Serra [7], who formulates it in a mathematically rigorous (Figure 2a) as a result, as the spacing between at least one
way in terms of viscous lattices. In this section, we give pair of points is wider than  pixels. When    pixels,
a brief application-oriented introduction to viscous recon- the result shown in Figure 2c is obtained. Here, the recon-
struction, beginning with an overview of standard morpho- struction has been constrained to lie inside the surround-
logical reconstruction [8]. ing points. This corresponds to the largest reconstructed
With standard morphological reconstruction, one recon- region constrained by the points. If the radius is further in-
structs a mask image from a marker image by iterating creased (i.e.   pixels), then the reconstruction cannot
geodesic dilations of the marker image inside the mask im- penetrate into the narrower parts between the points, as is
age until stability. We denote by the marker image and by seen in Figure 2d, which shows a reconstruction for a disc

the mask image. Both these images have the same size, with    pixels. Serra [7] demonstrates a method for
and
. The geodesic dilation with structuring element obtaining a line which approximates the distribution of the
 of the marker image with respect to the mask image
points. One calculates the maximum reconstruction within

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.
 (a) mask (b) marker (a) mask (b) standard opening

(c) reconstruction with disc of ra- (d) reconstruction with disc of (c) marker image (d) viscous reconstruction
dius 24 pixels radius 32 pixels

Figure 3. (a) Initial image. (b) Opening of

Figure 2. Viscous morphological reconstruc- image (a) with a disc of radius 5 pixels. (c)
tion of the mask (a) from the marker (b) using: Marker image obtained by eroding image (a)
(c) a disc of radius 24 pixels and (d) a disc of with a disc of radius 5 pixels. (d) Viscous
radius 32 pixels for both structuring elements reconstruction for mask (a) from marker (c).
and
. Images are of size 256 by 256 pix- Images are of size 256 by 256 pixels.
els.

the points (as done above) and the maximum reconstruction thick line, and would therefore be reconstructed too.
starting from a marker lying outside the points. The median The use of viscous reconstruction is a good solution to
line of the intersection of these two reconstructions gives a this problem. We begin by creating the marker image shown
good representation of the outline of the area sketched out in Figure 3c by eroding Figure 3a by a disc of radius  pix-
by the points. els. To reconstruct the initial image, we use equation 4, with
In the context of this paper, we are interested in the abil- a    pixel square, and
a disc of radius  pixels.
ity of the viscous reconstruction to reconstruct small details In order to reconstruct the small details, we append an ex-
while preventing certain elements from being reconstructed. tra geodesic dilation onto the reconstruction algorithm. The
The challenges faced in the art history application are illus- result of this reconstruction is shown in Figure 3d.
trated schematically in the simple binary example shown It remains to discuss some practical details of the use of
in Figure 3a. In this image, we wish to preserve the thick viscous reconstruction. The first is the choice of the struc-
line and all its details as accurately as possible, while re- turing elements and
. If one wants to ensure that all
moving the thin lines which intersect it. The thin lines are regions in the marker image are used as seeds for the re-
known to have a diameter of less than
pixels, but inter- construction process, then should be the same size as
sections can sometimes result in thicker regions. An open-
, as in the first example discussed. Alternatively, if
ing of Figure 3a with a disc-shaped structuring element of is set to the unit structuring element (or any structuring el-
radius  is shown in Figure 3b. As expected, the details on ement smaller than
), then any regions in the marker im-
the thick line have been smoothed, however, not all the thin age which are smaller than
after the first geodesic dila-
lines have been successfully removed. Using a larger struc- tion with will be eliminated. This is potentially useful,
turing element would smooth the thick line even more, and as it allows less severe filters to be used in the creation of
reconstruction cannot be used as the thin lines intersect the the marker image. This is demonstrated in the example of

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.
 Figure 3, where even though not all the thin lines are elim- (Figure 4b) is the additional opening performed during the
inated in the creation of the marker, they still do not appear reconstruction process. After one unit dilation, the cracks
in the result. The second implementation detail to discuss remaining in the marker image get eliminated by this open-
is the addition of an extra geodesic dilation at the end of the ing, as they are not wide enough to survive.
reconstruction procedure. For the first example, this is not Due to this extra ability of the reconstruction process
recommended as one aims to keep the reconstructed area to remove cracks, it is not necessary to use such a large
inside the markers, and the extra geodesic dilation step will structuring element during the initial erosion. We there-
certainly expand it further into the exterior region. For the fore decided to take some extra a priori information into
second example, where we wish to preserve small details on account during this erosion, namely that a large majority of
the thick line being reconstructed, this additional geodesic the cracks have a preferred orientation, as discussed in the
dilation is useful. introduction. For the image under consideration, this pre-
Having demonstrated the use of viscous reconstruction ferred orientation is horizontal. We therefore take as our
operators on binary images, we now move onto their appli- marker image an erosion of the initial image by a vertical
cation to the removal of thin cracks in greyscale images. line of length  pixels, shown in Figure 4e. The viscous
reconstruction from this marker image, using a

pixel
3. Application to the removal of cracks square for
, and a disc-shaped structuring element of ra-
dius for , is shown in Figure 4f. It is clear that while the
cracks are eliminated as efficiently as before, more structure
We present a comparison between various mathematical
remains in the strokes.
morphology based methods of removing cracks from infra-
Potential problems with this method arise when brush
red images of paintings. For this example, we use Figure 4a,
strokes are of a similar thickness to the cracks. In contrast
which corresponds to the lower sub-region indicated in Fig-
to the work in [4], we do not have any colour information
ure 1. Note that all morphological operations in this section
allowing us to distinguish between the two. Further infor-
are performed on the inverse of Figure 4a so that the objects
mation, such as the smoothness of the lines, will potentially
to be removed are lighter than the background, as this sim-
have to be taken into account in difficult cases. An exam-
plifies the use of the reconstruction operator. The resulting
ple of such a difficult case is shown in Figure 5a, the upper
images shown have been inverted again.
sub-region indicated in Figure 1. The same parameters were
An obvious way of removing the cracks which are
used to calculate the viscous reconstruction shown in Fig-
smaller than a given size is to use a morphological open-
ure 5b as for the previous example, that is, the marker was
ing with a disc which has a diameter larger than the cracks,
created by eroding the initial image with a vertical line of
but smaller than the brush strokes. The result of such an
length  pixels, and the viscous reconstruction was done
with a

pixel square for
, and a disc of radius
opening with a disc of radius is shown in Figure 4b. This
approach has the disadvantage of leaving a number of arte-
for . In the reconstruction, many of the light, thin brush
facts corresponding to the shape of the structuring element,
strokes in the complicated left half of the image are also
and of smoothing the boundaries of the strokes. The stan-
eliminated.
dard way of keeping the strokes as similar to the original as
possible is to reconstruct the original image (mask) from an
image eroded by the same disc used before (marker), which 4. Conclusion
is shown in Figure 4c. In this case, however, the cracks are
reconstructed too, as they have a similar greylevel and are We have demonstrated the application of viscous mor-
connected to the brush strokes. phological reconstruction to an art history problem dealing
In order to avoid reconstructing the narrow cracks, re- with the analysis of infra-red reflectograms of ancient panel
construction with viscosity can be used, with the structur- paintings. In these reflectograms, it is required that the thin
ing element used to do the opening ( in equation 4) set to lines (the cracks) be eliminated, while retaining as much de-
a disc slightly larger than the cracks. Figure 4d shows the tail as possible in the thicker lines (the brush strokes). The
result of this viscous reconstruction, which uses an erosion greylevels of the cracks and the brush strokes are similar,
of the (inverted) initial image by a disc of radius pixels as and the fact that they intersect prohibits the use of stan-
the marker. It is clear that this image contains fewer back- dard morphological reconstruction. Other available a pri-
ground artefacts than the image resulting from a standard ori information which has been taken into account is that
opening (Figure 4b). For the viscous reconstruction, struc- the cracks have a preferred orientation due to the mechani-
turing element
is set to a

square, and to a disc- cal properties of the wood on which the painting was done.
shaped structuring element of the same size as for the initial The suggested approach functions well except in more com-
erosion ( pixels). The main reason for the disappearance of plicated regions of a painting where the brush strokes have
the cracks which were still visible after the standard opening a similar width to the cracks. Further work on separating

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.
 (a) initial image (b) standard opening (c) ordinary reconstruction from ero-
sion by disc SE

(d) viscous reconstruction from ero- (e) line erosion (f) viscous reconstruction from (e)
sion by disc SE

Figure 4. (a) Initial image. (b) Standard opening of (a) with a disc of radius 6 pixels. (c) Reconstruction
of (a) using as marker an erosion of (a) by a disc of radius 6 pixels. (d) Viscous reconstruction of (a)
from the marker used for image (c). (e) Erosion of (a) by a vertical line of length 10 pixels. (f) Viscous
reconstruction of (a) using (e) as a marker.

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.
 strokes and cracks based on their smoothness remains to be
done.
Further applications of viscous morphological recon-
struction exist. It has been used, for example, in determin-
ing a poorly-visible outline of a heart muscle in a positron
image [7]. In the context in which it has been applied here,
it could be interesting to apply it to scratch removal in mo-
tion picture restoration. Such scratches have a similar ori-
entation specificity to the cracks on paintings, being always
vertically oriented with respect to the images [5].

Acknowledgements

Thank you to Thomas Walter of the Centre de Morpholo-

gie Mathématique, Ecole des Mines de Paris, France, for
help in the implementation of the viscous reconstruction op-
erator.

References

[1] F. S. Abas and K. Martinez. Classification of painting cracks

for content-based analysis. In Proceedings of IS&T/SPIE’s
(a) initial image 15th Annual Symposium on Electronic Imaging: Machine Vi-
sion Applications in Industrial Inspection XI, 2003.
[2] J. V. A. de Boer. Infrared Reflectography. - A Contribution to
the Examination of Earlier European Paintings. PhD thesis,
Univ. Amsterdam, 1970.
[3] P. de Willigen. A mathematical study on craquelure and other
mechanical damage in paintings. Technical report, Delft Uni-
versity of Technology, Faculty of Information Technology
and Systems, Department of Mathematics and Computer Sci-
ence, 1999.
[4] I. Giakoumis and I. Pitas. Digital restoration of painting
cracks. In Proceedings of the IEEE Int. Symposium on Cir-
cuits and Systems (ISCAS ’98), 1998.
[5] L. Joyeux, O. Buisson, B. Besserer, and S. Boukir. Detec-
tion and removal of line scratches in motion picture films. In
Proceeding of the IEEE Conference on Computer Vision and
Pattern Recognition, volume 1, pages 548–553, 1999.
[6] F. Mairinger. Strahlenuntersuchung an Kunstwerken. E. A.
Seemann, Berlin, 2003.
[7] J. Serra. Les treillis visqueux. Technical Report N-
51/99/MM, CMM, Ecole des Mines de Paris, 1999.
[8] P. Soille. Morphological Image Analysis: Principles and Ap-
plications. Springer-Verlag, 1999.

(b) viscous reconstruction

Figure 5. (a) A difficult image from which to

remove cracks. (b) A reconstruction with vis-
cosity of image (a).

Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP’03)
0-7695-1948-2/03 $17.00 © 2003 IEEE

Authorized licensed use limited to: Universitatsbibliothek der TU Wien. Downloaded on January 6, 2009 at 15:54 from IEEE Xplore. Restrictions apply.