Skeletonization by Gradient Regularization and Diffusion

Skeletonization by Gradient Regularization and diffusion F. Le Bourgeois H. Emptoz LIRIS INSA de Lyon, Villeurbanne France {Franck.lebourgeois,Hubert.emptoz}@liris.cnrs.fr Abstract This paper describes a skeletonization process for grayscale or color images based on the diffusion of the color gradient vectors by using a simple iterative regularization scheme. We propose to diffuse the original color gradient vectors to obtain the skeleton of the main contrasted objects. Contrary to a distance transform or thinning based skeleton, the gradient vectors diffusion is a straightforward, simple and efficient approach to compute the skeleton, which does not require the localization of the contours curvatures or a distance map. In comparison to other approaches based on potential field functions, our method does not require the segmentation of the objects or the precise localization of the contours. Our approach is very simple to implement and can be applied to natural noisy color images of old documents. 1. Introduction Skeletons are widely used in many computer vision applications and in particular in Document Image Analysis. They provide a simple and compact representation of shapes that preserves the topology of the objects. Images of documents are generally made of complex thin lines, which represent the ink of the drawings or printings. The skeletonization of documents images preserves the patterns and simplifies the extraction of features. Skeletonisation is widely used in Optical Character Recognition, document images vectorization and coding. In computer vision, there are many skeletonization algorithms suited for binary images but there are a reduced number of methods, which use grayscale or color information. Recently, many digital Libraries provide numerous color images of old manuscripts, which cannot be skeletonized in binary. Moreover, these documents show many defects (ink fading, ink bleed trough, complex background due to the paper texture, lack of regularity of the lines…), which make difficult their skeletonization. Our objective consist to develop a robust skeletonizition method, suited for old degraded documents images digitized directly in colors, which can tolerate the natural noise and the documents defects. 2. Previous works The skeleton can be defined in several ways either by the centers of the maximal disks contained in the original object [1] or by the intuitive grassfire paradigm introduced by Blum [2]. Morphology introduced by Serra [1] provides a well-founded theory to define and compute skeletons of binary images and it was later extended to grayscale images [3]. 2.1. Skeleton of binary images There are mainly four different approaches to compute binary images’ skeletons of delimited objects by their area or contour : 1) methods based on thinning or grassfire approach They are generally computed by iterative conditional thinning which iteratively deletes the nonskeleton points [4]. Heuristics and complex criteria are often used to stop the process and preserve the skeleton continuity and width . 2) The distance transform The distance transform is defined for each point of an object as the smallest distance from that point to the boundary of the object [5]. Skeletons and medial lines of objects can be computed by finding the local maxima of the distance map. The object can be entirely reconstructed by replacing each point of the skeleton by a discrete disc with a radius given by the distance transform. 3) Geometric methods based on voronoi diagram. The skeleton is computed from the voronoi graph of a set of points located on the boundary of the object [6]. This approach is theoretically well defined in a continuous space and provides fully connected skeletons. The main drawbacks of this approach are the sampling of the boundaries which defines the quality of the Voronoi diagram and the pruning of branches by using complex post-processing stages. 4) Methods based on Potential Field functions Skeletonization approaches based on potential fields for 2D and 3D objects identify skeleton by using a potential model instead of the distance transform. The pixels of the boundary are considered point charges generating the potential field inside the object using an electrostatic field function [7] or the Newton law [8]. But these approaches have some drawbacks such as the necessity to consider the distance to the border, or to verify the visibility of each point to the border or the localization of the curve curvatures. Generalized Potential field function has been used for 2D objects [9] and 3D objects [10]. For this approach, the potential at a point interior to the object is determined as a sum of potentials generated by the boundary of the object. This approach gives much smoother skeletons which are less sensitive to the noise. However, this approach requires the segmentation of the contour of the objects to skeletonize. In conclusion, most of the previous methods lead to difficult problems due to the nature of the discrete space. Most of previous methods are sensitive to noise and complex pruning operation must be applied to clean the spurious branches of the skeleton. Nevertheless, the approaches based on generalized potential fields are interesting for noisy images and could be extended to grayscale or color images. 2.2. Skeleton of grayscale or color images There is limited number of papers about the skeletonization using directly grayscale information without image segmentation [11]. The recent trend is using potential fields approaches by analogy either with electromagnetism or Newton law. These approaches seek to diffuse potential fields using various diffusion equations in order to define an edge strength function (also called skeleton strength function or pseudo distance map) [12][14][15][16]. The skeleton is extracted by tracking the ridges of this edge strength map. An homothetic erosion of the grayscale image is also used to extract skeletons [13]. However, this approach is applicable on images characterized by objects brighter or darker than the background. The skeletons from grayscale images are interesting for many applications and are robust to image noise. But those authors are focused on the physics theory (Newton's law of universal gravitation or Electromagnetism laws) that defines the underlying mathematical model of vector diffusion. We propose to simply abandon the physics framework and to develop freely another simple skeletonization algorithm which uses directly the diffusion of the image gradients from grayscale or color images by a progressive regularization. 3. Proposition Our objectives consist to find a sketonization approach suited for noisy images, which uses color or grayscale information. Our contribution can be classified into potential field approaches. We have fixed four objectives to reach : • To use of a simple diffusion framework (avoid heuristics, parameters, masks and selection of points) • To initialize the potential field with the gradients of the images without the selection of particular points of the object or its contour. • To be less sensitive to image noise in order to process natural digitized documents • To skeletonize directly grayscale or color images of large documents images with a reasonable computational cost. In order to achieve these goals, we need to make some assumptions. First of all, we assume that the contour of the objects to skeletonize has a higher gradient magnitude than in the background or the interior of the object. Secondly we assume that the gradients keep almost of the same magnitude along the contour of the same object. These assumptions are not restrictive for documents images. 3.1. Diffusion of gradient vectors We propose to use the gradients of the entire image as the initialization of a potential field. In a second step, we diffuse the gradient vector field by using a regularization process which preserves the divergence of the gradient vector field around the skeleton of the objects. We notice that in most analytic formula of vector diffusion process, a regularization of the vector field is achieved. We find this regularization step in the isotropic or anisotropic vector diffusion or during the calculation of motion vectors for optical flow. We propose to apply directly the regularization of the gradient vector field as a diffusion process without using any analogy to physics laws. If we consider a 1D signal (fig. 1), its derivatives cross zero for local extrema, which correspond exactly to both the skeleton SQ(I) and its complementary SQ(I). Figure 1 also shows the diffusion of gradient information into flat zones where the gradient is null or low by using a simple regularization process (1). The gradient vector field will flow into flat zones and the opposite vectors indicate the location of the skeleton and its complementary. The vector regularization (1) can be used to diffuse the gradient vectors into flat zones without computing distance map or singular points of the contours around high curvatures like for the Hamilton Jacobi skeletons [8] or potential fields approaches. (1) ∇I n +1 ( p ) = 1 N ( p) ∇I n (h ) with ∇I = h∈N ( P ) Ix Iy The regularization can be considered as an iterative smoothing of the gradient vectors using the 8connected neighbors N of each point p, with a uniform kernel. 3.2 Skeleton from the diffused gradient vectors There are several approaches to extract the skeleton from a diffused gradient vector field. Originally, the skeleton is located along lines where the divergence of the potential field is null. We choose to compute the maximal angle difference of the orientation of two adjacent symmetric pair of gradient vectors in a 3X3 neighborhood and we keep this value in a Skeleton Strength map SS (2). ( 2) SS ( p ) = Max h , k ∈N ( p ) h , k symmetric pair θ ( h) − θ (k ) where θ(h) and θ(k) are the gradient orientations of the symetric pair of adjacent gradient vectors in a 8connected neighborhood. If the Skeleton Strength reaches a maximal value of 180°, then the point belong to the skeleton of a line shape. If the skeleton strength decreases around 45° then the point belong either to the skeleton of a triangle shape or to spurious skeleton branches. Figure 2 gives the intermediate results of the skeletonization process, from the initial gradient orientation, their diffusion with different values of smoothness σ, the associated Strength Skeleton map. Figure 1 Gradient vectors diffusion by regularization. But the excessive regularization of the gradient vector field will progressively diverge. Points which are not the center of exact opposite gradients of the same magnitude will progressively translate and disappear. To keep the stability of the gradient field by vector regularization, we need either to reduce the number of iterations or to use a grassfire algorithm or a stop function to block the diffusion before reaching the loss of stability of the field. We choose to control precisely the regularization by stopping the flow (1) when the number of iteration n for non null gradient ∇In≠0 reach a user defined limit σ. The number of iteration for null gradient around flat zones must be unlimited. If σ=1 we have a classical grassfire algorithm which provides a sharp skeleton with spurious branches. For a higher value of σ, we obtain a smooth skeleton, robust to noise and suited for real images (fig. 2). For images of documents, the lines are enough thin to allow a reduced number of iterations to fill flat zones. Hence, we can increase the smoothness parameter σ to process particular noisy images without reaching the loss of stability of the gradient field by using the regularization as vector diffusion. Figure 2 From left to right and from top to bottom : original image, initial gradient orientation θ(∇In=0), diffusion with σ=1 θ(∇In=8), stable field θ(∇In=43), Strength Skeleton (SS), superposition on the original image, stable field with σ=40 θ(∇In=82), superposition on the image, skeleleton for degraded objects (σ=40), color legend for gradient orientation. With a large value of smoothness σ, the skeleton remains stable to the degradations of the contours. As we do not use the segmentation of the objects from the background, the differentiation between the SQ and the complementary SQ (skeleton of the background) must use the convergence or the divergence of the gradient field. As the gradient flows from dark areas toward brighter part of the image, SQ corresponds to a divergent gradient field and SQ to a convergent gradient field (Fig 3). Figure 3 SQ : Convergent field SQ: divergent field 3.3. Extension to grayscale and color images We use all the gradients of the image even for pixels from flat areas where the gradient orientation is not well defined. We have noticed that gradient vectors which are individually not correctly oriented around flat zones are statistically “on average” almost well oriented. Our key idea is that the apparent disorder of the gradient orientation around flat zones can be reoriented by the regularization of the gradient vector field by using (1). Fig. 4 shows the gradients field from a noisy color image and the field obtained after 100 iterations of the regularization. By using the regularization as a diffusion process, the gradient vectors having higher magnitude will reorient correctly the gradients of nearly flat zones. Color Gradient ∇I can be represented by the Jacobian matrix J of 3x2 elements. The problem consists to approximate the direction θ(∇I) and the magnitude |||∇I |||. Di Zenzo computes the magnitude and orientation of color gradient vectors with an eigen decomposition of the 2x2 matrix calculated from the inner product of the Jacobian matrix tJ J [17]. For the skeletonization, we introduce another color gradient approximation (3) based on the maximum value. Our objective is to represent the gradient vectors by the same expression ∇I=t(Ix,Iy) for color, grayscale or binary images. We define the components Ix, Iy of the color gradient vector∇I by taking the maximum absolute value of partial derivatives for each channel (3), but the sign of the derivatives is preserved in Ix, Iy (see Cmax function). The direction is given by (4) and gradient magnitude by (5). Iy = C max ∂R ∂ V ∂ B , , ∂y ∂y ∂y { Ix = C max C max {g k } = g k / g k > g p ∀ k ≠ p θ (∇ I ) = arctg Iy Ix (4) ∂R ∂V ∂B , , ∂x ∂x ∂ x } (3) ∇I = Max { Ix , Iy } (5) Our color gradient approximation is easier to implement, faster to compute for large images. The sensitivity to the smallest variation for all channels and directions is appropriate for documents images with large flat zones. Figure 5 describes the final skeletonization algorithm. Let ∇I0 the original gradient vector using (3) n=0 For each point p of the image do B(p)=0 repeat Stability =0 For each point p of the image do If (B(p) <σ) or (||∇I||=0) then ∇M=0 // regularize & diffuse by using (1) For each neighbors h of p / d8(h,p)=1 do Figure 4 Initial color gradients and their regularization We now explain how to compute a color gradient magnitude and orientation. Let a color image be a mapping I defined by : ∂R ∂R I : IN 2 → IN 3 ( x, y ) (R( x, y ),V ( x, y ), B( x, y )) ∂x ∂V ∇I = ∂x ∂B ∂x ∂y ∂V ∂y ∂B ∂y ∇M=∇M+∇I0 (h) ∇M=∇M /8 if (∇M≠0) then Stability+=|∇I1 (p)- ∇M| ∇I1 (p)= ∇M B(p)++; For each point p do ∇I0 (p) = ∇I1 (p) n++ Until (n==nitermax) or (stability==0) For each point p do Compute θ (p) by using (4) For each point p do Compute SS (p) by using (2) Figure 5 Algorithm of the skeletonisation scheme. 4. Results We apply our skeletonization on several documents images with σ=40 on average. Our approach provides smooth skeletons suited to noisy images and object defects (fig. 6). The proposed method does not require the object segmentation and need a reduced number of iterations. The smooth skeleton probably does not respect all the desirable properties (homotopy, full connected, invariance to isometric transformation, thinness, centered and exact location) but it can be useful for feature extraction or vectorization. Figure 6 Results on grayscale or color images. 5. Conclusion We have presented a simple skeletonization approach, which uses the gradient vector regularization to simultaneously diffuse gradient information into flat areas and smooth the gradient vector disorder due to the image noise. The diffusion process is simple and requires no computation of any distance map. After several iterations of the regularization step, the gradient vectors flow toward the image skeletons. This simple method can be applied directly to grayscale or color natural images of documents. It requires just a parameter σ which controls the smoothness of the skeleton. We are going to establish the link between our approach and methods based on potential fields, which require the computation of a distance map or the segmentation of the contour of the objects to skeletonize. 6. References [1] J. Serra, Ed., Image Analysis and Mathematical Morphology. New York: Academic, 1982. [2] H. Blum, R. N. Nagel, “Shape description using weighted symmetric axis features”, Pattern Recognition, no. 10, 1978, pp. 167-180. [3] S. R. Sternberg, “Grayscale morphology,” CVGIP, vol. 35, no. 3, pp. 333–355, Sept. 1986. [4] L. Lam, S. W. Lee, C. Y. Suen, “Thinning Methodologies, a Comprehensive Survey”, IEEE PAMI, Vol. 14, No. 9, pp. 869-885, September 1992. [5] G. Borgefors. “Distance Transformations in Digital Images” Computer Vision, Graphics, and Image Processing, Vol. 34, pp. 344-371, 1986. [6] D.Attali. “Squelettes et graphes de Voronoi 2D et 3D” . PhD thesis, Université Joseph FOURIER, Grenoble I, 1995. [7] T. Grigorishin, G. Abdel-Hamid and Y.-H. Yang, “Skeletonisation: An Electrostatic Field Based Approach”, Pattern Analysis & Applications, vol 1, pp. 163-177, 1998. [8] K. Siddiqi and al. “Hamilton-Jacobi skeletons”, Int. Journal of Computer Vision, 3:215–231, 2002. [9] Ahuja, N. and Chuang, J.-H. “Shape Representation using a Generalized Potential Field Model”, IEEE PAMI, Vol. 19, No. 2, pp.169-176, February 1997. [10] J. H. Chuang, C. H.,Tsai, M. C. Ko, “Skeletonization of Three-Dimensional Object Using Generalized Potential Field”, IEEE PAMI, Vol. 22, No. 11, pp. 1241-1251, November 2000. [11] S.W. Lee, Y. J. Kim, Direct Extraction of Topographic Features for grayscale character Recognition, IEEE PAMI, vol 17, n° 7, july 1995, pp 724-729 [12] Z. S. G. Tari, J. Shah, and H. Pien, “Extraction of shape skeletons from grayscale images”, Computer Vision and Image Understanding, 66:133–146, 1997. [13] D H. Chung and G. Sapiro, “Segmentation-Free Skeletonization of Gray-Scale Images via PDE”, in proc. Int Conference on Image Processing, Vol. 2, pp 927-930, 2000. [14] Jeong-Hun Jang, Ki-Sang Hong, “A Pseudo-Distance Map for the Segmentation-Free Skeletonization of GrayScale Images”, in Proc. of the International Conference On Computer Vision (ICCV-01), Vol. 2, pp. 18-25, 2001. [15] Zeyun Yu, Chandrajit L. Bajaj, “A Segmentation-Free Approach for Skeletonization of Gray-Scale Images via Anisotropic Vector Diffusion”, in proc. of CVPR’04, vol. 1, 2004, pp. 415-420. [16] Huaijun Qiu, E. R. Hancock, “Grey Scale Image Skeletonisation from Noise-Damped Vector Potential”, In Proc. of ICPR'04, Volume 2 pp. 839-842, 2004. [17] S. di Zenzo, Gradient of multi-Images, CVGIP, 33 pp. 116-125, 1986.