Image Segmentation - Region Growing Algorithm

Pi19404
March 1, 2013

Contents

Contents

2 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

0.1 Introduction
In this document we will look at task of image segmentation and use region growing algorithms to achieve this task and its application to underwater image segmentation.

0.2 Image Segmentation

The goal of image segmentation is to partition the image into subregions with homogeneous intensity (color or tex- ture) properties which will hopefully correspond to objects or object parts. Region growing algorithms is a method to extract similar parts of image. The region growing algorithms checks for similarity of the neighborhood pixels of given initial seed point and labels the pixels that meet the similarity criteria and mean value of region foor similarity evaluation is determined. The neighborhood pixels of the newly labelled points are analysed and similarity comparison is performed with newly computed mean value. In this way neighborhood can grow in a iterative manner till stopping criteria is met.

0.3 Region Growing Algorithm

Conider a set R which contains pixels that belongs to the region that is required to be segmented.Let set S represent a set of pixels called seed points. Seed points are pixels that is known to belong to object desired to be segmented.

3 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

For the present application we assume that we need identify the region in the image to which seed pixels belong ie other region of the image are not required. For every seed points 4/8 connected neighbors of seed point are analyzed for similarity with the seed pixel using some predefined criteria. Thus all the neighborhood pixels are marked to be compared with seed pixel. If the connected neighbor satisfies the criteria then it is included labelled as desired region and neighbor is unmarked. While all the connected neighbors of the present neighbor are marked. This process continues till no marked pixels remains which results in segmentation of region to which seed pixel belongs. As the region grows the similarity comparision can be peformed with the mean value of region rather than just the seed pixels.

0.4 4/8 conneced neighbors Recurive Algorithm

We will use a recurive algorithm to implement the task. Input to algorithm is input image and seed pixel location,the output is a labelled image. Input to recurive function will be location of pixel required to be analyzed. Inital call to recurive function is the seed pixel location. The function calls a ColorDistance function to check for similarity of the given pixel with the seed pixel provided the pixel has not been analyzed already. If the distance is less than a specified threshold pixel is marked in the output labelled image and recursive calls are initiated to neighbors of the current pixel. Labelling the output image indicates the pixel has been analyzed and any call to function labelled pixel locations will return without performing any action.

4 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

As algorithm progress all connected neighbors of seed pixels are used to initiate recursive calls .If these pixels satisfy the similarity criteria use recurive calls for their connected neighbors and so on. In this way all the connected pixel of seed pixel satisfying the similarity criteria are labelled in the output image and segmentation is achieved. The we can either consider 4/8 connected beighbors for analysis. In the present implementation we use a threshold of 20 . If euclidean distance between two pixels is less than 20 we consider the pixels to be similar. We can see that region growing algorithm stop at places when reflections of object fall on the water.Increasing the threshold is one option to overcome this problem.But it is not guranteed to work in all scenarios.This is shown below. The seed is choosen at location within the shown object. In

(a) Image

(b) thresh 20

(c) thresh 50

(d) input 2

(e) thresh 50

(f) thresh 80

Figure 1: Standard Seeded region growing

the second image we can see than even with high threshold of 80,the white over exposed region will never be labelled as belonging

5 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation to the object.

0.5 Modification of OverExposed Regions

The standard region growing algorithms use only color measurements for similarity criteria.Underwater images are affected by reflections and crinkle effects which leads to over exposure in some parts of the object being segmented. Due to these effects a large intensity change in observed in region withing the object being segmented ie sharp edge are observed within the object region. The standard Region growing algorithms would terminate at these region since large color difference is encountered wrt to seed pixel color or color of object required to be segmented. Thus to overcome this we compute the similarity measure based on the color,exposedness of region as well as local edge intensity. One simple technique is to check the distance ered to white color.If the distance is close to color and within a specified threshold ,which similarity threshold then we will include pixel object. of pixel being considwhite than seed pixel can be independent as belonging to the

However this modification may not always provide desired sementation as shown in the images below.

0.6 Using The Edge Map

In the second image we can see that due to connected white colored pixels the region grows belong the desired object. To over the effect of seeded region growing algorithm growing belond the object we use edge information. We compute the edge map using canny edge detection technique. mean value of edge about a small neighborhood of point being analysed is computed and added to the distance computed to white colored pixels. Thus if the color is white but region has a edge the distance will increase and if region has strong edge it will increase the distance belong the threshold and region growing will stop. though pixel is white in

6 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

(a) input 2

(b) thresh 80

(c) modified

(d) Image

(e) thresh 20

(f) thresh 50

Figure 2: Modified Seeded Region Growing Algorithm

color and connected to original object. Also we make one more modification of reducing the distance threshold to white pixels keeping it half of color threhold. In the above image the hand along with the object is de-

(a) Image

(b) thresh 20

(c) modified SRG

(d) edge map

Figure 3: Using Edge Map

tected since the color threshold is kept high.But the main point

7 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation to note is that due to addition of edge map the region does not overflow significantly from the desired object boundary

0.7 Implementation Details

0.7.1 Color Distance Computation
The color distance used is simply euclidean color distance between two color vectors. and is defined by function colorDist
1 2 3 4 5 6 7 8 9 10 11 12

float seed_fill :: ColorDist3 ( Scalar a , Scalar b ) { float a1 = a [2]; float a2 = a [1]; float a3 = a [0]; float b1 = b [2]; float b2 = b [1]; float b3 = b [0]; float dist =((( a1 - b1 ) *( a1 - b1 ) +( a2 - b2 ) *( a2 - b2 ) +( a3 - b3 ) *( a3 - b3 ) ) ) ; dist = sqrt ( dist /3) ; return dist ; }

0.7.2 Edge Map Computation

The edge map computation is performed before calling the seeded region growing algorithm. First A Laplacian of the RGB image is taken and mean value of all 3 channels is computed. Since edge detection also enhances the noise in the image gaussian blur is applied on the image before taking the laplacian.

1 2 3 4 5 6 7 8

Mat edge1 ; cv :: GaussianBlur ( input , input , Size (3 ,3) ,1) ; cv :: Laplacian ( input , edge1 , input . depth () ,1 ,1) ; vector <Mat > em ; cv :: split ( edge1 , em ) ; cv :: add ( em [0] , em [1] , edge ) ; cv :: add ( edge , em [2] , edge ) ; edge = edge /3;

8 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

The requirement is to obtain a normalized edge map that returns a value between 0-255. The tranformation function should give a value must be large for stronger edges and small for weak edges. The transformation function is monotonic. The transformation function should give a variation of edge map value must gradually increase/decrease according to the edge strength. A exponential transformation function satisfies these criteria. The transformation function used is
g
0

(x) =

exp(

I (x; y )=K )

K is parameter that determine how fast the edge strength falls Since value corresponding to the maximum edge intensity isg (x) = exp(1=K )
0

all the value below this are thresholded to 0

g (x)

=1

(x)
(1)

1 2 3 4 5 6 7

double K =1; edge . convertTo ( edge , CV_32FC (3) , -1.0/(255.0* K ) ,0) ; cv :: exp ( edge , edge ) ; cv :: threshold ( edge , edge , std :: exp ( -1/ K ) +0.001 , 0 , THRESH_TOZERO ) ; edge =1 - edge ; edge =255* edge ; edge . convertTo ( edge , CV_8UC (1) ,1 ,0) ;

0.7.3 Similarity Measure Implementation

In the code the main method to calculate the similarity is ColorDist The Color based similarity measure is computed by the function ColorDist3 which is just the euclidean distance between two RGB color vector.

9 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation The Euclidean distance of the current pixel with the seed pixel color and with mean pixel intesity value about small neighborhood or current pixel with the white pixel is computed We define a small neighborhood size over whih the mean is computed the default value is 5. The ROI is determined as follows
1 2 3 4 5

int s1 =x - bsize /2 >=0? x - bsize /2:0; int s2 =y - bsize /2 >=0? y - bsize /2:0; int s3 =( x - bsize /2+ bsize ) <= width ? bsize :x - bsize /2+ bsize - width -1; int s4 =( y - bsize /2+ bsize ) <= height ? bsize :y - bsize /2+ bsize - height -1; Rect r = Rect ( s1 , s2 , s3 , s4 ) ;

The mean value of the image pixels and edge pixels are computed in the defined ROI about the current pixel.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Mat rx = edge ; uchar * ptr1 =( uchar *) input . data ; uchar * ptr =( uchar *) rx . data ; for ( int i1 =0; i1 < r . height ; i1 ++) { for ( int j1 =0; j1 < r . width ; j1 ++) { f1 = f1 + ptr [( i1 + r . y ) * rx . step + rx . channels () *( j1 + r . x ) ]; f4 = f4 + ptr1 [( i1 + r . y ) * input . step + input . channels () *( j1 + r . x ) ]; f5 = f5 + ptr1 [( i1 + r . y ) * input . step + input . channels () *( j1 + r . x ) +1]; f6 = f6 + ptr1 [( i1 + r . y ) * input . step + input . channels () *( j1 + r . x ) +2]; } } f1 = f1 /( r . width * r . height ) ; f4 = f4 /( r . width * r . height ) ; f5 = f5 /( r . width * r . height ) ; f6 = f6 /( r . width * r . height ) ;

Next we compute the distance of the mean intensity value with white color value
1 2

Scalar cc = Scalar ( ceil ( f4 ) , ceil ( f5 ) , ceil ( f6 ) ) ; float dist2 = ColorDist3 ( cc , c ) ;

10 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation The final step is to specify a threshold .If the distance dist2 to white color value plus mean edge intesity is less than specifed threshold then it belongs to object and we return a similarity value of 0 else we compare the color distane measure to seed pixel value with threshold and decide if the pixel belongs to object or not.

0.8 Stack Based scan line method

In the line scan method ,the pixels in the same vertical/horizontal as the seed/current pixel are scanned.The length of scan is pre defined For the present example we set the minmum extent to be 0 and maximum extent to be image height/width. Here we look at horizontal scan algorithm. Consider a object to be segmented. Consider a seed pixel chosen within the object being segmented. and a horizontal line containing the seed pixel. The point S shown the figure is the seed pixel.

(a) Sample Object

The input to scan line algorithm are a pair of seed points points lying on a horizontal line. The pair of points have different x coordinates and same y co-ordinates. and if line above/below current line is to be scanned next. The scan line algorithm consists of two step ,finding the seed points and next is to mark the pixels between the seed points.

11 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

0.8.1 Initialization
The Seed Points can be found by identifying the points at which the scan line intersects the edges of image. Initial pair of seed points are the same as input seed pixel point. From the current point we keep on checking all the pixels to the left and right of current pixel using similarity criteria. The points at which the similarity criteria is not satisfied are local edge points. Points A,B shown in the figure determine the seed points.

0.8.2 Finding the Seed points

Let us assume we have seed points of current line. Next step is to determine extents of line above and below.Since pixel intensity cannot change abrubtly hence extents will be close to the points A and B. A point with same x co-ordinate as A and y co-ordinate of line above is one of seed points and other seed point is point with same co-ordinate as B and y co-ordinate is line above current line.

(b) Line Scanning

Thus for the above line we obtain the left edge as point A or point to left of A. and the right edge as point B or point to right of B. Let these new points be denoted as C and D. From point C we move towards left filling points which satisfy similarity criteria. If we reach the point D.Then Line has been successfully filled and scan can be performed for the line above and below the present line,

12 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation From point C as we move towards left we may encounter a edge before we reach point D Line segment (E ; F ) and (G; H ) denotes such a case. Thus line segment (E ; F ) and (G; H ) are considered the scanned line and line above/below this line segment will be scanned. Similarity if we move from the point D and end before reaching point C, then this new line segment will be filled and lines above and below this line segment will be scanned next. In this way entire image will be parsed by using connected line segments.

0.9 Implementation
In present application we use a stack based implementation. The elements of stack contain iformation above the line to be scanned. A class Line_Element is used to define the entry.It will contain the right and left co-ordinates of current line segment,y co-ordinate line segment above/below
1 2 3 4 5 6 7

class line_element { public : int left ; int right ; int y ; }

The first operation is to obtain the left and right edges of the line This is done by scanning left/right from the left and right pixel values.

1 2 3 4 5 6 7

for ( xa = p . left ; xa >= nMinX && check_color ( xa , p . y ) == true ; xa - -) ; if ( xa < p . left ) xa = xa +1; for ( xb = p . right ; xb < nMaxX && check_color ( xb , p . y ) == true ; xb ++) ; if ( xb > p . right ) xb = xb -1;

13 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

From left end of the line we determine the line segments encountered as we move towards the right and add pixel of line above and below these line segments to stack. In the below code segment.
xa

and

xb

are left and right ends of line

14 |

??

Image Segmentation - Region Growing Algorithm For UnderWater Image Segmentation

left = xa ; do { for ( xa = left ; xa < nMaxX && check_color ( xa , p . y ) == true ; xa ++) { SetPixel ( xa , p .y , fill_color ) ; } if ( xa > left ) { lines . push ( line_element ( left , xa -1 , p .y -1 , - p . nexty ) ) ; lines . push ( line_element ( left , xa -1 , p . y +1 , - p . nexty ) ) ; } for (; xa < xb && check_color ( xa , p . y ) == false ; xa ++) ; left = xa ; } while ( xa < xb ) ;

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

0.10 Future Work

Static color threholds are being used presently.A adaptive threshold would be required based on information content present in the image as well as strenght of edge . Look for further improvements in stack based algorithms. A better color distance criteria for similarity matching is required. Analysis in different color space and distance other than eculidean will be considered in the future work.

0.11 Code
The class seed_fill_stack defines a class for stack based scan line algorithm. The input to algorithm is input image and seed pixel location and output is labelled image. Code is available in repository https://github.com/pi19404/m19404/tree/master/RegionGrowing

15 |

??