Image Segmentation

ITS69204 Computer Vision and NLP

 ITS69204
Topic 8
Image Segmentation

Outline

8.1 How Image Segmentation Works?

8.2 Types of Image Segmentation
8.2.1 The approach
8.2.2 The types of techniques
8.3 Image Segmentation Techniques
8.3.1 Thresholding Method
8.3.2 Edge-Based Method
8.3.3 Region-Based Method
8.3.4 Clustering-Based Method

2
 ITS69204
Topic 8
Image Segmentation

8.1 How Image Segmentation Works

• Image Segmentation is the process of partitioning digital image into various

subgroups (of pixels) called Image Objects.
• The goal is to reduce the complexity of the image → thus analysing the image
becomes simpler.
• Various image segmentation algorithms can be used to split and group a certain
set of pixels together from the image.
• By doing so, we are actually assigning labels to pixels (that means pixels with the same
label are having similarity in them).

3
 ITS69204
Topic 8
Image Segmentation

8.1 How Image Segmentation Works

• Using these labels, we can specify:

o boundaries,
o draw lines,
o region of interest (ROI)

Segmentation to get major Segmentation to get

components (chair, table etc.) individual chairs

From a machine learning point of view, later, these identified labels can be further used for
both supervised and unsupervised training. 4
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation - Approach

• Segmentation is made possible by grouping similar pixels together and

separate out dissimilar pixels.
• This is done by following two approaches based on the image properties:
 Similarity Detection (region approach)
 Discontinuity detection (boundary approach)

5
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation - Approach

Similarity Detection (Region Approach)

• This approach relies on detecting similar pixels in an image – based on a
threshold, region growing, region spreading, and region merging.
• Machine learning algorithms like clustering relies on this approach of similarity
detection on an unknown set of features (in classification, it detects similarity
based on a pre-defined set of features).

6
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation - Approach

Discontinuity Detection (Boundary Approach)

• This is the opposite of similarity detection approach where the algorithm
rather searches for discontinuity.
• Image Segmentation Algorithms like Edge Detection, Point Detection, Line
Detection follows this approach – where edges get detected based on various
metrics of discontinuity (such as intensity).

7
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation – Types of Techniques

• Based on the two approaches, there are various forms of techniques can be
applied in Image Segmentation Algorithms.
• These techniques can be classified into 3 broader categories:
• Structural Segmentation Techniques
• Stochastic Segmentation Techniques
• Hybrid Techniques

8
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation – Types of Techniques

Structural Segmentation Techniques

Three (3) general steps in structural segmentation:
1. First, understand the structural information about the image to be
processed (information such as pixels, pixel density, distributions, histograms, color
distribution etc).

2. Second, understand the structural information of the ROI (region that we

are about to fetch from the image –identifying the target area)

3. Third, similarity based approach is applied.

9
 ITS69204
Topic 8
Image Segmentation

8.2 Types of Image Segmentation – Types of Techniques

Stochastic Segmentation Techniques

• In these group of algorithms, the primary information is to know the discrete
pixel values of the full image, rather than pointing out the structure of the
required portion of the image.
• ANN and Machine Learning based algorithms that use k-means etc. make use of
this approach.

Hybrid Techniques
• Make use of a combination of structural method and stochastic methods.
• Use both the structural information of a region as well as the discrete pixel
information of the image.

10
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques

Based on the image segmentation approaches and the type of processing that is
needed to be incorporated to attain a goal, the following image segmentation
techniques can be applied:
1. Threshold Method
2. Edge Based Segmentation
3. Region Based Segmentation
4. Clustering Based Segmentation

11
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

1. Threshold Method
• This is perhaps the most basic and yet powerful technique to identify the
required objects in an image.
• Based on the intensity, the pixels in an image get divided by comparing the
pixel’s intensity with a threshold value.

12
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

1. Threshold Method
• Thresholding is useful when the objects in the image in question are assumed
to be having more intensity than the background of the image.
• At its simpler level, the threshold value T is considered to be a constant.
• But this approach may be futile considering the amount of noise that the image
contains (so, we can either keep it constant or change it dynamically based on the image
properties)

• Based on that, thresholding can be categorized into:

A. Simple thresholding
B. Otsu binarization
C. Adaptive thresholding

13
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

A. Simple Thresholding
• This technique replaces the pixels in an image with either black or white.
• If the intensity of a pixel (Ii,j) at position (i,j) is less than the threshold (T),
then → replace with black pixel. If it is more → replace with white pixel.
• This is a binary approach to thresholding.

if 0 (replace with black pixel)

if 1 (replace with white pixel)

14
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

B. Otsu binarization
In global thresholding, an arbitrary value is used for threshold and it remains a
constant. But,

how to determine the correctness of the selected threshold?

A simpler but rather inept method is to trial-and-error.

15
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

B. Otsu binarization (continue)

• For example, take a bimodal image, a histogram with two peaks (one for the
background and one for the foreground).
• According to Otsu binarization, we can approximately take a value in the middle
of those peaks as the threshold value.

16
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

B. Otsu binarization (continue)

• Simply put, it automatically calculates a threshold value from image histogram
for a bimodal image.
• Disadvantage: Not all images are bimodal.
• As such, Otsu’s binarization is commonly used in scanned document, removing
unwanted colors from a document, pattern recognition etc.

17
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques - Thresholding

C. Adaptive Thresholding
Adaptive approach → the algorithm divides the image into smaller portions and
calculates the threshold for those portions of the image.
Advantages:
 can obtain different thresholds for different regions of the same image.
 gives better results for images with varying illumination.
 able to automatically calculate the threshold value.

The threshold value can be the mean of neighborhood area or it can be the weighted sum of
neighborhood values.

18
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Edge-Based

• Edge-based segmentation algorithms work to detect edges in an image, based
on various discontinuities in grey level:
o colour,
o texture,
o brightness,
o saturation,
o contrast etc.
• To enhance the results, supplementary processing steps is applied to connect
all the edges into edge chains that correspond better with image borders.

19
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Edge-Based

• Basic edge detection operators are used to detect edge discontinuities and
hence mark the edge boundaries.
o Sobel
o Canny
o Roberts
o Prewitts

Study and Comparison of Different Edge Detection Operators for Image Segmentation 20
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

• Region based methods work by creating segments by dividing image into

various components with similar characteristics.
• The techniques initially search for some seed points (either smaller parts or

considerably bigger chunks in the input image) .

• Next, certain approaches are employed,

i. either to add more pixels to the seed points
ii. or further diminish (shrink) the seed point to smaller segments and
merge with other smaller seed points.
• Hence, there are two basic techniques based on this method.
o Region Growing
o Region Splitting and Merging

21
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

A. Region Growing
It’s a bottom-up method where we begin with a smaller set of pixel and start
accumulating or iteratively merging it based on certain pre-determined similarity
constraints.
Algorithm steps:
1. starts with an arbitrary seed pixel and compare it with its neighboring pixels.
2. If there is a match in neighboring pixels, then they are added to the initial seed
pixel (region growing).
3. When reach saturation (region growth halt) → chooses another seed pixel.
4. Repeat process 1, 2, 3.

22
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

A. Region Growing
Advantages: this methods often achieve effective segmentation.
Disadvantages: When the algorithm lets a region grow completely before trying
other seeds → resulted in biases (the segmentation in favor of the first segmented region)
To counter this effect:-
 begin with the user inputs of similarities first
 no single region is allowed to dominate
 multiple regions allowed to grow simultaneously

23
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

A. Region Growing
Region growth, also a pixel based algorithm like thresholding but the major
difference is. Region growing techniques are preferable for noisy images, where it
is highly difficult to detect the edges.

Thresholding Region Growing

• Extracts a large region based of • Extracts only adjacent pixels.
similar pixels, from anywhere in • Preferable for noisy images.
the image.

24
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

A. Region Growing

25
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

B. Region Splitting and Merging

• This method is a combination of two basic techniques:
 region splitting
 region merging
• Splitting → iteratively dividing an image into regions of similar characteristics
• Merging → combining the adjacent regions that are similar to each other.

a) input image b) result of split and merge c) result of thresholding

26
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Region-Based

B. Region Splitting and Merging

2 Steps procedure:
1. top-down: split images into homogenous quadrant regions
2. Bottom-up: merge similar adjacent regions
3. Iterate until reach saturation (no further merging and splitting is possible)

This is a divide and conquers method as opposed to the region growth algorithm.

a) input image b) result of split and merge c) result of thresholding

27
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Clustering-Based

• Clustering are unsupervised algorithms (no pre-defined set of features, classes, or groups).
• It helps in fetching the underlying, hidden information from the data
(such as structures, clusters, and groupings that are usually unknown from a heuristic point of view).

• The clustering based techniques segment the image into clusters (disjoint
groups) of pixels with similar characteristics.
• Some of the efficient clustering algorithms:
 k-means
 improved k means
 fuzzy c-mean (FCM)
 improved fuzzy c mean algorithm (IFCM)

28
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Clustering-Based

Techniques Remarks

K-means a popular method due to its simplicity and

computational efficiency.

Improved K-means minimize the number of iterations in a k-means

algorithm.

Fuzzy C-means (FCM) allows pixels to be clustered into more than one
cluster. (a group of pixels can belong to more than one cluster or
group and they can have varying levels of associativity per group).

Improved FCM (IFCM) overcome slower processing time of FCM

29
 ITS69204
Topic 8
Image Segmentation

8.3 Image Segmentation Techniques – Clustering-Based

Possible advantages of fuzzy c-means:

• More robust against outliers
• High true positive rate and acceptable accuracy after just a few iterations
• additional information hidden in the values of the membership functions
• Fuzzy nature of the problem requires fuzzy method.

Flower species Input image K-means Fuzzy c-means

Avanda

Ascocentrum

30
 ITS69204
Topic 8
Image Segmentation

Thank You