Image Segmentation

Md. Zahidul Islam

Assistant Professor
Dept. of CSE
Dhaka International University
 What is Image Segmentation?
 In computer vision, Image Segmentation is the process of subdividing a
digital image into multiple segments(sets of pixels, also known as super
pixels).
 Image segmentation reduces pixel data to region-based information
 segmentation of an image which classifies pixels into objects or groups.

 Segmentation subdivides an image into its constituent regions or

groups.

 The level to which the subdivision is carried depends on the problem

being solved.

 That is, segmentation should stop when the objects of interest in an

application have been isolated.

 Image Segmentation? Cont…
 Image segmentation is an aspect of image processing.
 Image segmentation is a computer vision process.
 Image segmentation is the first step in image analysis.

Input Segmented Object Feature

image objects/image quantification vector Results
Image Classificati
Annotation Feature
segmentati on or
of objects extraction
on cluster

A typical image analysis pipeline.

Purposes of Image Segmentation
 The purpose of image segmentation is to partition an image
into meaningful regions with respect to a particular
application.
 The goal of segmentation is to simplify or change the
representation of an image into something that is more
meaningful and easier to analyze.
 The segmentation is based on measurements taken from the
image and might be grey level, colour, texture, depth or
motion.
 Image segmentation is typically used to locate objects and
boundaries in images
 How Does Image Segmentation Work?

 Image segmentation is a function that takes image inputs and produces

an output. The output is a mask or a matrix with various elements
specifying the object class or instance to which each pixel belongs.
 Several relevant heuristics, or high-level image features, can be useful
for image segmentation. These features are the basis for standard
image segmentation algorithms that use clustering techniques like
edges and histograms.
 An example of a popular heuristic is color. Graphics creators may use a
green screen to ensure the image background has a uniform color,
enabling programmatic background detection and replacement in post-
processing.
 How Does Image Segmentation Work?
Cont...
 Another example of a useful heuristic is contrast, image segmentation
programs can easily distinguish between a dark figure and a light background
(i.e., the sky). The program identifies pixel boundaries based on highly
contrasting values.
 Traditional image segmentation techniques based on such heuristics can be fast
and simple, but they often require significant fine-tuning to support specific
use cases with manually designed heuristics. They are not always sufficiently
accurate to use for complex images. Newer segmentation techniques use
machine learning and deep learning to increase accuracy and flexibility.
 Machine learning-based image segmentation approaches use model training to
improve the program’s ability to identify important features.
 Deep neural network technology is especially effective for image segmentation

tasks.
 How Does Image Segmentation Work?
Cont...
 There are various neural network designs and implementations suitable
for image segmentation. They usually contain the same basic
components:
 An encoder—a series of layers that extract image features using
progressively deeper, narrower filters. The encoder might be pre-
trained on a similar task (e.g., image recognition), allowing it to
leverage its existing knowledge to perform segmentation tasks.
 A decoder—a series of layers that gradually convert the encoder’s
output into a segmentation mask corresponding with the input image’s
pixel resolution.
 Skip connections—multiple long-range neural network connections
allowing the model to identify features at different scales to enhance
model accuracy.
 Why segmentation is useful ?

 Segmentation accuracy determines the eventual success

or failure of computerized analysis procedure.
 Improvement of pictorial information for human
interpretation/perception
 Mapping and Measurement - Automatic analysis of
remote sensing data from satellites to identify and
measure regions of interest. e.g. Petroleum reserves
 It might be possible to analyze the image in the
computer and provide cues (Ingit) to the radiologists to
help detect important/suspicious structures (e.g.:
Computed Aided Diagnosis, CAD)
 Application of Image
Segmentation
Usually image segmentation is an initial and vital
step in a series of processes aimed at overall image
understanding

Applications of image segmentation include

Identifying objects in a scene for object-based
measurements such as size and shape
Identifying objects in a moving scene for object-based
video compression (MPEG4)
Identifying objects which are at different distances from a
sensor using depth measurements from a laser range
finder enabling path planning for a mobile robots
 Application Field of Image Segmentation.

Image segmentation is a key building block of

computer vision technologies and algorithms. It is
used for many practical applications including
medical image analysis, computer vision for
autonomous vehicles, face recognition and
detection, video surveillance, and satellite image
analysis.
 Principal approaches
Segmentation algorithms generally are based on one of
2 basis properties of intensity values
discontinuity : to partition an image based on sharp
changes in intensity (such as edges in an image).
Discontinuity refers to the unexpected change in the intensity value of
pixels.

similarity : Similarity refers to partition the image

into different-different region based on predefined
criteria such as Thresholding , region merging, region
growing and region splitting etc.
 Segmentation Techniques

We will look at two very simple image

segmentation techniques that are based on the
greylevel histogram of an image
Thresholding
Clustering
 Detection of Similarities-
Thresholding

A. THRESHOLDING
 Thresholding is the simplest, powerful and most
frequently/widely used technique for image segmentation
 It is useful in discriminating foreground from the
background.
 Thresholding operation is used to convert a multilevel/gray
scale image into binary image
 The advantage of obtaining first a binary image is that it
reduces the complexity of the data and simplifies the
process of recognition and classification.
 Thresholding
 The most common way to convert a gray-level image into
a binary image is to select a single threshold
value(T).Then all the gray level values below T will be
classified as black(0) i.e. background and those above T
will be white(1) i.e. objects.
 The Thresholding operation is a grey value remapping
operation g defined by:
0 if f(x,y) < T
g(x,y)= Where (x,y) represents a gray value/
1 if f(x,y) ≥ T, are the coordinates of the threshold value poin
T represent threshold value
g(x,y) represents threshold image
f(x,y) represents gray level image pixels/
input image
 Thresholding

5 THRESHOLDING TECHNIQUES
1. MEAN TECHNIQUE- This technique used the mean value of the
pixels as the threshold value and works well in strict cases of the
images that have approximately half to the pixels belonging to the
objects and other half to the background.
2. P-TILE TECHNIQUE- Uses knowledge about the area size of the
desired object to the threshold an image.
3. HISTOGRAM DEPENDENT TECHNIQUE (HDT)- separates the two
homogonous region of the object and background of an image.
4. EDGE MAXIMIZATION TECHNIQUE (EMT)- Used when there are
more than one homogenous region in image or where there is a
change of illumination between the object and its background.
5. VISUAL TECHNIQUE- Improve people’s ability to accurately search
for target items.
 Segmentation Techniques

B.CLUSTERING
 Defined as the process of identifying groups of similar image primitive.
 It is a process of organizing the objects into groups based on its
attributes.
 An image can be grouped based on keyword (metadata) or its
content (description)
 KEYWORD- Form of font which describes about the image
keyword of an image refers to its different features
 CONTENT- Refers to shapes, textures or any other information
that can be inherited from the image itself. Image texture is a set
of metrics calculated in image processing designed to quantify the
perceived texture of an image. It gives us information about the
spatial arrangement of colour or intensities in an image or
selected region of an image.
 Another Segmentation Approaches

A.WATER BASED SEGMENTATION

Steps:
1. Derive surface image:
A variance image is derived from each image layer. Centred
at every pixel, a 3x3 moving window is used to derive its
variance for that pixel. The surface image for watershed
delineation is a weighted average of all variance images
from all image layers. Equal weight is assumed in this study.
 Segmentation Approaches

2. Delineate watersheds
From the surface image, pixels within a homogeneous
region form a watershed

3. Merge Segments
Adjacent watershed may be merged to form a new segment
with larger size according to their spectral similarity and a
given generalization level
 Region-Oriented Segmentation
 Region Splitting
 Region growing starts from a set of seed points. Seed points are grid
points selected to agglomerate the surrounding control volumes.
 An alternative is to start with the whole image as a single region and
subdivide the regions that do not satisfy a condition of homogeneity.
 Region Merging
 Region merging is the opposite of region splitting.
 Start with small regions (e.g. 2x2 or 4x4 regions) and merge the
regions that have similar characteristics (such as gray level, variance).
 Typically, splitting and merging approaches are used iteratively.

19
 Split and Merge Approach:
 This is a 2 step procedure:
 top-down: split image into
homogeneous quadrant regions
 bottom-up: merge similar adjacent
regions
 The algorithm includes:
Top-down
 successively subdivide image into
quadrant regions R i

 stop when all regions are

homogeneous: P(R ) = TRUE)
i

obtain quadtree structure

Bottom-up
 at each level, merge adjacent
regions R and R if P(R [ R ) = TRUE
i j i j

 Iterate until no further

splitting/merging is possible
 Segmentation Approaches

B. REGION-GROW APPROACH
 This approach relies on the homogeneity of spatially
localized features
 It is a well-developed technique for image segmentation.
It postulates that neighbouring pixels within the same
region have similar intensity values.
 The general idea of this method is to group pixels with the
same or similar intensities to one region according to a
given homogeneity criterion.
Segmentation Approaches

The region growing

algorithm of the
image which was
shown on the next
slide.
 REGION GROWING
 Group pixels or sub-regions into PIXEL
larger regions when
homogeneity criterion is satisfied 10AGGREGATION:
10 10 10 10 10 10
 Region grows around the seed 10 10 10 69 70 10 10
point based on similar properties 59 10 60 64 59 56 60
(grey level, texture, color) 10 59 10 60 70 10 62
PROS: 10 60 59 65 67 10 65
 Better in noisy image where
10 10 10 10 10 10 10
edges are hard to identify 10 10 10 10 10 10 10
CONS:
 Seed point must be specified
 Different seed point will give Homogeneity criteria:
different results  The difference between 2 pixel
values is less than or equal to 5
 Horizontal, vertical, diagonal
 Segmentation Approaches

C. EDGE-BASED METHODS
 Edge-based methods center around contour detection:
their weakness in connecting together broken contour
lines make them, too, prone to failure in the presence of
blurring.
 Segmentation Approaches

D. CONNECTIVITY-PRESERVING RELAXATION-
BASED METHOD
 Referred as active contour model
 The main idea is to start with some initial boundary shape
represented in the form of spline curves, and iteratively
modify it by applying various shrink/expansion operations
according to some energy function.