An Efficient Image Segmentation Algorithm Using Neutrosophic Graph Cut

Abstract

Segmentation is considered as an important step in image processing and computer vision applications, which divides an input image into various non-overlapping homogenous regions and helps to interpret the image more conveniently. This paper presents an efficient image segmentation algorithm using neutrosophic graph cut (NGC). An image is presented in neutrosophic set, and an indeterminacy filter is constructed using the indeterminacy value of the input image, which is defined by combining the spatial information and intensity information. The indeterminacy filter reduces the indeterminacy of the spatial and intensity information. A graph is defined on the image and the weight for each pixel is represented using the value after indeterminacy filtering. The segmentation results are obtained using a maximum-flow algorithm on the graph. Numerous experiments have been taken to test its performance, and it is compared with a neutrosophic similarity clustering (NSC) segmentation algorithm and a graph-cut-based algorithm. The results indicate that the proposed NGC approach obtains better performances, both quantitatively and qualitatively.

1. Introduction

With a classical definition, image segmentation refers to dividing an input image into several sub-images according to a pre-defined criterion where the sub-images are disjointed, homogenous and meaningful. Image segmentation is also known as an important and crucial step in many computer vision and pattern-recognition applications. Many researchers have been working on image segmentation, and works have been done [1].

Among the published works, graph-based segmentation algorithms constitute an important image segmentation category [2]. A graph G can be denoted as $G=\left(V,E\right)$ where V and E are a set of vertices and edges. On an image, vertices can be either pixels or regions, and edges connect the neighboring vertices [3]. A weight is a non-negative measure of dissimilarity which is associated with each edge using some property of the pixels.

In this paper, using the advantages of neutrosophic to interpret the indeterminacy on the image, we combine neutrosophic set into the graph cut for image segmentation. Neutrosophic set (NS) was an extension of the fuzzy set [4]. In NS theory, a member of a set has degrees to the truth, falsity, and indeterminacy, respectively [5]. Therefore, it has an ability to deal with the indeterminacy information and has attracted much attention in almost all engineering communities and subsequently a great number of works have been studied, such as NS-based color and texture segmentation [6,7,8,9,10,11,12,13,14], NS-based clustering [15,16,17], NS-based similarity for image thresholding [18], NS-based edge detection [19] and NS-based level set [20].

Firstly, the image is interpreted using neutrosophic set and indeterminacy degree is calculated accordingly. Then an indeterminacy filter is constructed using the indeterminacy value on the image which is defined by combining the spatial and intensity information. The indeterminacy filter reduces the indeterminacy in the intensity and spatial information respectively. A graph is defined on the image and the weight for each pixel is represented using the value after indeterminacy filtering, and the energy function is also redefined using the neutrosophic value. A maximum-flow algorithm on the graph is employed to obtain the final segmentation results.

The proposed method has the following new contributions: (1) an indeterminate filter is proposed to reduce the uncertain information in the image; and (2) a new energy function in graph model is defined in neutrosophic domain and used to segment the image with better performance.

The rest of the paper is structured: Section 2 briefly reviews the previous works. Section 3 describes the proposed method based on neutrosophic graph cut. Section 4 provides the experimental results. Conclusions are drawn in Section 5.

2. Previous Works

As mentioned in the Introduction Section, graph based image segmentation has gained much attention from the domain researchers with many published papers. A systematic survey work on graph-based image segmentation was conducted by Peng et al. [21]. In this survey, authors categorized the graph-based image segmentation methods into five groups. The first category is minimal spanning tree (MST)-based method. The MST is a popular concept in graph theory with numerous works. In [22], a hierarchical image segmentation method was proposed based on MST [22]. This method segmented the input image iteratively. At each iteration, one sub-graph was produced and, in the final segmentation, there were a given number of sub-graphs. In [23], a region merging procedure was adopted to produce a MST-based image segmentation algorithm using the differences between two sub graphs and inside graphs.

Cost-function-based graph cut methods constitute the second category. The most popular graph-based segmentation methods are in this category. Wu et al. [3] applied the graph theory to image segmentation and proposed the popular minimal cut method to minimize a cost function. A graph-based image segmentation approach namely normalized cut (Ncut) was presented [24]. It alleviates shortcomings of the minimal cut method by introducing an eigen system. Wang et al. [25] presented a graph-based method and a cost function and defined it as the ratio of the sum of different weights of edges along the cut boundary. Ding et al. [26] presented a cost function to alleviate the weakness of the minimal cut method, in which the similarity between two subgraphs was minimized, and the similarity within each subgraph was maximized. Another efficient graph-based image segmentation method was proposed in [27], and considered both the interior and boundary information. It minimized the ratio between the exterior boundary and interior region. The Mean-Cut incorporates the edge weight function [25] to minimize the mean edge weight on the cut boundary.

Methods based on Markov random fields (MRF) are in the third class, and the shortest-path-based methods are classified in the fourth class. Generally, MRF-based graph cut methods form a graph structure with a cost function and try to minimize that cost function to solve the segmentation problem. The shortest path based methods searched the shortest path between two vertices [21], and the boundaries of segments were achieved by employing the shortest path. The shortest-path-based segmentation methods need interaction from users.

The other graph-based methods are categorized into the fifth class. The random walker (RW) method by Grady [28] used a weighted graph to obtain labels of pixels and then these weights were considered as the likelihood that RW went across the edge. Finally, a pixel label was assigned by the seed point where the RW reached first.

3. Proposed Method

3.1. Neutrosophic Image

An element in NS is defined as: let $A=\left\{A_1,A_2,,\dots .,A_m\right\}$ as a set of alternatives in neutrosophic set. The alternative A_i is $\left\{T\left(A_i\right),I\left(A_i\right),F\left(A_i\right)\right\}/A_i$, where $T\left(A_i\right)$, $I\left(A_i\right)$ and $F\left(A_i\right)$ are the membership values to the true, indeterminate and false set.

An image I_m in NS is called neutrosophic image, denoted as I_NS which is interpreted using Ts, Is and Fs. Given a pixel P(x,y) in I_NS, it is interpreted as $P_{NS}\left(x,y\right)=\left\{Ts\left(x,y\right), Is\left(x,y\right),Fs\left(x,y\right)\right\}$. $Ts\left(x,y\right)$, $Is\left(x,y\right)$ and $Fs\left(x,y\right)$ represent the memberships belonging to foreground, indeterminate set and background, respectively.

Based on the intensity value and local spatial information, the true and indeterminacy memberships are used to describe the indeterminacy among local neighborhood as:

$$Ts\left(x,y\right)=\frac{g\left(x,y\right)-g_{\min }}{g_{\max }-g_{\min }}$$

(1)

$$Is\left(x,y\right)=\frac{Gd\left(x,y\right)-G{d}_{\min }}{G{d}_{\max }-G{d}_{\min }}$$

(2)

where g(x,y) and Gd(x,y) are the intensity and gradient magnitude at the pixel of (x,y) on the image.

We also compute the neutrosophic membership values based on the global intensity distribution which considers the indeterminacy on intensity between different groups. The neutrosophic c-means clustering (NCM) overcomes the disadvantages on handling indeterminate points in other algorithms [16]. Here, we use NCM to obtain the indeterminacy values between different groups on intensity to be segmented.

Using NCM, the truth and indeterminacy memberships are defined as:

$$K={\left[\frac{1}{{\varpi }_1}{\displaystyle \sum _{j=1}^C{\left(x_i-c_j\right)}^{-\frac{2}{m-1}}}+\frac{1}{{\varpi }_2}{\left(x_i-{\overline{c}}_{i\max }\right)}^{-\frac{2}{m-1}}+\frac{1}{{\varpi }_3}{\delta }^{-\frac{2}{m-1}}\right]}^{-1}$$

(3)

$$T{n}_{ij}=\frac{K}{{\varpi }_1}{\left(x_i-c_j\right)}^{-\frac{2}{m-1}}$$

(4)

$$I{n}_i=\frac{K}{{\varpi }_2}{\left(x_i-{\overline{c}}_{i\max }\right)}^{-\frac{2}{m-1}}$$

(5)

where $T_{n_{ij}}$ and $I_{n_i}$ are the true and indeterminacy membership value of point i, and the cluster centers is $c_j$. ${\overline{c}}_{i\max }$ is obtained using to indexes of the largest and second largest value of T_ij. They are updated at each iteration until $\left|T_{n_{ij}}^{(k+1)}-T_{n_{ij}}^{(k)}\right|<\epsilon$, where $\epsilon$ is a termination criterion.

3.2. Indeterminacy Filtering

A filter is newly defined based on the indeterminacy and used to remove the effect of indeterminacy information for segmentation, in which the kernel function is defined using a Gaussian function as follows:

$$G_I\left(u,v\right)=\frac{1}{2\pi {\sigma }_I^2}\exp \left(-\frac{u^2+v^2}{2{\sigma }_I^2}\right)$$

(6)

$${\sigma }_I\left(x,y\right)=f\left(I\left(x,y\right)\right)=aI\left(x,y\right)+b$$

(7)

where ${\sigma }_I$ is the standard deviation value where is defined as a function $f\left(·\right)$ associated to the indeterminacy degree. When the indeterminacy level is high, ${\sigma }_I$ is large and the filtering can make the current local neighborhood more smooth. When the indeterminacy level is low, ${\sigma }_I$ is small and the filtering takes a less smooth operation on the local neighborhood. The reason to use Gaussian function is that it can map the indeterminate degree to a filter weight more smooth.

An indeterminate filtering is taken on $T_s\left(x,y\right)$, and it becomes more homogeneous.

$$T_s^{\prime }\left(x,y\right)=Ts\left(x,y\right)\oplus G_{Is}\left(u,v\right)={\displaystyle \sum }_{v=y-m/2}^{y+m/2}{\displaystyle \sum }_{u=x-m/2}^{x+m/2}Ts\left(x-u,y-v\right)G_{Is}\left(u,v\right)$$

(8)

$$G_{Is}\left(u,v\right)=\frac{1}{2\pi {\sigma }_{Is}^2}\exp \left(-\frac{u^2+v^2}{2{\sigma }_{Is}^2}\right)$$

(9)

$${\sigma }_{Is}\left(x,y\right)=f\left(Is\left(x,y\right)\right)=aIs\left(x,y\right)+b$$

(10)

where $T_s^{\prime }$ is the indeterminate filtering result. $a$ and $b$ are the parameters in the linear function to transform the indeterminacy level to parameter value.

The filtering is also used on $T_{n_{ij}}\left(x,y\right)$ after NCM. The input of NCM is the local spatial neutrosophic value after indeterminacy filtering.

$$T{n}_{ij}^{\prime }\left(x,y\right)=T{n}_{ij}\left(x,y\right)\oplus G_{In}\left(u,v\right)={\displaystyle \sum }_{v=y-m/2}^{y+m/2}{\displaystyle \sum }_{u=x-m/2}^{x+m/2}T{n}_{ij}\left(x-u,y-v\right)G_{In}\left(u,v\right)$$

(11)

$$G_{In}\left(u,v\right)=\frac{1}{2\pi {\sigma }_{In}^2}\exp \left(-\frac{u^2+v^2}{2{\sigma }_{In}^2}\right)$$

(12)

$${\sigma }_{In}\left(x,y\right)=f\left(In\left(x,y\right)\right)=cIn\left(x,y\right)+d$$

(13)

where $T{n}_{ij}^{\prime }$ is the indeterminate filtering result on T_s and $m$ is the size of the filter kernel. $T{n}_{ij}^{\prime }$ is employed to construct a graph, and a maximum-flow algorithm is used to segment the image.

3.3. Neutrosophic Graph Cut

A cut C = (S,T) partitions a graph G = (V,E) into two subsets: S and T. The cut set of a cut C = (S,T) is the set $\{(u,v)\in E|u\in S,v\in T\}$ of edges that have one endpoint in S and the other endpoint in T. Graph cuts can efficiently solve image segmentation problems by formulating in terms of energy minimization, which is transformed into the maximum flow problem in a graph or a minimal cut of the graph.

The energy function often includes two components: data constrict $E_{data}$ and smooth constrict $E_{smooth}$ as:

$$E(f)=E_{data}(f)+E_{smooth}(f)$$

(14)

where f is a map which assigns pixels to different groups. $E_{data}$ measures the disagreement between f and the assigned region, which can be represented as a t-link, while $E_{smooth}$ evaluates the extent of how f is piecewise smooth and can be represented as an n-link in a graph.

Different models have different forms in the implementation of the energy function. The function based on Potts model is defined as:

$$E(f)={\displaystyle \sum _{p\in P}{D}_p(f_p)}+{\displaystyle \sum _{\{p,q\}\in N}{V}_{\{p,q\}}(f_p,f_q)}$$

(15)

where p and q are pixels, and N is the neighborhood of p. $D_p$ evaluates how appropriate a segmentation is for the pixel p.

In the proposed neutrosophic graph cut (NGC) algorithm, the data function $D_p$ and smooth function $V_{\{p,q\}}$ are defined as:

$$D_{ij}(p)=|T{n}_{ij}^{\prime }(p)-C_j|$$

(16)

$$V_{\{p,q\}}(f_p,f_q)=u\delta (f_p\ne f_q)$$

(17)

$$\delta (f_p\ne f_q)=\{\begin{array}{cc}1& \mathrm{if}{f}_p\ne f_q\\ 0& \mathrm{otherwise}\end{array}$$

(18)

where $u$ is a constant number in [0, 1] and used for a penalty of the disagree of labeling of pixel p and q.

After the energy function is redefined in the neutrosophic set domain, a maximum flow algorithm in graph cut theory is used to segment the objects from the background.

All steps can be summarized as:

• Step 1: Compute the local neutrosophic value $T_s$ and $I_s$.
• Step 2: Take indeterminate filtering on $T_s$ using $I_s$.
• Step 3: Use NCM algorithm on the filtered $T_s$ subset to obtain $T_n$ and $I_n$.
• Step 4: Filter $T_n$ using indeterminate filter based on $I_n$.
• Step 5: Define the energy function based on the $T_n{}^{\prime }$ value.
• Step 6: Partition the image using the maximum flow algorithm.

The flowchart of the proposed approach is shown in Figure 1 as:

To show the steps of the whole algorithm, some intermediate results are demonstrated using an example image in Figure 2.

4. Experimental Results

It is challenging to segment images having uncertain information such as noise. Different algorithms have been developed to solve this problem. To validate the performance of the NGC approach on image segmentation, we test it on many images and compare its performance with a newly published neutrosophic similarity clustering (NSC) method [12] which performed better than previous methods [6], and a newly developed graph cut (GC) method [29].

All experiments are taken using the same parameters: a = 10; b = 0.25; c = 10; d = 0.25; and u = 0.5.

4.1. Quantitatively Evaluation

Simulated noisy images are employed to compare the NGC with NSC and GC methods visually, and then their performances are tested quantitatively by using two metrics. In the NSC method [12], simulated noisy images were employed to evaluate its performance. To make the comparison fair and consistent, we use the same images and noise and test three algorithms on them.

A simulated image having intensities of 64, 128, and 192 is added with Gaussian noises and used to evaluate the performance of NGC, NSC, and GC algorithms. Figure 3a shows the original noisy images with noise mean values are 0 and variance values: 80, 100, and 120, respectively. Figure 3b–d lists results by the NSC, GC, and NGC methods, respectively. The results in Figure 3 also show the NGC performs visually better than NSC and GC methods on the simulated images with low contrast and noises. Pixels in Figure 3b,c that are segmented into wrong groups are assigned into the right groups by NGC method in Figure 3d. Boundary pixels, which are challenging to label, are also segmented into right categories by NGC.

Misclassification error (ME) is used to evaluate the segmentation performances [30,31,32]. The ME measures the percentage of background wrongly categorized into foreground, and vice versa.

$$\mathrm{M}\mathrm{E}=1-\frac{|B_o\cap B_T|+|F_o\cap F_T|}{\left|B_o\right|+\left|F_o\right|}$$

(19)

where F_o, B_o, F_T, and B_T are the object and background pixels on the ground truth image and the resulting image, respectively.

In addition, FOM [31] is used to evaluate the difference between the segmented results with the ground truth:

$$\mathrm{FOM}=\frac{1}{\max (N_I,N_A)}{\displaystyle \sum _{k=1}^{N_A}\frac{1}{1+\beta d^2(k)}}$$

(20)

where $N_I$ and $N_A$ are the numbers of the segment object and the true object pixels. $d\left(k\right)$ is the distance from the k_th actual pixel to the nearest segmented result pixel. $\beta$ is a constant and set as 1/9 in [31].

The quality of the noisy image is measured via a signal to noise ratio (SNR):

$$\mathrm{SNR}=10\log \left[\frac{{\displaystyle \sum _{r=1}^{H-1}{\displaystyle \sum _{c=1}^{W-1}{I}^2(r,c)}}}{{\displaystyle \sum _{r=1}^{H-1}{\displaystyle \sum _{c=1}^{W-1}{(I(r,c)-I_n(r,c))}^2}}}\right]$$

(21)

where I_n(r,c)and I(r,c) are the intensities of point (r,c) in the noisy and original images, respectively.

The results of ME and FOM are drawn in Figure 4 and Figure 5, where ✳ denotes NSC method, o denotes GC method, and + is NGC method. NGC method has the lowest ME values. All ME by NGC are smaller than 0.043, and all values from NSC and GC methods are larger than those from NGC method. The NGC obtains the best performance with ME = 0.0068 when SNR is 5.89 dB, while NSC has the lowest value ME = 0.1614 and GC ME = 0.0327. NGC also has bigger FOM than NSC and GC, especially at the low SNR. The comparison results are listed in

Table 1. Performance comparisons on evaluation metrics.

Metrics	NSC	GC	NGC
ME	0.247 ± 0.058	0.062 ± 0.025	0.015 ± 0.011
FOM	0.771 ± 0.025	0.897 ± 0.027	0.987 ± 0.012

. The mean and standard deviation of the ME and FOM are 0.247 ± 0.058 and 0.771 ± 0.025 using NSC method, 0.062 ± 0.025 and 0.897 ± 0.027 using GC method, 0.015 ± 0.011 and 0.987 ± 0.012 using NGC method, respectively. The NGC method achieves better performance with lesser values of ME and FOM than the NSC and GC methods.

4.2. Performance on Natural Images

Many images are employed to validate the NGC’s performance. We also compare the results with a newly developed image segmentation algorithm based on an improved kernel graph cut (KGC) algorithm [33]. Here, five images are randomly selected to show the NGC method’s segmentation performance. The first row in Figure 6, Figure 7, Figure 8, Figure 9 and Figure 10 shows the original images and segmentation results of NSC, GC, KGC, and NGC, respectively. The other rows demonstrate the results on the noisy images. The results by NGC have better quality than those of NSC, GC, and KGC visually. On the original images, the NGC and GC obtain similarly accurate results, while the KGC obtains under-segmented results. When the noise is increased, the NSC and GC are deeply affected and have a lot of over-segmentation, and the KGC results are under-segmentation and lose some details. However, NGC is not affected by noise and most pixels are categorized into the right groups, and the details on the boundary are well segmented.

Figure 6 shows the segmentation results on the “Lena” image. The results in the fourth columns are better than in the second and third columns. Regions of face, nose, mouth, and eyes are segmented correctly by NGC. The noisy regions as hair region and the area above the hat are also segmented correctly. However, the NSC and GC methods obtain wrong segmentations, especially in the region above the hat. The KGC results lose some detail information on face and eyes. In the observation, the NGC algorithm is better than NSC.

We also compared the performances of all methods on the “Peppers” image, as shown in Figure 7. As mentioned earlier for other comparisons, for zero noise level, GC, NGC, and KGC produced similar segmentations. GC, KGC and NGC methods produced better segmentation results than NSC in all noise levels. When the noise level increased, the efficiency of the proposed NGC method became more obvious. There were some wrong segmentation regions (black regions in gray pepper regions) in the GC results. Some of the background regions were also wrongly segmented by the GC method. More proper segmentations were obtained with the proposed NGC method. Especially, for noise levels 20 and 30, the NGC method’s segmentation achievement was visually better than the others, with less wrongly segmented regions produced. On this image, the KGC achieves similar performance as NGC on the segmentation results.

The comparison results on the “Woman” image are given in Figure 8. It is obvious that the NSC method produced worse segmentations when the noise level increased. The GC and KGC methods produced better results when compared to the NSC method, with more homogeneous regions produced. It is also worth mentioning that the GC, KGC and NGC methods produced the same segmentation results for the noiseless case. However, when the noise level increased, the face of the woman became more complicated. On the other hand, the proposed NGC method produced more distinctive regions when compared to other methods. On the results of KGC, the boundary of eyes and nose cannot be recognized. In addition, the edges of the produced regions by NGC were smoother than for the others.

We also compared these methods on the “Lake” image, as shown in Figure 9. In the comparisons, it is seen that GC, KGC and NGC methods produced better results than for the NSC method. The results are especially better at high noise levels. It should be specified that GC and KGC methods produced more homogeneous regions, but, in that case, the boundary information was lost. This is an important disadvantage of the GC method. On the other hand, the proposed NGC method also produced comparable homogeneous regions, while preserving the edge information. The proposed method especially yielded better results at high noise levels.

In Figure 10, a more convenient image was used for comparison purposes. While the blood cells can be considered as objects, the rest of the image can be considered as background. In the “Blood” image, the NSC and NGC methods produced similar segmentation results. The KGC has some wrong segmentation on the background region. The NGC has better results on the noisy blood images where the blood cells are extracted accurately and completely. The superiority of the NGC algorithm can also be observed in this image.

5. Conclusions

This study aims to develop an efficient method to segment images having uncertain information such as noise. To overcome this challenge, a novel image segmentation method is proposed based on neutrosophic graph cut in this paper. An image is mapped into the neutrosophic set domain and filtered using a newly defined indeterminacy filter. Then, a new energy function is designed according to the neutrosophic values after indeterminacy filtering. The indeterminacy filtering operation removes the indeterminacy in the global intensity and local spatial information. The segmentation results are obtained by maximum flow algorithm. Comparison results demonstrate the better performance of the proposed method than existing methods, in both quantitative and qualitative terms. It also shows that the presented method can segment the images properly and effectively, on both clean images and noisy images, because the indeterminacy information in the image has been handled well in the proposed approach.