1. Introduction
Agriculture is the single largest land use (LU) covering the Earth’s land surface [
1]. The increasing global population and the accompanying increase in food consumption are placing unparalleled demands on agricultural lands [
2]. Some of the negative impacts of these demands include the loss of biodiversity [
1], the degradation and destruction of natural ecosystems [
3], and an increase in greenhouse gas (GHG) emission [
4]. Ensuring food security while minimizing the negative impact of agriculture on the environment requires the use of sustainable agricultural practices [
2,
5]. Formulating agricultural and environmental policies that ensure sustainable agriculture requires the development of an agricultural monitoring system. The foundation of such a system is accurate and up-to-date agricultural LU maps [
6,
7]. Agricultural LU maps are essential input data for various processes such as the estimation of biomass and yield [
7], monitoring of the phenology of different agricultural LU types [
7], modeling of GHG variability [
8], estimation of the area of agricultural lands [
9], and control of area-based subsidies paid to farmers [
9].
The generation and continuous update of agricultural LU maps using traditional methods such as field surveys are inefficient and expensive [
8]. Remote Sensing (RS) provides a better alternative due to the frequency at which data can be acquired over large geographical areas [
10,
11]. The availability of high-resolution satellite images has increased the popularity of Object-Based Image Analysis (OBIA) over traditional pixel-based image analysis [
12]. Unlike pixels, which carry only spectral information, image objects additionally carry contextual and spatial information [
12], thereby making them more useful for subsequent processes such as classification. The advantages of OBIA over pixel analysis for generating agricultural LU maps have been reported by these authors [
10,
13,
14].
Image segmentation, which is the process of clustering image pixels into homogeneous objects, is a critical step in OBIA [
15]. Various authors [
16,
17,
18,
19,
20] have proved that the quality of segmentation has a direct impact on classification accuracy. One of the most popular segmentation algorithms is the Multiresolution Segmentation (MRS) algorithm proposed by Baatz et al. [
21]. MRS is a bottom–up region merging algorithm that starts with one-pixel objects and then in a pairwise manner merges smaller objects into bigger ones until a user-given scale threshold is met [
22]. In a recent review article by Ma et al. [
23], the MRS algorithm as implemented in the eCognition software [
24] accounted for 80.9% of 254 case studies the authors reviewed. This overwhelming popularity hinges on the fact that some exhaustive evaluation studies [
25,
26,
27] have had eCognition coming up tops. In eCognition, the three main parameters that influence the quality of the MRS segmentation are scale, shape, and compactness. To obtain optimal segmentation results, it is imperative to optimize these parameters.
To optimize any segmentation algorithm, the quality of the segmentation output of that algorithm for different parameter combinations ought to be evaluated. This can be done through visual inspection, supervised segmentation evaluation, or unsupervised segmentation evaluation [
28,
29]. Visual inspection is subjective and inherently limits the number of segmentation evaluations that can be done due to its laborious nature [
29]. The supervised evaluation methods assess a segmentation result by comparing it to a reference data and computing a global score (
GS) that represents the degree of similarity between the segmentation result and the reference data [
29]. The main limitation of supervised segmentation evaluation is that the acquisition of reference data is expensive and time-consuming [
29]. This makes unsupervised segmentation evaluation indispensable, as it does not rely on reference data but purely on the content of an image to evaluate the segmentation result [
29]. For the unsupervised evaluation methods, the
GS is a statistical measure that indicates the level of intra-region uniformity and/or inter-region dissimilarity within the segmentation result [
30]. In RS, two of the most used methods are the estimation of scale parameter (ESP) [
31,
32] tool and the objective function [
33]. The ESP tool only addresses the intra-region uniformity of segments by making use of local variance graphs [
34]. The objective function of Espindola et al. [
33] is a combined measure that addresses intra-region uniformity through average area-weighted variance (
WV) and inter-region dissimilarity through spatial autocorrelation using the global Moran’s I (
MI) [
35]. A comparative analysis by Grybas et al. [
36] showed that the objective function outperformed the ESP tool. Various variations [
19,
37,
38,
39,
40,
41,
42] of the objective function have been used in the literature.
To compute the
GS for each input image band, Espindola et al. [
33] separately normalized the
WV and
MI between zero and one before summing them up. Böck et al. [
43] identified a weakness with this normalization step, pointing out that the selection of which segmentation is optimal was dependent on the user-defined scale parameter range. They subsequently proposed the use of fixed ranges to normalize the
WV and
MI. This produced stable results regardless of the input range of the scale parameter. In the remainder of the paper, we call this modification of Böck et al. [
43] the Böck metric. Georganos et al. [
16] identified some limitations with the normalization approach of the Böck metric, which triggered them to propose a different approach. The problem with their approach is that it adds some level of subjectivity to the evaluation process, because it requires some initial empirical tests. This makes their proposal unusable within our context of having a metric that can be used for automated segmentation evaluation without any human intervention.
In this study, we aimed at proposing a new unsupervised evaluation metric for assessing the segmentation output of any segmentation algorithm. To do so, we modified the Böck metric and proposed absolute difference (AD) as a means of computing the
GS. We compared the Böck and AD metrics by separately using each of them in two unsupervised optimization approaches to optimize the parameters of the MRS algorithm to delineate agricultural parcels from 21 Sentinel-2 images of 10 × 10 km sizes in Lower Saxony, Germany. In the first optimization approach, as is mostly done in the literature [
20,
31,
37,
38,
39,
43,
44,
45], we optimized scale while keeping the shape and compactness parameters constant at their default values. In the second optimization approach, we employed Bayesian optimization to optimize all three MRS parameters. The optimal segmentation results identified by each metric were evaluated with parcels from the Land Parcel Identification System (LPIS), which is a spatial database of agricultural parcels and their land-use types as declared by farmers within the European Union (EU) [
46,
47]. The optimal segmentation results of the Böck and AD metrics were compared to each other per each optimization approach. Further, we compared the optimal segmentation results of the unsupervised Bayesian optimization approaches based on the Böck and AD metrics to the benchmark segmentation results of Tetteh et al. [
47], where they used supervised Bayesian optimization.
5. Discussion
The analysis of which metric was optimal for unsupervised segmentation evaluation within our experimental setup of using 21 tiles revealed that our metric (AD) was better than the Böck metric, whether one uses it within a default or Bayesian optimization approach. Visually and quantitatively, the segmentation results yielded by the AD metric were better than the Böck metric in different landscapes composed of diverse agricultural LU types.
For the default optimization approach, at tiles such as T3, where the Böck and AD metrics yielded very similar segmentation results, this is attributable to the fact that there was more clustering of objects as the scale was increased. This is captured by
Figure 7b, where all the
MI values were positive. Clustering normally occurs in areas where there are different LU types but with similar spectral behaviors sharing the same neighborhood or in areas highly dominated by a single LU type such as grasslands, as was the case of T3. Under those conditions, the
GS values of the Böck and AD metrics exhibited a common behavior (
Figure 7) and consequently selected similar scale values, leading to very similar segmentation results.
At other tiles such as T1, where there was an enormous disparity between the two metrics, the agricultural landscape is more diverse and interspersed with different LU types such as winter wheat, sugar beet, and maize. Consequently, they had more negative
MI values with increasing scale (
Figure 6b), which is indicative of the dispersion of objects. The Böck and AD metrics on such occasions differed in curve behavior and global minimum position (
Figure 6). Based on the trajectory of the Böck metric in
Figure 6a, one can safely conclude that the Böck metric would have further decreased if the scale value had further been increased. Our metric, on the other hand, as captured by
Figure 6b, exhibited a decreasing trend up to scale 190 and then started to increase in response to increasing
nWV and decreasing
MI. The benefit of not normalizing the
MI and using absolute difference to compute the
GS became manifest on such occasions, where there was a greater dispersion of agricultural parcels. The AD metric was initially more influenced by the
MI, but it was later more influenced by the
nWV as the scale increased and more
MI values became negative (
Figure 6b). With the AD metric, the
MI and
nWV values have a fair chance of impacting the
GS value depending on their respective magnitudes. The Böck metric, on the other hand, was continuously impacted by the
nMI (
Figure 6a). This can be attributed to the normalization approach applied to the
MI by the Böck metric. As captured by
Figure 6b, before normalization, all the originally negative
MI values were numerically smaller than their corresponding
nWV values. After normalizing the
MI to obtain the
nMI (
Figure 6a), those negative
MI values became numerically higher than their corresponding
nWV values, thereby continuously influencing the
GS of the Böck metric (
Figure 6a).
The Böck metric is more impacted by the
nMI than the
nWV in all agricultural landscapes. This behavior of the Böck metric has the potential of selecting large-scale values as optimal, thereby leading to the identification of under-segmented objects as optimal. This observation was also made by Georganos et al. [
16]. This particular behavior of the Böck metric becomes more problematic in areas with diverse LU types and a greater dispersion of objects, as previously shown in
Figure 5a. The more diverse the LU types and the more spectrally similar they behave, the higher the probability of selecting under-segmented objects as optimal using any segmentation evaluation metric, especially a metric that is purely based on the image content. Therefore, a good unsupervised segmentation evaluation metric must reduce over-segmentation but more importantly under-segmentation as the AD metric proved to be able to do, at least in comparison with the Böck metric. For subsequent processes such as object classification, under-segmentation is preferable to over-segmentation [
26,
58,
59]. In general, under-segmentation can largely be dealt with by using very high-resolution images in which visible boundaries between adjacent but spectrally similar parcels can be identified [
47].
For the unsupervised Bayesian optimization approach, the approach based on the AD metric outperformed that of the Böck metric at all the tiles, especially at T1, which is composed of diverse LU types. Interestingly, at T3, the Bayesian optimization approach based on the AD metric became better than the Böck metric. This is opposite to the default optimization results at T3, where Böck was marginally better than AD. Overall, in both optimization approaches, the AD metric consistently proved to be better suited for optimizing the segmentation of agricultural parcels in different landscapes. A look at the segmentation results for T1 (
Figure 9) clearly shows that the Bayesian optimization approach based on the AD metric generated more segments (
Figure 9d) with a higher segmentation quality than the results of the Bayesian approach based on the Böck metric (
Figure 9b). There was a greater clustering of objects at T2 (
Figure 10) and T17 (
Figure 11) based on the computed
MI values; hence, the Bayesian optimization approach based on the Böck (
Figure 10b and
Figure 11b) and the AD (
Figure 10d and
Figure 11d) metrics yielded very similar segmentation results.
As expected, the supervised Bayesian optimization approach performed better than all the unsupervised Bayesian optimization approaches at all the tiles used in our experiment. This is especially true for T1 (
Figure 9c) and T17 (
Figure 11c), where the landscape has diversified LU types. At T2, which is highly dominated by pome fruits, the segmentation quality was bad for all the optimization methods. Tetteh et al. [
47] in using the supervised Bayesian optimization approach to delineate agricultural parcels made this observation as well for T2 and attributed it to the small size and elongation of agricultural parcels present at that tile. This also holds for the unsupervised Bayesian optimization approaches tested in this research. The high correlation between the number of LU types and the difference in QR between the supervised Bayesian optimization approach and the two unsupervised Bayesian optimization approaches as captured by
Figure 13 indicated that at tiles with a smaller number of LU types, the unsupervised Bayesian approaches obtained results similar to the supervised approach. The supervised Bayesian approach was able to adapt more to diverse agricultural landscapes than the unsupervised Bayesian approaches. An example of this can be seen in
Figure 12c, where the supervised approach generated segments with well-defined boundaries and a better geometric match to the LPIS parcels than the two unsupervised Bayesian approaches in
Figure 12a,b, respectively. The adaptability of supervised segmentation optimization was also asserted by Yang et al. [
39] after testing a supervised optimization approach based on the information gain ratio and an unsupervised optimization approach based on
MI and
WV as was proposed by Espindola et al. [
33]. The major defect of any supervised optimization method is the reliance on reference data, which are tedious to obtain [
29]. An unsupervised method such as the Bayesian optimization approach based on our proposed AD metric provides a good alternative to supervised segmentation optimization.
Unlike the proposition of Georganos et al. [
16], our proposed metric is objective and fully automated. It does not require any human intervention to identify the optimal segmentation. The approach of Georganos et al. [
16] requires the user to compute a certain number of initial segmentations with unknown step intervals, something the authors mentioned has a great impact on the results. Additionally, using locally estimated scatterplot smoothing (LOESS) requires a user to specify the order of the polynomial and a span, which controls the level of smoothing. Since the optimal values of those user inputs cannot be known beforehand, the user has to experiment to identify the optimal settings for normalization, which violates the principle behind unsupervised segmentation evaluation.
6. Conclusions
In this study, we modified an existing unsupervised segmentation evaluation metric based on global variance and spatial autocorrelation [
43]. We proposed the use of absolute difference (AD) to combine the global variance and spatial autocorrelation. We tested the AD metric and the existing metric, named Böck, in identifying the optimal parameters for delineating agricultural parcels from Sentinel-2 images using the Multiresolution Segmentation (MRS) algorithm. We first tested both metrics at 21 tiles with different agricultural landscapes to optimize the scale parameter of the MRS algorithm through default optimization. In this default approach, we kept the shape and compactness parameters constant and increased the scale at equal intervals to determine the optimal one. The AD metric proved superior to the Böck metric in identifying the segmentation result with a better geometric match to reference agricultural parcels in the Land Parcel Identification System (LPIS). On average, the segmentation quality of the AD metric was over 6% higher than the Böck metric in this default approach. Our metric often identified segmentations that were least under-segmented as optimal, unlike the Böck metric. We separately used each metric in a Bayesian optimization routine to optimize the three main parameters of the MRS algorithm at the same 21 tiles. The Bayesian optimization approach based on the AD metric performed better than that of the Böck metric at all tiles. In the Bayesian optimization approach, the quality of the segmentation result of the AD metric was on average about 9% better than the Böck metric. A comparison of the segmentation results in this study to existing benchmark results obtained via supervised Bayesian optimization showed that the unsupervised Bayesian optimization approach based on the AD metric can be a good alternative. In areas where the number of land-use (LU) types was small, supervised and unsupervised Bayesian optimization obtained similar segmentation results. Supervised segmentation optimization methods require reference data, which are generally difficult and time-consuming to generate, especially for wide geographic areas such as regions and countries. The Bayesian optimization approach based on the AD metric solely depends on the image content to fine-tune the optimization process without any human intervention; hence, it can easily be used in any operational OBIA workflow to generate segmentations in near real time.
In a nutshell, our proposed metric performed better than its predecessor in identifying optimal segmentation. Identifying optimal segmentation is important for purposes of obtaining correct agricultural statistics such as the sizes of agricultural parcels. In the absence of reference data, a Bayesian optimization approach based on the AD metric can provide a means of fulfilling the aforementioned purpose in an automated and efficient manner with no human interaction. Even though we tested this optimization approach on the MRS algorithm within the thematic area of agriculture, it is easily applicable to any segmentation algorithm and different thematic areas.
Going into the future, one possible way of improving the results of the segmentation optimization process with our proposed metric will be to incorporate local variance and spatial autocorrelation in a multi-scale approach to refine under-segmented and over-segmented objects in subsequent steps as was done by Johnson et al. [
37]. Different weighting schemes for different agricultural landscapes can be applied to the normalized weighted variance and spatial autocorrelation before the computation of the global score for the AD metric. The impact of this weighting scheme on the identification of the optimal segmentation result would be analyzed accordingly. The impact of the segmentation results identified by the supervised and unsupervised Bayesian optimization approaches on object classification would be assessed. The 21 tiles we used in our experimental setup had relatively flat terrains. However, our proposed metric should work fairly well in other terrains as long as there is enough spectral dissimilarity (dispersion) between adjacent parcels in any geographical area. This hypothesis will be tested in the future.