Cropland mapping from Sentinel-2 time series data using object-based

image analysis
Ovidiu Csillik Mariana Belgiu
University of Salzburg University of Twente
Department of Geoinformatics – Z_GIS Faculty of Geo-Information Science and
Schillerstrasse 30, 5020 Earth Observation (ITC)
Salzburg, Austria P.O. Box 217, 7500 AE
ovidiu.csillik@sbg.ac.at Enschede, The Netherlands
m.belgiu@utwente.nl

Abstract

The increasing spatial and temporal resolution of Sentinel-2 data creates new premises for cropland mapping and monitoring at local,
regional and global scale. This paper reports the results of a study dedicated to cropland mapping from Sentinel-2 time-series data using
objects as spatial analysis units. The Sentinel-2 time-series data stack was automatically segmented using multi-resolution segmentation
algorithm and the resulting image objects were classified using the Time-Weighted Dynamic Time Warping (TWDTW) method. We
applied this approach to map wheat, maize, rice, sunflower and forest in an agricultural area situated in the south-eastern part of Romania.
The implemented cropland mapping framework yielded an overall accuracy of 93.43% and a kappa index of 92%. It has the advantage of
generating spatial vector dataset which can supports decision making in various agricultural management contexts.

Keywords: cropland, remote sensing, object-based image analysis, time-series data, Sentinel-2.

1 Introduction dynamics of the target crop types. Lebourgeois et al. (2017)

used simulated Sentinel-2 time series data, SPOT digital
Accurate and efficient cropland mapping is a pre-requisite for elevation model (DEM) and very high resolution (VHR)
developing efficient agricultural management policies which Pleiades images for mapping smallholder agriculture in
facilitate the implementation of sustainable food supply tropical areas. VHR images served as input data for
systems worldwide. delineating agricultural fields through segmentation, whereas
Remote sensing data has already been successfully used in the variables (spectral indices, reflectance) derived from
different cropland monitoring programs due to their global simulated time-series Sentinel-2 data were used for the
coverage (Inglada et al., 2015). Existing studies dedicated to classification purpose. This study revealed that multi-temporal
cropland mapping used MODIS data (Maus et al., 2016, information is essential for cropland mapping in small-scale
Arvor et al., 2011, Pringle et al., 2012, Qiu et al., 2017), agricultural regions. Despite the increasing interest of the
SPOT data (Desclée et al., 2006), Landsat data (Yan and Roy, remote sensing community in using Sentinel-2 data for
2015, Qiu et al., 2017) or, Landsat data combined with SPOT croplands mapping, none of the existing studies evaluated
(Matton et al., 2015) or MODIS data (Jia et al., 2014). Since how OBIA method performs when applied to cropland
its launch, Sentinel-2 data gained the attention of the remote mapping from time-series Sentinel-2 data. This study reports
sensing community for cropland mapping and monitoring due the classification results yielded for wheat, maize, rice,
to the high spatial resolution (e.g. 10 m spatial resolution of sunflower and forest mapping from a diverse agricultural
the blue, green, red and near-infrared spectral bands) and high landscape situated in Romania.
temporal resolution (5 days revisiting time with twin
satellites). The increased spatial resolution of the carried
MultiSpectral Instrument (MSI) sensor allows the application 2 Study area and data
of object-based image analysis (OBIA) method (Blaschke,
2010) for cropland mapping. This method has the advantage The study area is situated in the south-eastern part of
of generating ready-to-use spatial vector datasets which can Romania and it is divided into two sections by the Danube
be further used by decision-makers for developing sustainable River (Fig. 1). As depicted in Figure 1, the eastern part of the
agricultural management and monitoring policies (Matton et study area comprises compact, relatively large agricultural
al., 2015). Immitzer et al. (2016) evaluated for example the fields, whereas the western part is characterized by a highly
suitability of single-date Sentinel-2 data for crops mapping by fragmented agricultural landscape. The study area have an
applying Random Forest algorithm (Breiman, 2001) using extent of 2740×2335 pixels, covering 63.979 ha.
both pixel and object-based classifications. This study For the cropland classification purpose, we used 13
revealed that object-based and pixel-based Random Forest Sentinel-2 Level 1C images for the year 2016 (from February
achieved comparable results. The authors concluded that to September), well distributed across the agricultural
multi-temporal Sentinel-2 images might increase the calendar. The images were selected according to the cloud
classification results, because the multi-temporal spectral and coverage: the images with cloud coverage greater than 10%
phenological indices can better describe the vegetation were not used in the further classification framework. This
 AGILE 2017 – Wageningen, May 9-12, 2017

study focuses only on the visible bands of the Sentinel-2 3.2 Time-Weighted Dynamic Time Warping
sensor (band 2, band 3 and band 4) and near-infrared band method
(band 8) for the segmentation process, while the phenological
cycles of the land use/land cover classes of interest were TWDTW is a supervised classification method that uses the
computed using the Normalized Difference Vegetation Index temporal patterns of the training samples computed from
(NDVI). different indices and/or spectral features to classify the
agricultural fields of interest (Fig. 3). For this study, we
Figure 1: Study area localization in south-eastern part of randomly generated 2500 samples, resulting in 589 training
Romania. samples and 1660 validation samples. These samples were
classified through visual interpretation, relying on the visual
interpretation keys developed using the European Land Use
and Coverage Area Frame Survey (LUCAS) training samples
for the land cover/land use classes of interest. This dataset was
available only for 2015 and therefore, we could not use it
directly as training samples for Sentinel-2 data which was
available for 2016.

Figure 2: The methodological workflow used in this study.

3 Methodology

This section introduces the methodological framework

applied in this study: image segmentation, time-weighted
dynamic time warping and evaluation of the final
classification (Fig. 2).

3.1 Object-based image analysis framework

Figure 3: Temporal patterns of NDVI for wheat, maize, rice,
OBIA has gained an increasing attention in the last years sunflower and forest computed from Sentinel-2 NDVI time
due to the high classification accuracy yielded when applied series data. The vertical axis represents the value of NDVI,
to very high resolution satellite images (Blaschke, 2010). The while the horizontal axis represents the time.
method consists of two main steps: segmentation and
classification.
Through segmentation the image is partitioned into
contiguous spatial objects according to a homogeneity criteria
defined by users. Resulting objects (referred into the OBIA
literature as image objects) are classified using supervised or
unsupervised image classification procedure. In our study, we
applied the multi-resolution segmentation (Baatz and Schäpe,
2000) to segment the Sentinel-2 time-series data stack. To
reduce the subjectivity and the time required for the
segmentation algorithm tuning, we applied the Estimation of
Scale Parameter 2 (ESP2) tool (Drăguţ et al., 2014), which
automatically selects an objective scale parameter (SP) based
on local variance of objects. The red, green, blue and near-
infrared bands were used in the process of segmenting 13
different dates, resulting in a stack of 52 layers. We used all
these layers in order to identify all meaningful boundaries
which may appear during the timeframe of the analysis. In the
end, 13 different raster datasets were exported, representing
the mean NDVI values for each object. Resulting raster
datasets were then stacked and ready to be analysed using
Time-Weighted Dynamic Time Warping (TWDTW) approach
(Maus et al., 2016) (Fig. 2).
 AGILE 2017 – Wageningen, May 9-12, 2017

Applying TWDTW method requires three main steps (1) Figure 4: Object-based TWDTW classification of the study
generating the temporal patterns of the training samples based area. The settlements (gray color) are masked.
on the NDVI raster time series (Fig. 3), (2) applying the
TWDTW analysis and (3) classifying raster time series (Fig.
2). In our study we used the R package called dtwSat (Maus et
al., 2016) to execute the workflow towards obtaining a
TWDTW classification based on objects.
We applied a logistic TWDTW with α = −0.1 and β = 50,
which means that we added a time-weight to the DTW with a
low penalty for time warps smaller than 50 days and higher
penalty for bigger time warps (Maus et al. 2016). Two raster
datasets are created: the categorical map of classes of interest
and the TWDTW dissimilarity measure of the scene
classified.

3.3 Classification accuracy assessment

The accuracy of the obtained object-based classification was
evaluated using overall accuracy, producer’s accuracy, user’s
accuracy metrics (Congalton, 1991) and kappa coefficient
(Cohen, 1960), using 1660 validation samples.

4 Results and discussions

The objective of this study was to evaluate the performance

of OBIA method when applied to cropland mapping from
Sentinel-2 time-series data. The image objects delineated by Figure 5: Two subsets of the study area showing the
applying multi-resolution segmentation algorithm were further segmentation results superimposed on a RGB image from
classified using the TWDTW method (Fig. 4). We obtained a May: (A) highly fragmented agricultural landscape and (C)
classification accuracy of 93.43% and a kappa coefficient of relatively large agricultural parcels. For the same two subsets,
92% (Table 1). The lowest user’s accuracy was yielded for the the object-based classification is showed in (B) for small
rice class (78.8%) because of the confusion with the maize parcels and (D) for large parcels.
class, whereas the highest user’s accuracy was achieved by
the wheat class (99.72%), followed by the forest, maize and
sunflower classes. The lowest producer’s accuracy was also
obtained by the rice class (82.86%), followed by the maize
class with 86.68%. The high misclassification rate of the
maize class was caused by the confusion with rice and
sunflower. Better results are expected by including the red-
edge and SWIR bands in the image analysis procedure.
Immitzer et al. (2016) showed in their study dedicated to
cropland mapping from single-date Sentinel-2 data that among
the five most important spectral bands two are located in the
red-edge and one in the SWIR.
The most critical step of OBIA is the segmentation of the
images into meaningful objects (Belgiu and Drǎguţ, 2014).
Most of the exiting segmentation algorithms require users’
intervention for selecting the segmentation parameters. In our
study, we used the ESP-2 tool to automatically identify the SP
for crop fields delineation from time-series data. We applied
the ESP-2 tool using a hierarchical segmentation approach to
ensure a faster segmentation, because each of the iteration is
built on the previous obtained level of segmentation. The final segmented. The under-segmentation increases the
obtained SP was 186, using shape 0.1 and compactness 0.5, misclassification rate especially when smaller agricultural
resulting in 4479 objects with a mean area of 1428 pixels of fields cultivated with different crops are merged together. In
10m resolution. the eastern part of our study area, the agricultural fields were
The diversity of the agricultural landscape can greatly over-segmented, because of the utilization of irrigation
impact the segmentation results obtained by applying the systems that caused spectral variation within the same
ESP-2 tool. The western part of our study area for example is agriculture field (Fig. 5). Over-segmentation is preferable to
characterized by small agricultural parcels (Fig. 5). under-segmentation (Gao et al., 2011), because smaller
Consequently, narrower and elongated fields were under- objects belonging to the same agriculture field can be easily
 AGILE 2017 – Wageningen, May 9-12, 2017

Table 1: Classification accuracy achieved by applying the TWDTW method to objects, as spatial analysis units. PA
stands for Producer’s accuracy, UA stands for User’s accuracy and OA for Overall accuracy.
Reference class
Map class Total UA (%)
Wheat Maize Rice Sunflower Forest Unclassified
Wheat 360 0 0 1 0 0 361 99.72
Maize 0 332 30 14 1 0 377 88.06
Rice 0 32 145 3 3 1 174 78.80
Sunflower 3 19 0 175 0 0 197 88.83
Forest 1 0 0 1 335 0 337 99.41
Unclassified 0 0 0 0 0 204 204 100
Total 364 383 175 194 339 205 1660

PA (%) 98.90 86.68 82.86 90.21 98.82 99.51 OA (%) = 93.43

merged together in the further image analysis steps. Under- ready-to-use spatial vector datasets. The availability of vector-
segmentation results on the other hand cannot be easily ruled based agricultural fields datasets at regional and global level
out. One possible solution to this problem could be the could benefit agricultural management and monitoring
division of the study area into two regions based on the agencies including the European Common Agriculture Policy
characteristics (and thus complexity) of the agricultural fields. (CAP) and its European Agricultural Guarantee Fund (EAGF)
In this way, the highly fragmented agricultural landscape program towards developing sustainable agricultural practices
present in the western part of the study area could be and agricultural subsidies programs for food security.
segmented independently from the eastern part, where larger
agricultural fields dominate. The same procedure has been
successfully implemented by d’Oleire-Oltmanns and Tiede Acknowledgements
(2014) for gully delineation from QuickBird image. In a
previous study, Matton et al. (2015) argued that the slightly The work of Ovidiu Csillik was supported by the Austrian
better classification results produced by the object-based
Science Fund (FWF) through the Doctoral College GIScience
method do not balance out the high processing time required
for the segmentation step. In our case, the segmentation of (DK W1237-N23).
time-series layers proved to be a less computationally-
intensive process (1h 50min, using 16 cores with 2.90 GHz
and 32GB memory).
In spite of the increasing temporal resolution of Sentinel-2 References
data, the presence of clouds significantly reduced the number
of available scenes. To address the problem of cloud Arvor, D., Jonathan, M., Meirelles, M. S. P., Dubreuil, V. and
obscuration, radar data such as RADARSAT-2 or Sentinel-1 Durieux, L. Classification of MODIS EVI time
data can be used in combination with Sentinel-2 for cropland series for crop mapping in the state of Mato Grosso,
mapping. Brazil. International Journal of Remote Sensing, 32:
Our study revealed also that TWDTW is an efficient method 7847-7871, 2011.
for cropland mapping using phenological pattern of target Baatz, M. and Schäpe, A. Multiresolution segmentation – an
crops computed from NDVI time-series data. In the future optimization approach for high quality multi-scale
study, we plan to evaluate how this method performs when image segmentation. In: Strobl, J., Blaschke, T. and
additional indices such as Enhanced Vegetation Index (EVI) Griesebner, G., (eds.) Angewandte Geographische
or Normalized Difference Water Index (NDWI) (McFeeters, Informations-Verarbeitung XII, 12-23, 2000.
1996) and the vegetation red edge bands (band 5, 6, 7 and 8a, Belgiu, M. and Drǎguţ, L. Comparing supervised and
respectively) are used. Furthermore, we plan to evaluate how unsupervised multiresolution segmentation
OBIA performs in different geographical regions where the approaches for extracting buildings from very high
agricultural fields have different spatial structures and where resolution imagery. ISPRS Journal of
the agricultural practices differ. Photogrammetry and Remote Sensing, 96: 67-75,
2014.
Blaschke, T. Object based image analysis for remote sensing.
5 Conclusion ISPRS Journal of Photogrammetry and Remote
Sensing, 65: 2-16, 2010.
This study showed that OBIA can be successfully used for Breiman, L. Random Forest. Machine Learning, 45(1): 5-32,
cropland mapping from Sentinel-2 time-series data, achieving 2001.
an overall accuracy of 93.43%. One of the advantages of this Cohen, J. A coefficient of agreement for nominal scales.
method consists in reducing the salt-and-pepper effect Educational and psychological measurement, 20(1):
occurring usually in per pixel classifications and in generating 37-46, 1960.
 AGILE 2017 – Wageningen, May 9-12, 2017

Congalton, R. G. A review of assessing the accuracy of features. International Journal of Remote Sensing,
classifications of remotely sensed data. Remote 17: 1425-1432, 1996.
sensing of environment, 37(1): 35-46, 1991. Pringle, M. J., Denham, R. J. and Devadas, R. Identification
D’Oleire-Oltmanns, S. and Tiede, D. Specific target objects- of cropping activity in central and southern
specific scale levels? Application of the estimation Queensland, Australia, with the aid of MODIS
of scale parameter 2 (ESP 2) tool for the MOD13Q1 imagery. International Journal of
identification of scale levels for distinct target Applied Earth Observation and Geoinformation, 19:
objects. South Eastern European Journal of Earth 276-285, 2012.
Observation and Geomatics, 3(2S): 579-584, 2014. Qiu, B., Luo, Y., Tang, Z., Chen, C., Lu, D., Huang, H., Chen,
Desclée, B., Bogaert, P. and Defourny, P. Forest change Y., Chen, N. and Xu, W. Winter wheat mapping
detection by statistical object-based method. Remote combining variations before and after estimated
Sensing of Environment, 102: 1-11, 2006. heading dates. ISPRS Journal of Photogrammetry
Drăguţ, L., Csillik, O., Eisank, C. and Tiede, D. Automated and Remote Sensing, 123: 35-46, 2017.
parameterisation for multi-scale image segmentation Yan, L. and Roy, D. P. Improved time series land cover
on multiple layers. ISPRS Journal of classification by missing-observation-adaptive
Photogrammetry and Remote Sensing, 88: 119-127, nonlinear dimensionality reduction. Remote Sensing
2014. of Environment, 158: 478-491, 2015.
Gao, Y., Mas, J. F., Kerle, N. and Navarrete Pacheco, J. A.
Optimal region growing segmentation and its effect
on classification accuracy. International Journal of
Remote Sensing, 32, 3747-3763, 2011.
Immitzer, M., Vuolo, F. and Atzberger, C. First Experience
with Sentinel-2 Data for Crop and Tree Species
Classifications in Central Europe. Remote Sensing,
8(3): 166, 2016.
Inglada, J., Arias, M., Tardy, B., Hagolle, O., Valero, S.,
Morin, D., Dedieu, G., Sepulcre, G., Bontemps, S.,
Defourny, P. and Koetz, B. Assessment of an
Operational System for Crop Type Map Production
Using High Temporal and Spatial Resolution
Satellite Optical Imagery. Remote Sensing, 7(9):
12356-12379, 2015.
Jia, K., Liang, S., Zhang, N., Wei, X., Gu, X., Zhao, X., Yao,
Y. and Xie, X. Land cover classification of finer
resolution remote sensing data integrating temporal
features from time series coarser resolution data.
ISPRS Journal of Photogrammetry and Remote
Sensing, 93: 49-55, 2014.
Lebourgeois, V., Dupuy, S., Vintrou, É., Ameline, M., Butler,
S. and Bégué, A. A Combined Random Forest and
OBIA Classification Scheme for Mapping
Smallholder Agriculture at Different Nomenclature
Levels Using Multisource Data (Simulated Sentinel-
2 Time Series, VHRS and DEM). Remote Sensing,
9(3): 259, 2017.
Matton, N., Canto, G. S., Waldner, F., Valero, S., Morin, D.,
Inglada, J., Arias, M., Bontemps, S., Koetz, B. and
Defourny, P. An automated method for annual
cropland mapping along the season for various
globally-distributed agrosystems using high spatial
and temporal resolution time series. Remote
Sensing, 7: 13208-13232, 2015.
Maus, V., G, C., Cartaxo, R., Sanchez, A., Ramos, F. M. and
Queiroz, G. R. D. A Time-Weighted Dynamic Time
Warping Method for Land-Use and Land-Cover
Mapping. IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sensing,
9(8): 3729-3739, 2016.
McFeeters, S. K. The use of the Normalized Difference Water
Index (NDWI) in the delineation of open water