research-article

A constrastive semi-supervised deep learning framework for land cover classification of satellite time series with limited labels

Authors:

Raffaele Gaetano,

Roberto InterdonatoAuthors Info & Claims

Volume 567, Issue C

https://doi.org/10.1016/j.neucom.2023.127031

Published: 04 March 2024 Publication History

Abstract

In this work, we present a new semi-supervised learning framework to cope with satellite image time series (SITS) classification in a data paucity scenario, considering extreme low levels of supervision. The proposed methodology, referred as S 3 ITS (Semi-Supervised Satellite Image Time Series classification method), is based on temporal convolutional neural networks and it takes advantage of both labelled and unlabelled information. S 3 ITS enforces the data to be projected in a discriminative manifold via contrastive learning, in order to produce a data representation where samples belonging to the same category are closer than the ones belonging to different ones. Pseudo-labelling is employed on unlabelled samples to take the most out of the available unlabelled information. Experiments on two study sites described by SITS of Sentinel-2 images highlight the quality of the proposed method with respect to common SITS-based classification methods and recent machine learning approaches especially tailored for the semi-supervised classification of multi-variate time series data.

References

[1]

Kolecka N., Ginzler C., Pazur R., Price B., Verburg P.H., Regional scale mapping of grassland mowing frequency with sentinel-2 time series, Remote Sens. 10 (8) (2018) 1221.

[2]

Pelletier C., Webb G.I., Petitjean F., Temporal convolutional neural network for the classification of satellite image time series, Remote. Sens. 11 (5) (2019) 523.

[3]

Wulder M.A., Masek J.G., Cohen W.B., Loveland T.R., Woodcock C.E., Opening the archive: How free data has enabled the science and monitoring promise of landsat author links open overlay panel, Remote Sens. Environ. 122 (2012) 2–10.

[4]

Guttler F., Ienco D., Nin J., Teisseire M., Poncelet P., A graph-based approach to detect spatiotemporal dynamics in satellite image time series, ISPRS J. Photogramm. Remote Sens. 130 (2017) 92–107.

[5]

Abade N.A., Júnior O.A.d.C., Guimarães R.F., de Oliveira S.N., Comparative analysis of MODIS time-series classification using support vector machines and methods based upon distance and similarity measures in the Brazilian Cerrado-Caatinga boundary, Remote Sens. 7 (9) (2015) 12160–12191.

[6]

Castillo-Navarro J., Saux B.L., Boulch A., Audebert N., Lefèvre S., Semi-supervised semantic segmentation in earth observation, Mach. Learn. (2021).

[7]

Ding Y., Zhao X., Zhang Z., Cai W., Yang N., Zhan Y., Semi-supervised locality preserving dense graph neural network with ARMA filters and context-aware learning for hyperspectral image classification, IEEE Trans. Geosci. Remote. Sens. 60 (2022) 1–12.

[8]

Bagnall A.J., Lines J., Bostrom A., Large J., Keogh E.J., The great time series classification bake off: A review and experimental evaluation of recent algorithmic advances, Data Min. Knowl. Discov. 31 (3) (2017) 606–660.

Digital Library

[9]

Ruiz A.P., Flynn M., Large J., Middlehurst M., Bagnall A.J., The great multivariate time series classification bake off: A review and experimental evaluation of recent algorithmic advances, Data Min. Knowl. Discov. 35 (2) (2021) 401–449.

Digital Library

[10]

Fawaz H.I., Forestier G., Weber J., Idoumghar L., Muller P., Deep learning for time series classification: A review, Data Min. Knowl. Discov. 33 (4) (2019) 917–963.

Digital Library

[11]

Triguero I., García S., Herrera F., Self-labeled techniques for semi-supervised learning: Taxonomy, software and empirical study, Knowl. Inf. Syst. 42 (2) (2015) 245–284.

[12]

Fan H., Zhang F., Wang R., Huang X., Li Z., Semi-supervised time series classification by temporal relation prediction, in: ICASSP, IEEE, 2021, pp. 3545–3549.

[13]

Marussy K., Búza K., SUCCESS: A new approach for semi-supervised classification of time-series, in: Rutkowski L., Korytkowski M., Scherer R., Tadeusiewicz R., Zadeh L.A., Zurada J.M. (Eds.), Artificial Intelligence and Soft Computing - 12th International Conference, in: Lecture Notes in Computer Science, vol.7894, ICAISC 2013, Zakopane, Poland, June 9-13, 2013, Proceedings, Part I, Springer, 2013, pp. 437–447.

[14]

Frank J., Mannor S., Pineau J., Precup D., Time series analysis using geometric template matching, IEEE Trans. Pattern Anal. Mach. Intell. 35 (3) (2013) 740–754.

[15]

Castellanos M.G., Bergmeir C., Triguero I., Rodríguez Y., Benítez J.M., Self-labeling techniques for semi-supervised time series classification: An empirical study, Knowl. Inf. Syst. 55 (2) (2018) 493–528.

[16]

Jawed S., Grabocka J., Schmidt-Thieme L., Self-supervised learning for semi-supervised time series classification, in: PAKDD. Vol. 12084, 2020, pp. 499–511.

[17]

Zhang X., Gao Y., Lin J., Lu C., TapNet: Multivariate time series classification with attentional prototypical network, in: AAAI, 2020, pp. 6845–6852.

[18]

Ienco D., dos Santos D.P., de Carvalho A.C.P.L.F., Evaluate pseudo labeling and CNN for multi-variate time series classification in low-data regimes, in: ICANN. Vol. 12895, 2021, pp. 126–137.

[19]

Hu T., Huang X., Li J., Benediktsson J.A., Yang J., Gong J., A semi-supervised approach towards land cover mapping with sentinel-2 desnse time-series imagery, in: IGARSS, IEEE, 2019, pp. 2423–2426.

[20]

Zhou Z., Li M., Tri-training: Exploiting unlabeled data using three classifiers, IEEE Trans. Knowl. Data Eng. 17 (11) (2005) 1529–1541.

Digital Library

[21]

Sitokonstantinou V., Koukos A., Kontoes C., Bartsotas N.S., Karathanassi V., Semi-supervised phenology estimation in cotton parcels with sentinel-2 time-series, in: IGARSS, IEEE, 2021, pp. 8491–8494.

[22]

Interdonato R., Ienco D., Gaetano R., Ose K., DuPLO: A DUal view point deep learning architecture for time series classification, ISPRS J. Photogramm. Remote Sens. 149 (2019) 91–104.

[23]

Rußwurm M., Körner M., Self-attention for raw optical satellite time series classification, ISPRS J. Photogramm. Remote Sens. 169 (2020) 421–435.

[24]

Garnot V.S.F., Landrieu L., Giordano S., Chehata N., Satellite image time series classification with pixel-set encoders and temporal self-attention, in: CVPR, Computer Vision Foundation / IEEE, 2020, pp. 12322–12331.

[25]

Chopra S., Hadsell R., LeCun Y., Learning a similarity metric discriminatively, with application to face verification, in: CVPR, 2005, pp. 539–546.

[26]

Cascante-Bonilla P., Tan F., Qi Y., Ordonez V., Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning, in: AAAI, AAAI Press, 2021, pp. 6912–6920.

[27]

Hagolle O., Huc M., Villa Pascual D., Dedieu G., A multi-temporal and multi-spectral method to estimate aerosol optical thickness over land, for the atmospheric correction of FormoSat-2, LandSat, vENμs and sentinel-2 images, Rem. Sens. 7 (3) (2015) 2668–2691.

[28]

Inglada J., Vincent A., Arias M., Tardy B., iota2-a25386, Zenodo, 2016.

[29]

Dupuy S., Gaetano R., Mézo L.L., Mapping land cover on reunion island in 2017 using satellite imagery and geospatial ground data, Data Brief 28 (2020).

[30]

Jolivot A., Lebourgeois V., Leroux L., et al., Harmonized in situ datasets for agricultural land use mapping and monitoring in tropical countries, Earth Syst. Sci. Data 13 (12) (2021) 5951–5967.

[31]

Wang Z., Yan W., Oates T., Time series classification from scratch with deep neural networks: A strong baseline, in: IJCNN, IEEE, 2017, pp. 1578–1585.

[32]

Yue Z., Wang Y., Duan J., Yang T., Huang C., Tong Y., Xu B., TS2Vec: Towards universal representation of time series, in: AAAI, AAAI Press, 2022, pp. 8980–8987.

[33]

Ismail-Fawaz A., Devanne M., Weber J., Forestier G., Enhancing time series classification with self-supervised learning, in: 15th International Conference on Agents and Artificial Intelligence: ICAART 2023, INSTICC, SciTePress, 2023, pp. 1–8.

[34]

Tan P.-N., Steinbach M., Kumar V., Introduction to Data Mining, First Ed., Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2005.

[35]

Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay E., Scikit-learn: Machine learning in Python, J. Mach. Learn. Res. 12 (2011) 2825–2830.

[36]

Kingma D.P., Ba J., Adam: A method for stochastic optimization, 2014, CoRR abs/1412.6980.

[37]

van der Maaten L., Hinton G., Visualizing Data Using t-SNE, J. Mach. Learn. Res. 9 (2008) 2579–2605.

Index Terms

A constrastive semi-supervised deep learning framework for land cover classification of satellite time series with limited labels
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
      2. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Semi-supervised Time Series Classification Model with Self-supervised Learning
Abstract
Semi-supervised learning is a powerful machine learning method. It can be used for model training when only part of the data are labeled. Unlike discrete data, time series data generally have some temporal relation, which can be ...
Highlights
- Self-supervised temporal relation learning can assist supervised model for time series classification.
Semi-supervised multi-label classification using incomplete label information
Highlights
- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
Abstract
Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...

Comments

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 567, Issue C

Jan 2024

157 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 04 March 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents