Just Accepted
TorchGeo: Deep Learning With Geospatial Data
Abstract
Remotely sensed geospatial data are critical for applications including precision agriculture, urban planning, disaster monitoring and response, and climate change research, among others. Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available. However, the variance in data collection methods and handling of geospatial metadata make the application of deep learning methodology to remotely sensed data nontrivial. For example, satellite imagery often includes additional spectral bands beyond red, green, and blue and must be joined to other geospatial data sources that may have differing coordinate systems, bounds, and resolutions. To help realize the potential of deep learning for remote sensing applications, we introduce TorchGeo, a Python library for integrating geospatial data into the PyTorch deep learning ecosystem. TorchGeo provides data loaders for a variety of benchmark datasets, composable datasets for uncurated geospatial data sources, samplers for geospatial data, and transforms that work with multispectral imagery. TorchGeo is also the first library to provide pre-trained models for multispectral satellite imagery (e.g., models that use all bands from the Sentinel-2 satellites), allowing for advances in transfer learning on downstream remote sensing tasks with limited labeled data. We use TorchGeo to create reproducible benchmark results on existing datasets and benchmark our proposed method for preprocessing geospatial imagery on the fly. TorchGeo is open source and available on GitHub: https://github.com/microsoft/torchgeo.
References
[1]
Michael Abrams, Robert Crippen, and Hiroyuki Fujisada. 2020. ASTER global digital elevation model (GDEM) and ASTER global water body dataset (ASTWBD). Remote Sensing 12, 7 (2020), 1156.
[2]
Wingtra AG. 2021. WingtraOne mapping drone and Sony RX1R II camera. https://wingtra.com/mapping-drone-wingtraone/image-quality/
[3]
European Environment Agency. 2016. European Digital Elevation Model (EU-DEM). https://www.eea.europa.eu/en/datahub/datahubitem-view/d08852bc-7b5f-4835-a776-08362e2fbf4b.
[4]
Swedish Forest Agency. 2021. Forest Damages - Larch Casebearer 1.0.
[5]
Steve Ahlswede, Christian Schulz, Christiano Gava, Patrick Helber, Benjamin Bischke, Michael Förster, Florencia Arias, Jörn Hees, Begüm Demir, and Birgit Kleinschmit. 2022. TreeSatAI Benchmark Archive: A multi-sensor, multi-label dataset for tree species classification in remote sensing. Earth System Science Data Discussions 2022 (2022), 1–22.
[6]
Matej Aleksandrov et al. 2018. eo-learn. https://github.com/sentinel-hub/eo-learn
[7]
PaddlePaddle Authors. 2022. PaddleRS, Awesome Remote Sensing Toolkit based on PaddlePaddle. https://github.com/PaddlePaddle/PaddleRS.
[8]
Kumar Ayush, Burak Uzkent, Chenlin Meng, Kumar Tanmay, Marshall Burke, David Lobell, and Stefano Ermon. 2021. Geography-aware self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. Computer Vision Foundation, virtual conference, 10181–10190.
[9]
Gerald Baier, Antonin Deschemps, Michael Schmitt, and Naoto Yokoya. 2021. Synthesizing optical and SAR imagery from land cover maps and auxiliary raster data. IEEE Transactions on Geoscience and Remote Sensing 60 (2021), 1–12.
[10]
Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2019. Multimodal Machine Learning: A Survey and Taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 2(2019), 423–443. https://doi.org/10.1109/TPAMI.2018.2798607
[11]
Wele Gedara Chaminda Bandara and Vishal M Patel. 2022. A transformer-based Siamese network for change detection. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Kuala Lumpur, Malaysia, 207–210.
[12]
Charles T Bargeron and David J Moorhead. 2007. EDDMapS—early detection and distribution mapping system for the southeast exotic pest plant council. Wildland weeds 10, 4 (2007), 4–8.
[13]
Favyen Bastani, Piper Wolters, Ritwik Gupta, Joe Ferdinando, and Aniruddha Kembhavi. 2023. SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision. Computer Vision Foundation, Paris, France, 16772–16782.
[14]
Adrian Boguszewski, Dominik Batorski, Natalia Ziemba-Jankowska, Tomasz Dziedzic, and Anna Zambrzycka. 2021. LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands, Water and Roads From Aerial Imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, virtual conference, 1102–1110.
[15]
Jirka Borovec et al. 2020. TorchMetrics: Machine learning metrics for distributed, scalable PyTorch applications. https://github.com/PyTorchLightning/metrics
[16]
Claire Boryan, Zhengwei Yang, Rick Mueller, and Mike Craig. 2011. Monitoring US agriculture: the US Department of Agriculture, National Agricultural Statistics Service, Cropland Data Layer Program. Geocarto International 26, 5 (2011), 341–358.
[17]
Ben Boyter. 2018. scc: Sloc, Cloc and Code. https://github.com/boyter/scc.
[18]
Daniele Rege Cambrin, Luca Colomba, and Paolo Garza. 2023. CaBuAr: California burned areas dataset for delineation. IEEE Geoscience and Remote Sensing Magazine 11, 3 (2023), 106–113.
[19]
Daniele Rege Cambrin and Paolo Garza. 2024. QuakeSet: A Dataset and Low-Resource Models to Monitor Earthquakes through Sentinel-1. arXiv preprint arXiv:2403.18116(2024).
[20]
Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu, and Markus Reichstein. 2021. Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences. John Wiley & Sons, United Kingdom.
[21]
Javiera Castillo-Navarro, Bertrand Le Saux, Alexandre Boulch, Nicolas Audebert, and Sébastien Lefèvre. 2022. Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance suite, dataset analysis and multi-task network study. Machine Learning 111, 9 (2022), 3125–3160.
[22]
Hao Chen, Zipeng Qi, and Zhenwei Shi. 2021. Remote sensing image change detection with transformers. IEEE Transactions on Geoscience and Remote Sensing 60 (2021), 1–14.
[23]
Hao Chen and Zhenwei Shi. 2020. A spatial-temporal attention-based method and a new dataset for remote sensing image change detection. Remote Sensing 12, 10 (2020), 1662.
[24]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning. PMLR, Vienna, Austria, 1597–1607.
[25]
Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote Sensing Image Scene Classification: Benchmark and State of the Art. Proc. IEEE 105, 10 (2017), 1865–1883. https://doi.org/10.1109/JPROC.2017.2675998
[26]
Gong Cheng, Junwei Han, Peicheng Zhou, and Lei Guo. 2014. Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS Journal of Photogrammetry and Remote Sensing 98 (2014), 119–132. https://doi.org/10.1016/j.isprsjprs.2014.10.002
[27]
Alex Clark et al. 2010. Pillow: The friendly PIL fork (Python Imaging Library). https://github.com/python-pillow/Pillow
[28]
Andrea Codegoni, Gabriele Lombardi, and Alessandro Ferrari. 2023. TINYCD: A (not so) deep learning model for change detection. Neural Computing and Applications 35, 11 (2023), 8471–8486.
[29]
Rodrigo Caye Daudt, Bertr Le Saux, and Alexandre Boulch. 2018. Fully convolutional Siamese networks for change detection. In 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, Athens, Greece, 4063–4067.
[30]
Rodrigo Caye Daudt, Bertr Le Saux, Alexandre Boulch, and Yann Gousseau. 2018. Urban change detection for multispectral earth observation using convolutional neural networks. In IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Valencia, Spain, 2115–2118.
[31]
João Lucas de Sousa Almeidaet al. 2024. TerraTorch: a Python toolkit for fine-tuning Geospatial Foundation Models (GFMs). https://github.com/IBM/terratorch
[32]
Tristan Deleu, Tobias Würfl, Mandana Samiei, Joseph Paul Cohen, and Yoshua Bengio. 2019. Torchmeta: A meta-learning library for PyTorch. arXiv preprint arXiv:1909.06576(2019).
[33]
Ilke Demir, Krzysztof Koperski, David Lindenbaum, Guan Pang, Jing Huang, Saikat Basu, Forest Hughes, Devis Tuia, and Ramesh Raskar. 2018. DeepGlobe 2018: A challenge to parse the earth through satellite images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. Computer Vision Foundation, Salt Lake City, Utah, 172–181.
[34]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Miama, FL, USA, 248–255.
[35]
Matthias Drusch, Umberto Del Bello, Sébastien Carlier, Olivier Colin, Veronica Fernandez, Ferran Gascon, Bianca Hoersch, Claudia Isola, Paolo Laberinti, Philippe Martimort, et al. 2012. Sentinel-2: ESA’s optical high-resolution mission for GMES operational services. Remote sensing of Environment 120 (2012), 25–36.
[36]
Burak Ekim, Timo T Stomberg, Ribana Roscher, and Michael Schmitt. 2023. MapInWild: A remote sensing dataset to address the question of what makes nature wild. IEEE Geoscience and Remote Sensing Magazine 11, 1 (2023), 103–114.
[37]
Haoqi Fan, Tullie Murrell, Heng Wang, Kalyan Vasudev Alwala, Yanghao Li, Yilei Li, Bo Xiong, Nikhila Ravi, Meng Li, Haichuan Yang, et al. 2021. PyTorchVideo: A deep learning library for video understanding. In Proceedings of the 29th ACM International Conference on Multimedia. ACM, Chengdu, China, 3783–3786.
[38]
Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428(2019).
[39]
Lewis Fishgold et al. 2017. Raster Vision: An open source library and framework for deep learning on satellite and aerial imagery (2017-2023).Azavea/Element 84, Robert Cheetham. https://doi.org/10.5281/zenodo.8018177
[40]
Steve Foga, Pat L Scaramuzza, Song Guo, Zhe Zhu, Ronald D Dilley Jr, Tim Beckmann, Gail L Schmidt, John L Dwyer, M Joseph Hughes, and Brady Laue. 2017. Cloud detection algorithm comparison and validation for operational Landsat data products. Remote Sensing of Environment 194 (2017), 379–390.
[41]
Radiant Earth Foundation. 2020. CV4A Competition Kenya Crop Type Dataset. https://doi.org/10.34911/RDNT.DW605X Version 1.0, Radiant MLHub.
[42]
Radiant Earth Foundation. 2022. Sentinel-2 Cloud Cover Segmentation Dataset (Version 1). https://doi.org/10.34911/rdnt.hfq6m7.
[43]
USDA Farm Service Agency (FSA). 2015. National Agriculture Imagery Program (NAIP). USDA Geospatial Data Gateway.
[44]
Martin Hermann Paul Fuchs and Begüm Demir. 2023. HySpecNet-11k: A large-scale hyperspectral dataset for benchmarking learning-based hyperspectral image compression methods. In IGARSS 2023-2023 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 1779–1782.
[45]
Anthony Fuller, Koreen Millard, and James Green. 2024. CROMA: Remote sensing representations with contrastive radar-optical masked autoencoders. Advances in Neural Information Processing Systems 36 (2024).
[46]
Todd Gamblin, Matthew LeGendre, Michael R Collette, Gregory L Lee, Adam Moody, Bronis R De Supinski, and Scott Futral. 2015. The Spack package manager: bringing order to HPC software chaos. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, Austin, Texas, 1–12.
[47]
Vivien Sainte Fare Garnot and Loic Landrieu. 2021. Panoptic segmentation of satellite image time series with convolutional temporal attention networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. Computer Vision Foundation, virtual conference, 4872–4881.
[48]
Vivien Sainte Fare Garnot, Loic Landrieu, and Nesrine Chehata. 2022. Multi-modal temporal attention models for crop mapping from satellite time series. ISPRS Journal of Photogrammetry and Remote Sensing 187 (2022), 294–305.
[49]
GBIF. 2020. GBIF: The Global Biodiversity Information Facility. https://www.gbif.org/
[50]
GDAL/OGR contributors. 2022. GDAL/OGR Geospatial Data Abstraction software Library. Open Source Geospatial Foundation. https://doi.org/10.5281/zenodo.5884351
[51]
Nora Gourmelon, Thorsten Seehaus, Matthias Braun, Andreas Maier, and Vincent Christlein. 2022. Calving Fronts and Where to Find Them: A Benchmark Dataset and Methodology for Automatic Glacier Calving Front Extraction from SAR Imagery. Earth System Science Data Discussions 2022 (2022), 1–37.
[52]
Sarah J. Graves, Sergio Marconi, Dylan Stewart, Ira Harmon, Ben G. Weinstein, Yuzi Kanazawa, Victoria M. Scholl, Maxwell B. Joseph, Joseph McClinchy, et al. 2021. Data science competition for cross-site delineation and classification of individual trees from airborne remote sensing data. bioRxiv (2021).
[53]
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, et al. 2020. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733(2020).
[54]
Rocchina Guarini, Rosa Loizzo, Claudia Facchinetti, Francesco Longo, Beatrice Ponticelli, Marco Faraci, Michele Dami, Massimo Cosi, Leonardo Amoruso, Vito De Pasquale, et al. 2018. PRISMA hyperspectral mission products. In IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Valencia, Spain, 179–182.
[55]
Ritwik Gupta, Richard Hosfelt, Sandra Sajeev, Nirav Patel, Bryce Goodman, Jigar Doshi, Eric Heim, Howie Choset, and Matthew Gaston. 2019. xBD: A Dataset for Assessing Building Damage from Satellite Imagery. arxiv:1911.09296 [cs.CV]
[56]
Iksha Gurung, Muthukumaran Ramasubhramanian, Brian Freitag, Aaron Kaulfus, Manil Maskey, Rahul Ramachandran, and Hamed Alemohammad. 2023. Tropical cyclone wind speed estimation: A large scale training data set and community benchmarking. Earth and Space Science 10, 3 (2023), e2022EA002693.
[57]
NASA Harvest, Radiant Earth Foundation, and TaQadam. 2022. Rwanda Field Boundary Competition Dataset (Version 1.0). https://doi.org/10.34911/RDNT.G580WW.
[58]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, virtual conference, 9729–9738.
[59]
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. EuroSAT: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 7 (2019), 2217–2226.
[60]
Dan Hendrycks, Kimin Lee, and Mantas Mazeika. 2019. Using pre-training can improve model robustness and uncertainty. In International Conference on Machine Learning. PMLR, Long Beach, California, 2712–2721.
[61]
Daniel J. Hofmann and Bhargav Kowshik. 2018. RoboSat: Semantic segmentation on aerial and satellite imagery. https://github.com/mapbox/robosat.
[62]
Collin H Homer, Joyce A Fry, Christopher A Barnes, et al. 2012. The national land cover database. US Geological Survey fact sheet 3020, 4 (2012), 1–4.
[63]
Di Hu, Xuhong Li, Lichao Mou, Pu Jin, Dong Chen, Liping Jing, Xiaoxiang Zhu, and Dejing Dou. 2020. Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition. In Computer Vision – ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 68–84.
[64]
Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. 2016. What makes ImageNet good for transfer learning?arXiv preprint arXiv:1608.08614(2016).
[65]
iNaturalist. 2023. iNaturalist. Available from https://www.inaturalist.org.
[66]
Neal Jean, Sherrie Wang, Anshul Samar, George Azzari, David Lobell, and Stefano Ermon. 2019. Tile2Vec: Unsupervised representation learning for spatially distributed data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI, Honolulu, Hawaii, 3967–3974.
[67]
Z. Jin, C. Lin, C. Weigl, J. Obarowski, and D. Hale. 2021. Smallholder Cashew Plantations in Benin. https://doi.org/10.34911/rdnt.hfv20i Radiant MLHub.
[68]
Krishna Karra, Caitlin Kontgis, Zoe Statman-Weil, Joseph C Mazzariello, Mark Mathis, and Steven P Brumby. 2021. Global land use/land cover with Sentinel 2 and deep learning. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. IEEE, Brussels, Belgium, 4704–4707.
[69]
Hannah Kerner, Snehal Chaudhari, Aninda Ghosh, Caleb Robinson, Adeel Ahmad, Eddie Choi, Nathan Jacobs, Chris Holmes, Matthias Mohr, Rahul Dodhia, et al. 2024. Fields of The World: A Machine Learning Benchmark Dataset For Global Agricultural Field Boundary Segmentation. arXiv preprint arXiv:2409.16252(2024).
[70]
Asanobu Kitamoto, Jared Hwang, Bastien Vuillod, Lucas Gautier, Yingtao Tian, and Tarin Clanuwat. 2024. Digital Typhoon: Long-term satellite image dataset for the spatio-temporal modeling of tropical cyclones. Advances in Neural Information Processing Systems 36 (2024).
[71]
Dominik Koßmann, Viktor Brack, and Thorsten Wilhelm. 2022. Seasonet: A seasonal scene classification, segmentation and retrieval dataset for satellite imagery over Germany. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Kuala Lumpur, Malaysia, 243–246.
[72]
Wenjuan Li, Alexis Comar, Marie Weiss, Sylvain Jay, Gallian Colombeau, Raul Lopez-Lozano, Simon Madec, and Frédéric Baret. 2021. A double swath configuration for improving throughput and accuracy of trait estimate from UAV images. Plant Phenomics (2021).
[73]
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. 2018. Progressive neural architecture search. In Proceedings of the European Conference on Computer Vision (ECCV). Computer Vision Foundation, Munich, Germany, 19–34.
[74]
Yang Long, Gui-Song Xia, Shengyang Li, Wen Yang, Michael Ying Yang, Xiao Xiang Zhu, Liangpei Zhang, and Deren Li. 2021. On creating benchmark dataset for aerial image interpretation: Reviews, guidances, and Million-AID. IEEE Journal of selected topics in applied earth observations and remote sensing 14(2021), 4205–4230.
[75]
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101(2017).
[76]
Dengsheng Lu, Qi Chen, Guangxing Wang, Lijuan Liu, Guiying Li, and Emilio Moran. 2016. A survey of remote sensing-based aboveground biomass estimation methods in forest ecosystems. International Journal of Digital Earth 9, 1 (2016), 63–105.
[77]
Emmanuel Maggiori, Yuliya Tarabalka, Guillaume Charpiat, and Pierre Alliez. 2017. Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, Fort Worth, TX, USA, 3226–3229.
[78]
Oscar Mañas, Alexandre Lacoste, Xavier Giro-i Nieto, David Vazquez, and Pau Rodriguez. 2021. Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data. arXiv preprint arXiv:2103.16607(2021).
[79]
Diego Marcos, Michele Volpi, Benjamin Kellenberger, and Devis Tuia. 2018. Land cover mapping at very high resolution with rotation equivariant CNNs: Towards small yet accurate models. ISPRS Journal of Photogrammetry and Remote Sensing 145 (2018), 96–107.
[80]
Manil Maskey, Rahul Ramachandran, Muthukumaran Ramasubramanian, Iksha Gurung, Brian Freitag, Aaron Kaulfus, Drew Bollinger, Daniel J Cecil, and Jeffrey Miller. 2020. Deepti: Deep-learning-based tropical cyclone intensity estimation system. IEEE journal of selected topics in applied Earth observations and remote sensing 13(2020), 4271–4281.
[81]
Francisco Massa et al. 2016. torchvision: Datasets, Transforms and Models specific to Computer Vision. https://github.com/pytorch/vision
[82]
Stuart K McFeeters. 1996. The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features. International journal of remote sensing 17, 7 (1996), 1425–1432.
[83]
David J. Mulla. 2013. Twenty five years of remote sensing in precision agriculture: Key advances and remaining knowledge gaps. Biosystems Engineering 114, 4 (2013), 358–371. https://doi.org/10.1016/j.biosystemseng.2012.08.009 Special Issue: Sensing Technologies for Sustainable Agriculture.
[84]
T Nathan Mundhenk, Goran Konjevod, Wesam A Sakla, and Kofi Boakye. 2016. A large contextual dataset for classification, detection and counting of cars with deep learning. In European Conference on Computer Vision. Springer, Amsterdam, Netherlands, 785–800.
[85]
NASA-IMPACT. 2021. ETCI 2021 Competition on Flood Detection. https://nasa-impact.github.io/etci2021/
[86]
Andrea Nascetti, Ritu Yadav, Kirill Brodt, Qixun Qu, Hongwei Fan, Yuri Shendryk, Isha Shah, and Christine Chung. 2023. BioMassters: A Benchmark Dataset for Forest Biomass Estimation using Multi-modal Satellite Time-series. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Curran Associates, Inc., New Orleans, Louisiana.
[87]
Vishal Nedungadi, Ankit Kariryaa, Stefan Oehmcke, Serge Belongie, Christian Igel, and Nico Lang. 2024. MMEarth: Exploring multi-modal pretext tasks for geospatial representation learning. arXiv preprint arXiv:2405.02771(2024).
[88]
Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, and Neil Houlsby. 2019. In-domain representation learning for remote sensing. arXiv preprint arXiv:1911.06721(2019).
[89]
Yuhao Nie, Xiatong Li, Andea Scott, Yuchi Sun, Vignesh Venugopal, and Adam Brandt. 2023. SKIPP’D: A SKy Images and Photovoltaic Power Generation Dataset for short-term solar forecasting. Solar Energy 255(2023), 171–179.
[90]
NOAA Office of Satellite and Product Operations. 1994. NOAA Geostationary Operational Environmental Satellite (GOES) I-M and N-P Series Imager Data. NOAA National Centers for Environmental Information. https://doi.org/10.25921/Z9JQ-K976
[91]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., Vancouver, Canada, 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
[92]
Jay S Pearlman, Pamela S Barry, Carol C Segal, John Shepanski, Debra Beiso, and Stephen L Carman. 2003. Hyperion, a space-based imaging spectrometer. IEEE Transactions on Geoscience and Remote Sensing 41, 6 (2003), 1160–1173.
[93]
Radiant Earth Foundation and IDinsight. 2022. AgriFieldNet Competition Dataset. https://doi.org/10.34911/rdnt.wu92p1 Version 1.0.
[94]
Krishna Rao, A Park Williams, Jacqueline Fortin Flefil, and Alexandra G Konings. 2020. SAR-enhanced mapping of live fuel moisture content. Remote Sensing of Environment 245 (2020), 111797.
[95]
Colorado J Reed, Ritwik Gupta, Shufan Li, Sarah Brockman, Christopher Funk, Brian Clipp, Kurt Keutzer, Salvatore Candido, Matt Uyttendaele, and Trevor Darrell. 2023. Scale-MAE: A scale-aware masked autoencoder for multiscale geospatial representation learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4088–4099.
[96]
Gyri Reiersen, David Dao, Björn Lütjens, Konstantin Klemmer, Kenza Amara, Attila Steinegger, Ce Zhang, and Xiaoxiang Zhu. 2022. ReforesTree: A dataset for estimating tropical forest carbon stock with deep learning and aerial imagery. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. PKP Publishing Services, virtual conference, 12119–12125.
[97]
Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski. 2020. Kornia: An open source differentiable computer vision library for PyTorch. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Computer Vision Foundation, Snowmass Village, CO, USA, 3674–3683.
[98]
George A Riggs, Dorothy K Hall, and Vincent V Salomonson. 1994. A snow index for the Landsat thematic mapper and moderate resolution imaging spectroradiometer. In Proceedings of IGARSS’94-1994 IEEE International Geoscience and Remote Sensing Symposium, Vol. 4. IEEE, Pasadena, CA, USA, 1942–1944.
[99]
Caleb Robinson, Le Hou, Kolya Malkin, Rachel Soobitsky, Jacob Czawlytko, Bistra Dilkina, and Nebojsa Jojic. 2019. Large Scale High-Resolution Land Cover Mapping With Multi-Resolution Data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, Long Beach, California, 12726–12735.
[100]
John Rogan, Janet Franklin, Doug Stow, Jennifer Miller, Curtis Woodcock, and Dar Roberts. 2008. Mapping land-cover modifications over large areas: A comparison of machine learning algorithms. Remote Sensing of Environment 112, 5 (2008), 2272–2283.
[101]
Esther Rolf, Nikolay Malkin, Alexandros Graikos, Ana Jojic, Caleb Robinson, and Nebojsa Jojic. 2021. Resolving label uncertainty with implicit generative models. (2021).
[102]
Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, and Solomon Hsiang. 2021. A generalizable and accessible approach to machine learning with global satellite imagery. Nature communications 12, 1 (2021), 1–11.
[103]
David Rolnick, Priya L Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, et al. 2019. Tackling climate change with machine learning. arXiv preprint arXiv:1906.05433(2019).
[104]
Franz Rottensteiner, Gunho Sohn, Jaewook Jung, Markus Gerke, Caroline Baillard, Sebastien Benitez, and Uwe Breitkopf. 2012. The ISPRS benchmark on urban object classification and 3D building reconstruction. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences I-3 (2012), Nr. 1 1, 1 (2012), 293–298.
[105]
David P Roy, Michael A Wulder, Thomas R Loveland, Curtis E Woodcock, Richard G Allen, Martha C Anderson, Dennis Helder, James R Irons, David M Johnson, Robert Kennedy, et al. 2014. Landsat-8: Science and product vision for terrestrial global change research. Remote sensing of Environment 145 (2014), 154–172.
[106]
Maurizio Santoro. 2018. GlobBiomass - global datasets of forest biomass. https://doi.org/10.1594/PANGAEA.894711
[107]
Pasquale L Scaramuzza, Michelle A Bouchard, and John L Dwyer. 2011. Development of the Landsat data continuity mission cloud-cover assessment algorithms. IEEE Transactions on Geoscience and Remote Sensing 50, 4 (2011), 1140–1154.
[108]
Michael Schmitt, Lloyd Haydn Hughes, Chunping Qiu, and Xiao Xiang Zhu. 2019. SEN12MS–A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion. arXiv preprint arXiv:1906.07789(2019).
[109]
Maja Schneider, Tobias Schelte, Felix Schmitz, and Marco Körner. 2023. EuroCrops: The Largest Harmonized Open Crop Dataset Across the European Union. Scientific Data 10, 1 (Sept. 2023), 612. https://doi.org/10.1038/s41597-023-02517-0
[110]
A Shah, T Lillianne, and M Manil. 2021. Marine Debris Dataset for Object Detection in Planetscope Imagery. URL: https://doi. org/10.34911/rdnt. 9r6ekg(2021).
[111]
Li Shen, Yao Lu, Hao Chen, Hao Wei, Donghai Xie, Jiabao Yue, Rui Chen, Yue Zhang, Ao Zhang, Shouye Lv, et al. 2021. S2Looking: A Satellite Side-Looking Dataset for Building Change Detection. arXiv preprint arXiv:2107.09244(2021).
[112]
Shuchang Shen, Sachith Seneviratne, Xinye Wanyan, and Michael Kirley. 2023. FireRisk: A Remote Sensing Dataset for Fire Risk Assessment with Benchmarks Using Supervised and Self-supervised Learning. arXiv preprint arXiv:2303.07035(2023).
[113]
M. Simard, T. Fatoyinbo, C. Smetanka, V. H. Rivera-Monroy, E. Castaneda-Mova, N. Thomas, and T. Van der Stocken. 2019. Global Mangrove Distribution, Aboveground Biomass, and Canopy Height. https://doi.org/10.3334/ORNLDAAC/1665
[114]
Rolf Simoes, Gilberto Camara, Gilberto Queiroz, Felipe Souza, Pedro R Andrade, Lorena Santos, Alexandre Carvalho, and Karine Ferreira. 2021. Satellite image time series analysis for big earth observation data. Remote Sensing 13, 13 (2021), 2428.
[115]
Wojciech Sirko, Sergii Kashubin, Marvin Ritter, Abigail Annkah, Yasser Salah Eddine Bouchareb, Yann Dauphin, Daniel Keysers, Maxim Neumann, Moustapha Cisse, and John Quinn. 2021. Continental-scale building detection from high resolution satellite imagery. arXiv preprint arXiv:2107.12283(2021).
[116]
Xiao-Peng Song, Matthew C Hansen, Peter Potapov, Bernard Adusei, Jeffrey Pickering, Marcos Adami, Andre Lima, Viviana Zalles, Stephen V Stehman, Carlos M Di Bella, et al. 2021. Massive soybean expansion in South America since 2000 and implications for conservation. Nature sustainability 4, 9 (2021), 784–792.
[117]
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. 2021. How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers. arXiv preprint arXiv:2106.10270(2021).
[118]
Adam Stewart, Nils Lehmann, Isaac Corley, Yi Wang, Yi-Chia Chang, Nassim Ait Ali Braham, Shradha Sehgal, Caleb Robinson, and Arindam Banerjee. 2024. SSL4EO-L: Datasets and foundation models for Landsat imagery. Advances in Neural Information Processing Systems 36 (2024).
[119]
Adam J. Stewart, Caleb Robinson, Isaac A. Corley, Anthony Ortiz, Juan M. Lavista Ferres, and Arindam Banerjee. 2022. TorchGeo: Deep Learning With Geospatial Data. In Proceedings of the 30th International Conference on Advances in Geographic Information Systems (SIGSPATIAL ’22). Association for Computing Machinery, Seattle, Washington, 1–12. https://doi.org/10.1145/3557915.3560953
[120]
Gencer Sumbul, Marcela Charfuelan, Begüm Demir, and Volker Markl. 2019. BigEarthNet: A large-scale benchmark archive for remote sensing image understanding. In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium. IEEE, Yokohama, Japan, 5901–5904.
[121]
Timu Sumisu. 2019. Canadian Building Footprints. https://github.com/microsoft/CanadianBuildingFootprints
[122]
Xian Sun, Peijin Wang, Zhiyuan Yan, Feng Xu, Ruiping Wang, Wenhui Diao, Jin Chen, Jihao Li, Yingchao Feng, Tao Xu, Martin Weinmann, Stefan Hinz, Cheng Wang, and Kun Fu. 2021. FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery.
[123]
Yonatan Tarazona, Fernando Benitez-Paez, Jakub Nowosad, Fabian Drenkhan, and Martín E Timaná. 2024. scikit-eo: A Python package for Remote Sensing Data Analysis. Journal of Open Source Software 9, 99 (2024), 6692.
[124]
Maxar Technologies. 2021. WorldView satellite imagery.
[125]
TensorFlow. 2021. TensorFlow Datasets, A collection of ready-to-use datasets. https://www.tensorflow.org/datasets
[126]
Xin-Yi Tong, Gui-Song Xia, Qikai Lu, Huanfeng Shen, Shengyang Li, Shucheng You, and Liangpei Zhang. 2020. Land-cover classification with high-resolution remote sensing images using transferable deep models. Remote Sensing of Environment 237 (2020), 111322.
[127]
Gabriel Tseng, Ivan Zvonkov, Catherine Lilian Nakalembe, and Hannah Kerner. 2021. CropHarvest: A global dataset for crop-type classification. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
[128]
Compton J Tucker. 1979. Red and photographic infrared linear combinations for monitoring vegetation. Remote sensing of Environment 8, 2 (1979), 127–150.
[129]
Mehmet Ozgur Turkoglu, Stefano D’Aronco, Gregor Perich, Frank Liebisch, Constantin Streit, Konrad Schindler, and Jan Dirk Wegner. 2021. Crop mapping from image time series: Deep learning with multi-scale label hierarchies. Remote Sensing of Environment 264 (2021), 112603.
[130]
Adam Van Etten, Dave Lindenbaum, and Todd M Bacastow. 2018. SpaceNet: A remote sensing dataset and challenge series. arXiv preprint arXiv:1807.01232(2018).
[131]
Cees J Van Westen. 2013. Remote sensing and GIS for natural hazards assessment and disaster risk management. Treatise on geomorphology 3 (2013), 259–298.
[132]
Anna X Wang, Caelin Tran, Nikhil Desai, David Lobell, and Stefano Ermon. 2018. Deep transfer learning for crop yield prediction with remote sensing data. In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies. ACM, Menlo Park and San Jose, California, 1–5.
[133]
Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zhong. 2021. LoveDA: A remote sensing land-cover dataset for domain adaptive semantic segmentation. arXiv preprint arXiv:2110.08733(2021).
[134]
Shuai Wang. 2019. TorchSat. https://github.com/sshuair/torchsat
[135]
Yi Wang, Conrad M Albrecht, Nassim Ait Ali Braham, Chenying Liu, Zhitong Xiong, and Xiao Xiang Zhu. 2024. Decoupling common and unique representations for multimodal self-supervised learning. In 18th European Conference on Computer Vision, ECCV 2024. Springer, 1–19.
[136]
Yi Wang, Nassim Ait Ali Braham, Zhitong Xiong, Chenying Liu, Conrad M Albrecht, and Xiao Xiang Zhu. 2023. SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation. IEEE Geoscience and Remote Sensing Magazine 11, 3 (2023), 98–106.
[137]
Zhecheng Wang, Rajanie Prabha, Tianyuan Huang, Jiajun Wu, and Ram Rajagopal. 2024. SkyScript: A large and semantically diverse vision-language dataset for remote sensing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 5805–5813.
[138]
Global Forest Watch. 2002. Global forest watch. World Resources Institute, Washington, DC Available from http://www. globalforestwatch. org (accessed March 2002)(2002).
[139]
Ben G Weinstein, Sergio Marconi, Stephanie Bohlman, Alina Zare, and Ethan White. 2019. Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks. Remote Sensing 11, 11 (2019), 1309.
[140]
Western Cape Department of Agricultureand Radiant Earth Foundation. 2021. Crop Type Classification Dataset for Western Cape, South Africa. https://doi.org/10.34911/rdnt.j0co8q Version 1.0.
[141]
Falcon William and The PyTorch Lightning team. 2019. PyTorch Lightning. https://doi.org/10.5281/zenodo.3828935
[142]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. HuggingFace’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771(2019).
[143]
CosmiQ Works. 2019. Solaris. https://github.com/CosmiQ/solaris
[144]
Qiusheng Wu and Lucas Prado Osco. 2023. samgeo: A Python package for segmenting geospatial data with the Segment Anything Model (SAM). Journal of Open Source Software 8, 89 (Sept. 2023), 5663. https://doi.org/10.21105/joss.05663
[145]
Zhitong Xiong, Yi Wang, Fahong Zhang, Adam J Stewart, Joëlle Hanna, Damian Borth, Ioannis Papoutsis, Bertrand Le Saux, Gustau Camps-Valls, and Xiao Xiang Zhu. 2024. Neural plasticity-inspired foundation model for observing the Earth crossing modalities. arXiv e-prints (2024), arXiv–2403.
[146]
Yi Yang and Shawn Newsam. 2010. Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, San Jose, California, 270–279.
[147]
Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Anjali Chourdia, Artyom Astafurov, Caroline Chen, Ching-Feng Yeh, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Edward Z. Yang, Jason Lian, Jay Mahadeokar, Jeff Hwang, Ji Chen, Peter Goldsborough, Prabhat Roy, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville-Bélair, and Yangyang Shi. 2021. TorchAudio: Building Blocks for Audio and Speech Processing. arXiv preprint arXiv:2110.15018(2021).
[148]
Christopher Yeh, Chenlin Meng, Sherrie Wang, Anne Driscoll, Erik Rozi, Patrick Liu, Jihyeon Lee, Marshall Burke, David B Lobell, and Stefano Ermon. 2021. SustainBench: Benchmarks for monitoring the sustainable development goals with machine learning. arXiv preprint arXiv:2111.04724(2021).
[149]
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks?arXiv preprint arXiv:1411.1792(2014).
[150]
Jiaxuan You, Xiaocheng Li, Melvin Low, David Lobell, and Stefano Ermon. 2017. Deep Gaussian process for crop yield prediction based on remote sensing data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. PKP Publishing Services, San Francisco, California.
[151]
Nanshan You, Jinwei Dong, Jianxi Huang, Guoming Du, Geli Zhang, Yingli He, Tong Yang, Yuanyuan Di, and Xiangming Xiao. 2021. The 10-m crop type maps in Northeast China during 2017–2019. Scientific data 8, 1 (2021), 41.
[152]
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. 2019. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867(2019).
[153]
Guanheng Zhang et al. 2016. torchtext: Data loaders and abstractions for text and NLP. https://github.com/pytorch/text
[154]
Jixian Zhang. 2010. Multi-source remote sensing data fusion: status and trends. International Journal of Image and Data Fusion 1, 1 (2010), 5–24.
[155]
Wei Zhang, Ping Tang, and Lijun Zhao. 2019. Remote sensing image scene classification using CNN-CapsNet. Remote Sensing 11, 5 (2019), 494.
[156]
Zhuo Zheng, Ailong Ma, Liangpei Zhang, and Yanfei Zhong. 2021. Change is everywhere: Single-temporal supervised object change detection in remote sensing imagery. In Proceedings of the IEEE/CVF International Conference on Computer Vision. Computer Vision Foundation, virtual conference, 15193–15202.
[157]
Zhuo Zheng, Yanfei Zhong, Junjue Wang, and Ailong Ma. 2020. Foreground-aware relation network for geospatial object segmentation in high spatial resolution remote sensing imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision Foundation, virtual conference, 4096–4105.
[158]
Weixun Zhou, Shawn Newsam, Congmin Li, and Zhenfeng Shao. 2018. PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS Journal of Photogrammetry and Remote Sensing 145 (2018), 197–209.
[159]
Xiao Xiang Zhu, Jingliang Hu, Chunping Qiu, Yilei Shi, Jian Kang, Lichao Mou, Hossein Bagheri, Matthias Häberle, Yuansheng Hua, Rong Huang, et al. 2019. So2Sat LCZ42: A benchmark dataset for global local climate zones classification. arXiv preprint arXiv:1912.12171(2019).
[160]
Xiao Xiang Zhu, Zhitong Xiong, Yi Wang, Adam J Stewart, Konrad Heidler, Yuanyuan Wang, Zhenghang Yuan, Thomas Dujardin, Qingsong Xu, and Yilei Shi. 2024. On the Foundations of Earth and Climate Foundation Models. arXiv preprint arXiv:2405.04285(2024).
Index Terms
- TorchGeo: Deep Learning With Geospatial Data
Recommendations
TorchGeo: deep learning with geospatial data
SIGSPATIAL '22: Proceedings of the 30th International Conference on Advances in Geographic Information SystemsRemotely sensed geospatial data are critical for applications including precision agriculture, urban planning, disaster monitoring and response, and climate change research, among others. Deep learning methods are particularly promising for modeling many ...
Monitoring crop phenology with street-level imagery using computer vision
Highlights- Method to automatically collect side looking imagery for agriculture parcels.
- Crop types and phenology obtained from computer vision.
- Demonstration over monthly surveyed totalling 400,000 pictures for 17 crop types.
- Deep ...
AbstractStreet-level imagery holds a significant potential to scale-up in-situ data collection. This is enabled by combining the use of cheap high-quality cameras with recent advances in deep learning compute solutions to derive relevant thematic ...
Comparing satellites and vegetation indices for cover crop biomass estimation
Highlights- Adequate biomass is required for cover crops to provide significant ecological benefits.
AbstractCost-share programs based on measures of participation rather than performance are available to farmers who plant cover crops. However, cover crops only provide significant ecological benefits like reduced nutrient loss when adequate ...
Comments
Information & Contributors
Information
Published In
EISSN:2374-0361
Table of ContentsCopyright © 2024 Copyright held by the owner/author(s).
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Online AM: 16 December 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 64Total Downloads
- Downloads (Last 12 months)64
- Downloads (Last 6 weeks)64
Reflects downloads up to 28 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in