RIRNet: A Direction-Guided Post-Processing Network for Road Information Reasoning
Abstract
:1. Introduction
2. Related Works
2.1. Deep Learning-Based Road Extraction Methods
2.2. Road Connectivity Modeling
3. Methodology
3.1. Direction-Guided Information Inference Module
3.2. Road Direction Inference Task Branch
Algorithm 1. Road direction label generation |
; ; ; |
do then do end for that: |
else: invalid |
else if end for |
3.3. Loss Function
4. Experiments and Result Analysis
4.1. Datasets
4.1.1. Massachusetts Dataset
4.1.2. DeepGlobe Dataset
4.1.3. CHN6-CUG Dataset
4.1.4. Anyi County Dataset
4.2. Experimental Settings
4.3. Evaluation Indicators
4.4. Evaluation of Results
4.5. Ablation Experiments
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Lian, R.; Wang, W.; Mustafa, N.; Huang, L. Road Extraction Methods in high-Resolution Remote Sensing Images: A Comprehensive Review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 5489–5507. [Google Scholar] [CrossRef]
- Abdollahi, A.; Pradhan, B.; Shukla, N.; Chakraborty, S.; Alamri, A. Deep Learning Approaches Applied to Remote Sensing Datasets for Road Extraction: A State-Of-The-Art Review. Remote Sens. 2020, 12, 1444. [Google Scholar] [CrossRef]
- Chen, Z.; Wang, C.; Li, J.; Fan, W.; Du, J.; Zhong, B. Adaboost-Like End-to-End Multiple Lightweight U-Nets for Road Extraction from Optical Remote Sensing Images. Int. J. Appl. Earth Obs. Geoinf. 2021, 100, 102341. [Google Scholar] [CrossRef]
- Shan, B.; Fang, Y. A Cross Entropy Based Deep Neural Network Model for Road Extraction from Satellite Images. Entropy 2020, 22, 535. [Google Scholar] [CrossRef] [PubMed]
- Chen, S.-B.; Ji, Y.-X.; Tang, J.; Luo, B.; Wang, W.-Q.; Lv, K. DBRANet: Road Extraction by Dual-Branch Encoder and Regional Attention Decoder. IEEE Geosci. Remote Sens. Lett. 2021, 19, 1–5. [Google Scholar] [CrossRef]
- Yang, K.; Yi, J.; Chen, A.; Liu, J.; Chen, W. ConDinet++: Full-Scale Fusion Network Based on Conditional Dilated Convolution to Extract Roads From Remote Sensing Images. IEEE Geosci. Remote Sens. Lett. 2021, 19, 1–5. [Google Scholar] [CrossRef]
- Pan, H.; Jia, Y.; Lv, Z. An Adaptive Multifeature Method for Semiautomatic Road Extraction From High-Resolution Stereo Mapping Satellite Images. IEEE Geosci. Remote Sens. Lett. 2018, 16, 201–205. [Google Scholar] [CrossRef]
- Chen, Z.; Deng, L.; Luo, Y.; Li, D.; Junior, J.M.; Gonçalves, W.N.; Nurunnabi, A.A.M.; Li, J.; Wang, C.; Li, D. Road Extraction in Remote Sensing Data: A Survey. Int. J. Appl. Earth Obs. Geoinf. 2022, 112, 102833. [Google Scholar] [CrossRef]
- Wang, Y.; Seo, J.; Jeon, T. NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Nonlocal Operations. IEEE Geosci. Remote Sens. Lett. 2021, 19, 3000105. [Google Scholar] [CrossRef]
- Xu, Y.; Chen, H.; Du, C.; Li, J. MSACon: Mining Spatial Attention-Based Contextual Information for Road Extraction. IEEE Trans. Geosci. Remote Sens. 2021, 60, 5604317. [Google Scholar] [CrossRef]
- Wang, S.; Mu, X.; Yang, D.; He, H.; Zhao, P. Road Extraction from Remote Sensing Images Using the Inner Convolution Integrated Encoder-Decoder Network and Directional Conditional Random Fields. Remote Sens. 2021, 13, 465. [Google Scholar] [CrossRef]
- Lu, X.; Zhong, Y.; Zheng, Z.; Zhang, L. GAMSNet: Globally Aware Road Detection Network with Multi-Scale Residual Learning. ISPRS J. Photogramm. Remote Sens. 2021, 175, 340–352. [Google Scholar] [CrossRef]
- Cira, C.-I.; Alcarria, R.; Manso-Callejo, M.-Á.; Serradilla, F. A Deep Learning-Based Solution for Large-Scale Extraction of the Secondary Road Network from High-Resolution Aerial Orthoimagery. Appl. Sci. 2020, 10, 7272. [Google Scholar] [CrossRef]
- Wei, Y.; Zhang, K.; Ji, S. Simultaneous Road Surface and Centerline Extraction From Large-Scale Remote Sensing Images Using cnn-Based Segmentation and Tracing. IEEE Trans. Geosci. Remote Sens. 2020, 58, 8919–8931. [Google Scholar] [CrossRef]
- Zhou, M.; Sui, H.; Chen, S.; Wang, J.; Chen, X. BT-RoadNet: A Boundary and Topologically-Aware Neural Network for Road Extraction from High-Resolution Remote Sensing Imagery. ISPRS J. Photogramm. Remote Sens. 2020, 168, 288–306. [Google Scholar] [CrossRef]
- Zhu, Q.; Zhang, Y.; Wang, L.; Zhong, Y.; Guan, Q.; Lu, X.; Zhang, L.; Li, D. A Global Context-Aware and Batch-Independent Network for Road Extraction from VHR Satellite Imagery. ISPRS J. Photogramm. Remote Sens. 2021, 175, 353–365. [Google Scholar] [CrossRef]
- Jing, R.; Gong, Z.; Zhu, W.; Guan, H.; Zhao, W. Island road centerline extraction based on a multiscale united feature. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 3940–3953. [Google Scholar] [CrossRef]
- Li, J.; Meng, Y.; Dorjee, D.; Wei, X.; Zhang, Z.; Zhang, W. Automatic Road Extraction from Remote Sensing Imagery Using Ensemble Learning and Postprocessing. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 10535–10547. [Google Scholar] [CrossRef]
- Zhang, Z.; Liu, Q.; Wang, Y. Road extraction by deep residual u-net. IEEE Geosci. Remote Sens. Lett. 2018, 15, 749–753. [Google Scholar] [CrossRef]
- Yang, X.; Li, X.; Ye, Y.; Lau, R.Y.K.; Zhang, X.; Huang, X. Road Detection and Centerline Extraction Via Deep Recurrent Convolutional Neural Network U-Net. IEEE Trans. Geosci. Remote Sens. 2019, 57, 7209–7220. [Google Scholar] [CrossRef]
- Gao, X.; Sun, X.; Zhang, Y.; Yan, M.; Xu, G.; Sun, H.; Jiao, J.; Fu, K. An End-to-End Neural Network for Road Extraction From Remote Sensing Imagery by Multiple Feature Pyramid Network. IEEE Access 2018, 6, 39401–39414. [Google Scholar] [CrossRef]
- Li, X.; Wang, Y.; Zhang, L.; Liu, S.; Mei, J.; Li, Y. Topology-Enhanced Urban Road Extraction via a Geographic Feature-Enhanced Network. IEEE Trans. Geosci. Remote Sens. 2020, 58, 8819–8830. [Google Scholar] [CrossRef]
- Chen, J.; Yang, L.; Wang, H.; Zhu, J.; Sun, G.; Dai, X.; Deng, M.; Shi, Y. Road Extraction from High-Resolution Remote Sensing Images via Local and Global Context Reasoning. Remote Sens. 2023, 15, 4177. [Google Scholar] [CrossRef]
- Cheng, G.; Wang, Y.; Xu, S.; Wang, H.; Xiang, S.; Pan, C. Automatic Road Detection and Centerline Extraction via Cascaded End-to-End Convolutional Neural Network. IEEE Trans. Geosci. Remote Sens. 2017, 55, 3322–3337. [Google Scholar] [CrossRef]
- Liu, Y.; Yao, J.; Lu, X.; Xia, M.; Wang, X.; Liu, Y. RoadNet: Learning to Comprehensively Analyze Road Networks in Complex Urban Scenes From High-Resolution Remotely Sensed Images. IEEE Trans. Geosci. Remote Sens. 2018, 57, 2043–2056. [Google Scholar] [CrossRef]
- Lu, X.; Zhong, Y.; Zheng, Z.; Liu, Y.; Zhao, J.; Ma, A.; Yang, J. Multi-Scale and Multi-Task Deep Learning Framework for Automatic Road Extraction. IEEE Trans. Geosci. Remote Sens. 2019, 57, 9362–9377. [Google Scholar] [CrossRef]
- Zhou, L.; Zhang, C.; Wu, M. D-LinkNet: LinkNet With Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2018, Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar] [CrossRef]
- Wu, Q.; Luo, F.; Wu, P.; Wang, B.; Yang, H.; Wu, Y. Automatic Road Extraction from High-Resolution Remote Sensing Images Using a Method Based on Densely Connected Spatial Feature-Enhanced Pyramid. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 14, 3–17. [Google Scholar] [CrossRef]
- Luo, L.; Wang, J.-X.; Chen, S.-B.; Tang, J.; Luo, B. BDTNet: Road Extraction by Bi-Direction Transformer From Remote Sensing Images. IEEE Geosci. Remote Sens. Lett. 2022, 19, 2505605. [Google Scholar] [CrossRef]
- Liu, X.; Wang, Z.; Wan, J.; Zhang, J.; Xi, Y.; Liu, R.; Miao, Q. RoadFormer: Road Extraction Using a Swin Transformer Combined with a Spatial and Channel Separable Convolution. Remote Sens. 2023, 15, 1049. [Google Scholar] [CrossRef]
- Ding, C.; Weng, L.; Xia, M.; Lin, H. Non-Local Feature Search Network for Building and Road Segmentation of Remote Sensing Image. ISPRS Int. J. Geo-Inf. 2021, 10, 245. [Google Scholar] [CrossRef]
- Zhao, H.; Zhang, H.; Zheng, X. RFE-LinkNet: LinkNet with Receptive Field Enhancement for Road Extraction from High Spatial Resolution Imagery. IEEE Access 2023, 11, 106412–106422. [Google Scholar] [CrossRef]
- Gao, L.; Song, W.; Dai, J.; Chen, Y. Road Extraction from High-Resolution Remote Sensing Imagery Using Refined Deep Residual Convolutional Neural Network. Remote Sens. 2019, 11, 552. [Google Scholar] [CrossRef]
- Ding, L.; Bruzzone, L. DiResNet: Direction-Aware Residual Network for Road Extraction in VHR Remote Sensing Images. IEEE Trans. Geosci. Remote Sens. 2020, 59, 10243–10254. [Google Scholar] [CrossRef]
- Tao, C.; Qi, J.; Li, Y.; Wang, H.; Li, H. Spatial Information Inference Net: Road extraction Using Road-Specific Contextual Information. ISPRS J. Photogramm. Remote Sens. 2019, 158, 155–166. [Google Scholar] [CrossRef]
- Pan, X.; Shi, J.; Luo, P.; Wang, X.; Tang, X. Spatial as Deep: Spatial CNN for Traffic Scene Understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018. [Google Scholar]
- Volodymyr, M. Machine Learning for Aerial Image Labeling; University of Toronto: Toronto, ON, Canada, 2013. [Google Scholar]
- Demir, I.; Koperski, K.; Lindenbaum, D.; Pang, G.; Huang, J.; Basu, S.; Hughes, F.; Tuia, D.; Raskar, R. Deepglobe 2018: A Challenge to Parse the Earth through Satellite Images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2018, Salt Lake City, UT, USA, 18–22 June 2018. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015. [Google Scholar]
- Chen, L.C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018. [Google Scholar]
- Yuan, Y.; Huang, L.; Guo, J.; Zhang, C.; Chen, X.; Wang, J. Ocnet: Object Context Network for Scene Parsing. arXiv 2018, arXiv:1809.00916. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual Attention Network for Scene Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019. [Google Scholar]
Post-Processing Method | Road Extraction Methods | F1 (%) | IoU (%) |
---|---|---|---|
RIRNet | UNet [39] | 77.66 (+0.77) | 63.48 (+1.02) |
DeepLabV3+ [40] | 75.60 (+0.34) | 60.77 (+0.43) | |
OCNet [41] | 77.91 (+6.23) | 62.96 (+7.1) | |
DANet [42] | 75.99 (+9.04) | 61.27 (+10.95) | |
ResUNet [19] | 78.00 (+1.11) | 63.94 (+1.49) | |
RFE-LinkNet [32] | 80.38 (+3.81) | 70.86 (+2.59) |
Post-Processing Method | Road Extraction Methods | F1 (%) | IoU (%) |
---|---|---|---|
RIRNet | UNet [39] | 74.63 (+1.15) | 59.52 (+1.45) |
DeepLabV3+ [40] | 75.09 (+2.63) | 60.11 (+3.29) | |
OCNet [41] | 76.98 (+4.83) | 62.57 (+6.14) | |
DANet [42] | 74.42 (+3.96) | 59.26 (+4.86) | |
ResUNet [19] | 78.12 (+5.02) | 64.09 (+6.49) | |
RFE-LinkNet [32] | 83.08 (+2.20) | 74.23 (+2.75) |
Post-Processing Method | Road Extraction Methods | F1 (%) | IoU (%) |
---|---|---|---|
RIRNet | UNet [39] | 75.34 (+2.74) | 60.43 (+3.45) |
DeepLabV3+ [40] | 76.52 (+0.94) | 61.98 (+0.71) | |
OCNet [41] | 76.72 (+0.83) | 62.23 (+1.09) | |
DANet [42] | 76.50 (+1.27) | 61.95 (+1.65) | |
ResUNet [19] | 77.93 (+6.53) | 63.85 (+8.33) | |
RFE-LinkNet [32] | 76.38 (+3.34) | 65.96 (+3.83) |
Post-Processing Method | Road Extraction Methods | F1 (%) | IoU (%) |
---|---|---|---|
RIRNet | UNet [39] | 67.87 (+3.16) | 58.74 (+2.19) |
DeepLabV3+ [40] | 74.84 (+1.47) | 64.44 (+1.16) | |
OCNet [41] | 69.95 (+4.96) | 60.77 (+4.66) | |
DANet [42] | 71.86 (+9.56) | 61.60 (+6.55) | |
ResUNet [19] | 72.56 (+1.74) | 62.86 (+1.84) | |
RFE-LinkNet [32] | 74.16 (+0.93) | 64.61 (+0.69) |
UNet | DeepLabV3+ | DANet | ResUNet | RFE-LinkNet | RIRNet | |
---|---|---|---|---|---|---|
Params (Mb) | 13.40 | 59.44 | 54.36 | 13.04 | 34.29 | 2.85 |
FLOPs (Gbps) 1 | 124.36 | 90.35 | 313.70 | 323.73 | 40.84 | 42.88 |
Road Direction Inference Task | Information Reasoning Module | F1 (%) | IoU (%) |
---|---|---|---|
× | × | 77.11 (−3.77) | 67.45 (−4.20) |
√ | × | 81.42 (+0.54) | 72.20 (+0.73) |
× | √ | 82.51 (+1.63) | 73.24 (+1.77) |
√ | √ | 83.08 (+2.20) | 74.23 (+2.75) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhou, G.; He, C.; Wang, H.; Xie, Q.; Chen, Q.; Hong, L.; Chen, J. RIRNet: A Direction-Guided Post-Processing Network for Road Information Reasoning. Remote Sens. 2024, 16, 2666. https://doi.org/10.3390/rs16142666
Zhou G, He C, Wang H, Xie Q, Chen Q, Hong L, Chen J. RIRNet: A Direction-Guided Post-Processing Network for Road Information Reasoning. Remote Sensing. 2024; 16(14):2666. https://doi.org/10.3390/rs16142666
Chicago/Turabian StyleZhou, Guoyuan, Changxian He, Hao Wang, Qiuchang Xie, Qiong Chen, Liang Hong, and Jie Chen. 2024. "RIRNet: A Direction-Guided Post-Processing Network for Road Information Reasoning" Remote Sensing 16, no. 14: 2666. https://doi.org/10.3390/rs16142666