Efficient 3D Reconstruction of Multiple Plants from UAV Images with Deep Learning
Pages 369 - 375
Abstract
Acquiring the 3D structure of plants is a critical task in the agricultural industry. Existing methods of generating 3D point clouds for multiple plants require a long processing time. In this paper, a 3D reconstruction method for numerous plants is proposed. Firstly, camera parameters in different viewpoints are obtained from the aerial image of plants by incremental structure from motion. Subsequently, the learning-based multi-view stereo takes images and the corresponding camera parameters as inputs to acquire initial depth maps. Finally, the depth maps are filtered and fused to produce a complete and dense 3D point cloud. We conducted experiments on an agricultural orchard dataset to compare with other methods. Experimental results demonstrate that our method reconstructs point clouds of plants with good quality while having a lower running time.
References
[1]
Henrik Aanæs, Rasmus Ramsbøl Jensen, George Vogiatzis, Engin Tola, and Anders Bjorholm Dahl. 2016. Large-scale data for multiple-view stereopsis. International Journal of Computer Vision 120 (2016), 153–168.
[2]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3 (2009), 24.
[3]
EF Berra and MV Peppa. 2020. Advances and challenges of UAV SFM MVS photogrammetry and remote sensing: Short review. In 2020 IEEE Latin American GRSS & ISPRS Remote Sensing Conference (LAGIRS). IEEE, 533–538.
[4]
Zhen Chen, Hui Lv, Lu Lou, and John H Doonan. 2022. Fast and accurate 3D reconstruction of plants using mvsnet and multi-view images. In Advances in Computational Intelligence Systems: Contributions Presented at the 20th UK Workshop on Computational Intelligence, September 8-10, 2021, Aberystwyth, Wales, UK 20. Springer, 390–399.
[5]
Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong Tan, and Ping Tan. 2020. Cascade cost volume for high-resolution multi-view stereo and stereo matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2495–2504.
[6]
Xian-Feng Han, Hamid Laga, and Mohammed Bennamoun. 2019. Image-based 3D object reconstruction: State-of-the-art and trends in the deep learning era. IEEE transactions on pattern analysis and machine intelligence 43, 5 (2019), 1578–1604.
[7]
Jakob Iglhaut, Carlos Cabo, Stefano Puliti, Livia Piermattei, James O’Connor, and Jacqueline Rosette. 2019. Structure from motion photogrammetry in forestry: A review. Current Forestry Reports 5 (2019), 155–168.
[8]
San Jiang, Wanshou Jiang, and Lizhe Wang. 2021. Unmanned aerial vehicle-based photogrammetric 3d mapping: A survey of techniques, applications, and challenges. IEEE Geoscience and Remote Sensing Magazine 10, 2 (2021), 135–171.
[9]
Chenhui Lin, Hong Wang, Chengliang Liu, and Liang Gong. 2020. 3D reconstruction based plant-monitoring and plant-phenotyping platform. In 2020 3rd World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM). IEEE, 522–526.
[10]
Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2117–2125.
[11]
Fusang Liu, Pengcheng Hu, Bangyou Zheng, Tao Duan, Binglin Zhu, and Yan Guo. 2021. A field-based high-throughput method for acquiring canopy architecture using unmanned aerial vehicle images. Agricultural and Forest Meteorology 296 (2021), 108231.
[12]
Elias Marks, Federico Magistri, and Cyrill Stachniss. 2022. Precise 3D reconstruction of plants from UAV imagery combining bundle adjustment and template matching. In 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2259–2265.
[13]
Paul Merrell, Amir Akbarzadeh, Liang Wang, Philippos Mordohai, Jan-Michael Frahm, Ruigang Yang, David Nistér, and Marc Pollefeys. 2007. Real-time visibility-based fusion of depth maps. In 2007 IEEE 11th International Conference on Computer Vision. Ieee, 1–8.
[14]
Seishi Ninomiya. 2022. High-throughput field crop phenotyping: current status and challenges. Breeding Science 72, 1 (2022), 3–18.
[15]
Niall O’Mahony, Sean Campbell, Lenka Krpalkova, Daniel Riordan, Joseph Walsh, Aidan Murphy, and Conor Ryan. 2019. Computer Vision for 3D Perception: A Review. In Intelligent Systems and Applications: Proceedings of the 2018 Intelligent Systems Conference (IntelliSys) Volume 2. Springer, 788–804.
[16]
Yeping Peng, Mingbin Yang, Genping Zhao, and Guangzhong Cao. 2021. Binocular-vision-based structure from motion for 3-D reconstruction of plants. IEEE Geoscience and Remote Sensing Letters 19 (2021), 1–5.
[17]
Jessada Phattaralerphong and Hervé Sinoquet. 2005. A method for 3D reconstruction of tree crown volume from photographs: assessment with 3D-digitized plants. Tree Physiology 25, 10 (2005), 1229–1242.
[18]
Worasit Sangjan, Rebecca J McGee, and Sindhuja Sankaran. 2022. Optimization of UAV-based imaging and image processing orthomosaic and point cloud approaches for estimating biomass in a forage crop. Remote Sensing 14, 10 (2022), 2396.
[19]
Johannes L Schonberger and Jan-Michael Frahm. 2016. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4104–4113.
[20]
Johannes L Schönberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. 2016. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. Springer, 501–518.
[21]
Sergio Vélez, Rubén Vacas, Hugo Martín, David Ruano-Rosa, and Sara Álvarez. 2022. High-Resolution UAV RGB Imagery Dataset for Precision Agriculture and 3D Photogrammetric Reconstruction Captured over a Pistachio Orchard (Pistacia vera L.) in Spain. Data 7, 11 (2022), 157.
[22]
Fangjinhua Wang, Silvano Galliani, Christoph Vogel, Pablo Speciale, and Marc Pollefeys. 2021. Patchmatchnet: Learned multi-view patchmatch stereo. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 14194–14203.
[23]
Xiang Wang, Chen Wang, Bing Liu, Xiaoqing Zhou, Liang Zhang, Jin Zheng, and Xiao Bai. 2021. Multi-view stereo in the deep learning era: A comprehensive review. Displays 70 (2021), 102102.
[24]
Huang Yao, Rongjun Qin, and Xiaoyu Chen. 2019. Unmanned aerial vehicle for remote sensing applications—A review. Remote Sensing 11, 12 (2019), 1443.
[25]
Yao Yao, Zixin Luo, Shiwei Li, Tian Fang, and Long Quan. 2018. Mvsnet: Depth inference for unstructured multi-view stereo. In Proceedings of the European conference on computer vision (ECCV). 767–783.
[26]
Yao Yao, Zixin Luo, Shiwei Li, Tianwei Shen, Tian Fang, and Long Quan. 2019. Recurrent mvsnet for high-resolution multi-view stereo depth inference. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5525–5534.
[27]
Yanchao Zhang, Hanxuan Wu, and Wen Yang. 2019. Forests growth monitoring based on tree canopy 3D reconstruction using UAV aerial photogrammetry. Forests 10, 12 (2019), 1052.
[28]
Genping Zhao, Weitao Cai, Zhuowei Wang, Heng Wu, Yeping Peng, and Lianglun Cheng. 2022. Phenotypic parameters estimation of plants using deep learning-based 3-D reconstruction from single RGB image. IEEE Geoscience and Remote Sensing Letters 19 (2022), 1–5.
Index Terms
- Efficient 3D Reconstruction of Multiple Plants from UAV Images with Deep Learning
Recommendations
Time-of-Flight Camera Calibration for Improved 3D Reconstruction of Indoor Scenes
ISCID '14: Proceedings of the 2014 Seventh International Symposium on Computational Intelligence and Design - Volume 02Calibration for Time-of-Flight (ToF) cameras are different from traditional color cameras, in that their calibration involves additional error compensation steps for the depth camera caused by complicated factors. Without this process, there may be ...
Dense 3D reconstruction combining depth and RGB information
Dense 3D reconstruction has important applications in many fields. The existing depth information based methods are typically constrained in their effective camera-object distance which should be from 0.4m to 4m. We present a novel method that can ...
3D reconstruction method for tree seedlings based on point cloud self-registration
Highlights- A 3D reconstruction system was developed by using a Kinect v2 for tree seedlings.
AbstractThe 3D reconstruction of tree seedlings can help to assess phenotypic architectures, conceive virtual urban landscapes and design computer games. The existing multicamera photograph technology already has the capability to accurately ...
Comments
Information & Contributors
Information
Published In
February 2024
757 pages
ISBN:9798400709234
DOI:10.1145/3651671
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Provincial Agricultural Science and Technology Innovation and Extension Project of Guangdong Province
- Guangzhou Science and Technology Plan Project
Conference
ICMLC 2024
ICMLC 2024: 2024 16th International Conference on Machine Learning and Computing
February 2 - 5, 2024
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 18Total Downloads
- Downloads (Last 12 months)18
- Downloads (Last 6 weeks)6
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format