Abstract
On-tree fruit detection in orchards is important for yield estimation, mapping and automatic harvesting in modern agriculture. This paper proposes a real-time detection framework for on-tree mango based on SSD (Single shot Multi Box Detector) network, a state-of-the-art object detection algorithms based on deep learning. The mango image dataset used in this paper was gathered from outdoor mango orchards. Firstly, the dataset was annotated and converted to a trainable dataset for SSD network. Secondly, the author designed new sampling strategies and image distortions at the image pre-processing stage to optimize data augmentation techniques. Moreover, the default box proposal methods of SSD network were improved by redesigning the shapes of default boxes on multiple feature maps according to our own dataset. Finally, to explore which classification network is most suitable for mango detection, an experiment was presented to compare the detection performance of SSD network with the VGG16 and ZFNet as base network respectively. Almond dataset was also used to verify our proposed method. Experimental results demonstrated that, with optimization of data augmentation techniques and default box proposals, our improved VGG16-based SSD network can achieve higher performance than Faster R-CNN in on-tree mango detection, with F1 score of 0.911 at 35 FPS for 400 × 400 input image, which is a real-time detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Syal, A., Garg, D., Sharma, S.: Apple fruit detection and counting using computer vision techniques. In: IEEE International Conference on Computational Intelligence & Computing Research, pp. 1–6 (2015)
Kapach, K., Barnea, E., Mairon, R., Edan, Y., Ben-Shahar, O.: Computer vision for fruit harvesting robots – state of the art and challenges ahead. Int. J. Comput. Vis. Robot. 3(1–2), 4–34 (2012)
Wang, Q., Nuske, S., Bergerman, M., Singh, S.: Automated crop yield estimation for apple orchards. In: Desai, J., Dudek, G., Khatib, O., Kumar, V. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 88, pp. 745–758. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-319-00065-7_50
Kadmiry, B., Wong, C.K.: Perception scheme for fruits detection in trees for autonomous agricultural robot applications. In: International Conference on Image & Vision Computing, New Zealand, pp. 1–6 (2016)
Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. arXiv:1612.08242 [cs.CV]
Liu, W., et al.: SSD: Single Shot MultiBox Detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Suchet, B., James, U.: Deep fruit detection in orchards. arXiv:1610.03677 [cs.RO]
Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_23
Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)
Li, Y., He, K., Sun, J.: R-FCN: object detection via region-based fully convolutional networks. In: Advances in Neural Information Processing Systems, pp. 379–387 (2016)
Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)
Stein, M., Bargoti, S., Underwood, J.: Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors 16(11), 1915 (2016). https://doi.org/10.3390/s16111915
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: NIPS (2015)
Bargoti, S.: Pychet Labeller - an object annotation toolbox (2016). https://github.com/acfr/pychetlabeller
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Acknowledgement
This work was supported in part by the National Nature Science Foundation of China (NSFC 61673163), Hunan Provincial Natural Science Foundation of China (2016JJ3045), and Hunan Key Laboratory of Intelligent Robot Technology in Electronic Manufacturing (No. 2018002).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, Q., Zhu, W., Long, J., Wang, Y., Sun, W., Wu, W. (2018). A Real-Time Detection Framework for On-Tree Mango Based on SSD Network. In: Chen, Z., Mendes, A., Yan, Y., Chen, S. (eds) Intelligent Robotics and Applications. ICIRA 2018. Lecture Notes in Computer Science(), vol 10985. Springer, Cham. https://doi.org/10.1007/978-3-319-97589-4_36
Download citation
DOI: https://doi.org/10.1007/978-3-319-97589-4_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97588-7
Online ISBN: 978-3-319-97589-4
eBook Packages: Computer ScienceComputer Science (R0)