research-article

FPGA implementation of HOOFR bucketing extractor-based real-time embedded SLAM applications

Authors:

Dai Duong Nguyen,

Abdelhafid El Ouardi,

Sergio Rodriguez,

Samir BouazizAuthors Info & Claims

Journal of Real-Time Image Processing, Volume 18, Issue 3

Pages 525 - 538

https://doi.org/10.1007/s11554-020-00986-9

Published: 01 June 2021 Publication History

Abstract

Feature extraction is an important vision task in many applications like simultaneous localization and mapping (SLAM). In the recent computing systems, FPGA-based acceleration have presented a strong competition to GPU-based acceleration due to its high computation capabilities and lower energy consumption. In this paper, we present a high-level synthesis implementation on a SoC-FPGA of a feature extraction algorithm dedicated for SLAM applications. We choose HOOFR extraction algorithm which provides a robust performance but requires a significant computation on embedded CPU. Our system is dedicated for SLAM applications so that we also integrated bucketing detection method in order to have a homogeneous distribution of keypoints in the image. Moreover, instead of optimizing performance by simplifying the original algorithm as in many other researches, we respected the complexity of HOOFR extractor and have parallelized the processing operations. The design has been validated on an Intel Arria 10 SoC-FPGA with a throughput of 54 fps at

1226 \times 370

pixels (handling 1750 features) or 14 fps at

1920 \times 1080

pixels (handling 6929 features).

References

[1]

Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras (2016). arXiv preprint arXiv:1610.06475

[2]

Mei, C., Sibley, G., Cummins, M., Newman, P.M., Reid, I.D.: A constant-time efficient stereo slam system. In: BMVC 2009, pp. 1–11 (2009)

[3]

Lowe DG Distinctive image features from scale-invariant keypoints Int. J. Comput. Vis. 2004 60 2 91-110

[4]

Bay, H., Tuytelaars, T., Van Gool, L., Surf: Speeded up robust features. In: Computer Vision–ECCV 2006. Springer, Berlin, pp. 404–417 (2006)

[5]

Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In 2011 IEEE International Conference on Computer Vision (ICCV). IEEE, pp. 2564–2571 (2011)

[6]

Aldegheri, S., Bombieri, N., Bloisi, D.D., Farinelli, A.: Data flow orb-slam for real-time performance on embedded gpu boards. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5370–5375 (2019)

[7]

Nguyen, D.-D., El Ouardi, A., Aldea, E., Bouaziz, S.: Hoofr: an enhanced bio-inspired feature extractor. In 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, pp. 2977–2982 (2016)

[8]

Pereira, K., Athanas, P., Lin, H., Feng, W.: Spectral method characterization on FPGA and GPU accelerators. In: 2011 International Conference on Reconfigurable Computing and FPGAs (ReConFig). IEEE, pp. 487–492 (2011)

[9]

Weber R, Gothandaraman A, Hinde RJ, and Peterson GD Comparing hardware accelerators in scientific applications: a case study IEEE Trans. Parallel Distrib. Syst. 2011 22 1 58-68

[10]

De Schryver, C., Shcherbakov, I., Kienle, F., Wehn, N., Marxen, H., Kostiuk, A., Korn, R.: An energy efficient fpga accelerator for monte carlo option pricing with the Beston model. In: 2011 International Conference on Reconfigurable Computing and FPGAs (ReConFig). IEEE, pp. 468–474 (2011)

[11]

Pauwels K, Tomasi M, Alonso JD, Ros E, and Van Hulle MM A comparison of fpga and gpu for real-time phase-based optical flow, stereo, and local image features IEEE Trans. Comput 2012 61 7 999-1012

[12]

Morales, V.M., Horrein, P.-H., Baghdadi, A., Hochapfel, E., Vaton, S.: Energy-efficient fpga implementation for binomial option pricing using openCL. In: Design, Automation and Test in Europe Conference and Exhibition (DATE), 2014. IEEE, pp. 1–6 (2014)

[13]

Helali A, Ameur H, Górriz J, Ramírez J, and Maaref H Hardware implementation of real-time pedestrian detection system Neural Comput. Appl. 2020

[14]

Mami S, Lahbib Y, and Mami A A new HLS allocation algorithm for efficient DSP utilization in FPGAs J. Signal Process. Syst. 2019 92 153-171

[15]

Intel FPGA SDK for OpenCL Standard Edition: Programming Guide. INTEL (2014)

[16]

Jelodari PT, Kordasiabi MP, Sheikhaei S, and Forouzandeh B Fpga implementation of an adaptive window size image impulse noise suppression system J. Real-Time Image Process. 2017 16 2015-2026

[17]

Marin, Y., Mitéran, J., Dubois, J., Heyrman, B., Ginhac, D.: An FPGA-based design for real-time super resolution reconstruction. In: Proceedings of the 12th International Conference on Distributed Smart Cameras, pp. 1–2 (2018)

[18]

Yao, L., Feng, H., Zhu, Y., Jiang, Z., Zhao, D., Feng, W.: An architecture of optimised sift feature detection for an FPGA implementation of an image matcher. In: International Conference on Field-Programmable Technology, 2009. FPT 2009. IEEE, pp. 30–37 (2009)

[19]

Bouris, D., Nikitakis, A., Papaefstathiou, I.: Fast and efficient FPGA-based feature detection employing the surf algorithm. In: 2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp. 3–10 (2010)

[20]

Chiu L-C, Chang T-S, Chen J-Y, and Chang NY-C Fast sift design for real-time visual feature extraction IEEE Trans. Image Process. 2013 22 8 3158-3167

[21]

Lee K A design of an optimized orb accelerator for real-time feature detection Int. J. Control Autom. 2014 7 3 213-218

[22]

Weberruss, J., Kleeman, L., Boland, D., Drummond, T.: FPGA acceleration of multilevel orb feature extraction for computer vision. In: 2017 27th International Conference on Field Programmable Logic and Applications (FPL). IEEE, pp 1–8 (2017)

[23]

Sun, R., Liu, P., Wang, J., Accetti, C., Naqvi, A.A.: A 42fps full-hd ORB feature extraction accelerator with reduced memory overhead. In: 2017 International Conference on Field Programmable Technology (ICFPT). IEEE, 2017, pp. 183–190 (2017)

[24]

Pu, Y., Peng, J., Huang, L., Chen, J.: An efficient knn algorithm implemented on FPGA based heterogeneous computing system using opencl. In: 2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, pp. 167–170 (2015)

[25]

Muslim FB, Ma L, Roozmeh M, and Lavagno L Efficient fpga implementation of opencl high-performance computing applications via high-level synthesis IEEE Access 2017 5 2747-2762

[26]

Luo L, Wu Y, Qiao F, Yang Y, Wei Q, Zhou X, Fan Y, Xu S, Liu X, and Yang H Design of FPGA-based accelerator for convolutional neural network under heterogeneous computing framework with openCL Int. J. Reconfig. Comput. 2018

[27]

Zhang S, Wu Y, Men C, He H, and Liang K Research on opencl optimization for fpga deep learning application PloS ONE 2019

[28]

Pire, T., Fischer, T., Civera, J., De Cristóforis, P., Berlles, J.J.: Stereo parallel tracking and mapping for robot localization. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE, pp. 1373–1378 (2015)

[29]

Konolige K and Agrawal M Frameslam: from bundle adjustment to real-time visual mapping IEEE Trans. Robot. 2008 24 5 1066-1077

[30]

Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Computer Vision—ECCV 2006. Springer, Berlin, pp. 430–443 (2006)

[31]

Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012, pp. 510–517 (2012)

[32]

Tola E, Lepetit V, and Fua P Daisy: an efficient dense descriptor applied to wide-baseline stereo IEEE Trans. Pattern Anal. Mach. Intell. 2010 32 5 815-830

[33]

Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

[34]

Bianconi F and Fernández A Evaluation of the effects of gabor filter parameters on texture classification Pattern Recognit. 2007 40 12 3325-3335

[35]

Fularz M, Kraft M, Schmidt A, and Kasinski A A high-performance fpga-based image feature detector and matcher based on the fast and brief algorithms Int. J. Adv. Robot. Syst. 2015 12 10 141

Cited By

Fan ZHao YZhi TGuo QDu Z(2023)Hardware Acceleration for SLAM in Mobile SystemsJournal of Computer Science and Technology10.1007/s11390-021-1523-538:6(1300-1322)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11390-021-1523-5
Gao SWang YChen ZZhou FWang RGuo N(2022)Design and Implementation of Local Threshold Segmentation Based on FPGAJournal of Electrical and Computer Engineering10.1155/2022/65328522022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6532852
Eyvazpour RShoaran MKarimian G(2022)Hardware implementation of SLAM algorithms: a survey on implementation approaches and platformsArtificial Intelligence Review10.1007/s10462-022-10310-556:7(6187-6239)Online publication date: 23-Nov-2022
https://dl.acm.org/doi/10.1007/s10462-022-10310-5

Recommendations

FPGA prototyping of a RISC processor core for embedded applications

Application-specific processors offer an attractive option in the design of embedded systems by providing high performance for a specific application domain. In this work, we describe the use of a reconfigurable processor core based on an RISC ...
A low-area yet performant FPGA implementation of Shabal
SAC'10: Proceedings of the 17th international conference on Selected areas in cryptography

In this paper, we present an efficient FPGA implementation of the SHA-3 hash function candidate Shabal [7]. Targeted at the recent Xilinx Virtex-5 FPGA family, our design achieves a relatively high throughput of 2 Gbit/s at a cost of only 153 slices, ...
Real-time embedded systems powered by FPGA dynamic partial self-reconfiguration: a case study oriented to biometric recognition applications

This work aims to pave the way for an efficient open system architecture applied to embedded electronic applications to manage the processing of computationally complex algorithms at real-time and low-cost. The target is to define a standard ...

Comments

Information & Contributors

Information

Published In

cover image Journal of Real-Time Image Processing

Journal of Real-Time Image Processing Volume 18, Issue 3

Jun 2021

545 pages

ISSN:1861-8200

Issue’s Table of Contents

© Springer-Verlag GmbH Germany, part of Springer Nature 2020.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 June 2021

Accepted: 12 May 2020

Received: 18 September 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fan ZHao YZhi TGuo QDu Z(2023)Hardware Acceleration for SLAM in Mobile SystemsJournal of Computer Science and Technology10.1007/s11390-021-1523-538:6(1300-1322)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1007/s11390-021-1523-5
Gao SWang YChen ZZhou FWang RGuo N(2022)Design and Implementation of Local Threshold Segmentation Based on FPGAJournal of Electrical and Computer Engineering10.1155/2022/65328522022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/6532852
Eyvazpour RShoaran MKarimian G(2022)Hardware implementation of SLAM algorithms: a survey on implementation approaches and platformsArtificial Intelligence Review10.1007/s10462-022-10310-556:7(6187-6239)Online publication date: 23-Nov-2022
https://dl.acm.org/doi/10.1007/s10462-022-10310-5

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents