Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Collaborative SLAM and AR-guided navigation for floor layout inspection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper presents how visual SLAM and AR interfaces are integrated for inspecting the floor layouts, which is a standard stage in building a semiconductor fabrication plant (commonly called a fab). The proposed system supports multiple inspectors working in a large fab floor through client–server communications. The inspectors/clients scan the fab floor using smartphones to construct the floor map collaboratively, and then, its discrepancy from the given floor plan is checked by the server. The inspection results are broadcast to all clients and visualized via AR to help the inspectors move to unscanned areas. The experiments made with the proposed system show its accuracy and efficiency. By solving a real-world industrial problem, the proposed system proves the benefits that can be brought by integrating collaborative SLAM technique and AR-guided navigation interface.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Barbieri, L., Marino, E.: An augmented reality tool to detect design discrepancies: a comparison test with traditional methods. In: International Conference on Augmented Reality, Virtual Reality and Computer Graphics, pp. 99–110. Springer (2019)

  2. Bruno, F., Barbieri, L., Marino, E., Muzzupappa, M., D’Oriano, L., Colacino, B.: An augmented reality tool to detect and annotate design variations in an industry 4.0 approach. Int. J. Adv. Manuf. Technol. 105(1–4), 875–887 (2019)

  3. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)

    Article  Google Scholar 

  4. Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartogr: Int. J. Geogr. Inf. Geovisualization 10(2), 112–122 (1973)

    Article  Google Scholar 

  5. DüNser, A., Billinghurst, M., Wen, J., Lehtinen, V., Nurminen, A.: Exploring the use of handheld AR for outdoor navigation. Comp. Gr. 36(8), 1084–1095 (2012)

    Article  Google Scholar 

  6. Egodagamage, R., Tuceryan, M.: Distributed monocular visual slam as a basis for a collaborative augmented reality framework. Comp. Gr. 71, 113–123 (2018)

    Article  Google Scholar 

  7. Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2017)

    Article  Google Scholar 

  8. Forster, C., Lynen, S., Kneip, L., Scaramuzza, D.: Collaborative monocular slam with multiple micro aerial vehicles. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3962–3970. IEEE (2013)

  9. Georgel, P., Benhimane, S., Sotke, J., Navab, N.: Photo-based industrial augmented reality application using a single keyframe registration procedure. In: 2009 8th IEEE International Symposium on Mixed and Augmented Reality, pp. 187–188. IEEE (2009)

  10. Georgel, P., Schroeder, P., Benhimane, S., Hinterstoisser, S., Appel, M., Navab, N.: An industrial augmented reality solution for discrepancy check. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–4. IEEE Computer Society (2007)

  11. Golodetz, S., Cavallari, T., Lord, N.A., Prisacariu, V.A., Murray, D.W., Torr, P.H.: Collaborative large-scale dense 3d reconstruction with online inter-agent pose optimisation. IEEE Trans. Vis. Comp. Gr. 24(11), 2895–2905 (2018)

    Article  Google Scholar 

  12. Hough, P.V.: Method and means for recognizing complex patterns (1962). US Patent 3,069,654

  13. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al.: Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on User interface software and technology, pp. 559–568. ACM (2011)

  14. Kahn, S., Bockholt, U., Kuijper, A., Fellner, D.W.: Towards precise real-time 3d difference detection for industrial applications. Comp. Ind. 64(9), 1115–1128 (2013)

    Article  Google Scholar 

  15. Koch, C., Neges, M., König, M., Abramovici, M.: Natural markers for augmented reality-based indoor navigation and facility maintenance. Autom. Const. 48, 18–30 (2014)

    Article  Google Scholar 

  16. Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: an accurate o (n) solution to the pnp problem. Int. J. Comp Vis. 81(2), 155 (2009)

    Article  Google Scholar 

  17. Makita, K., Vincent, T., Ebisuno, S., Kourogi, M., Ishikawa, T., Okuma, T., Yoshida, M., Nigay, L., Kurata, T.: Mixed reality navigation on a tablet computer for supporting machine maintenance in wide-area indoor environment. In: Serviceology for Designing the Future, pp. 109–124. Springer (2016)

  18. Marder-Eppstein, E.: Project tango. In: ACM SIGGRAPH 2016 Real-Time Live!, p. 40. ACM (2016)

  19. Mohanarajah, G., Usenko, V., Singh, M., D’Andrea, R., Waibel, M.: Cloud-based collaborative 3d mapping in real-time with low-cost robots. IEEE Trans. Autom. Sci. Eng. 12(2), 423–431 (2015)

    Article  Google Scholar 

  20. Mulloni, A., Seichter, H., Schmalstieg, D.: User experiences with augmented reality aided navigation on phones. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 229–230. IEEE (2011)

  21. Mur-Artal, R., Tardós, J.D.: Fast relocalisation and loop closing in keyframe-based slam. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 846–853. IEEE (2014)

  22. Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  23. Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136. IEEE (2011)

  24. Palmarini, R., Erkoyuncu, J.A., Roy, R., Torabmostaedi, H.: A systematic review of augmented reality applications in maintenance. Robot. Comp. Integr. Manuf. 49, 215–228 (2018)

    Article  Google Scholar 

  25. Perla, R., Gupta, G., Hebbalaguppe, R., Hassan, E.: Inspectar: An augmented reality inspection framework for industry. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 355–356. IEEE (2016)

  26. Polvi, J., Taketomi, T., Moteki, A., Yoshitake, T., Fukuoka, T., Yamamoto, G., Sandor, C., Kato, H.: Handheld guides in inspection tasks: augmented reality versus picture. IEEE Trans. Vis. Comp. Gr. 24(7), 2118–2128 (2018)

    Article  Google Scholar 

  27. Riazuelo, L., Civera, J., Montiel, J.M.: C2tam: a cloud framework for cooperative tracking and mapping. Robot. Auton. Syst. 62(4), 401–413 (2014)

    Article  Google Scholar 

  28. Schmuck, P., Chli, M.: Multi-uav collaborative monocular slam. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3863–3870. IEEE (2017)

  29. Strasdat, H., Montiel, J., Davison, A.J.: Scale drift-aware large scale monocular slam. Robot. : Sci. Syst. VI 2(3), 7 (2010)

    Google Scholar 

  30. Suzuki, S., et al.: Topological structural analysis of digitized binary images by border following. Comp. Vis Gr. Image Process. 30(1), 32–46 (1985)

    Article  Google Scholar 

  31. Takata, S., Kirnura, F., van Houten, F.J., Westkamper, E., Shpitalni, M., Ceglarek, D., Lee, J.: Maintenance: changing role in life cycle management. CIRP Ann. 53(2), 643–655 (2004)

    Article  Google Scholar 

  32. Verizon Media (2019): Google Maps AR navigation is rolling out to a handful of users. https://www.engadget.com/2019/02/10/google-maps-ar-navigation-early-rollout/. Accessed: 18 March 2019

  33. Wasenmüller, O., Meyer, M., Stricker, D.: Augmented reality 3d discrepancy check in industrial applications. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 125–134. IEEE (2016)

  34. Webel, S., Becker, M., Stricker, D., Wuest, H.: Identifying differences between cad and physical mock-ups using ar. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 281–282. IEEE (2007)

Download references

Acknowledgements

This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00861) and National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2017M3C4A7066316).

Author information

Authors and Affiliations

  1. Dept. of Computer Science and Engineering, Korea University, Seoul, South Korea

    Kihyun Yu, JeongHyeon Ahn, Jongmin Lee & JungHyun Han

  2. Iaan Co., Ltd, Bucheon, South Korea

    Myungho Kim

Authors
  1. Kihyun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. JeongHyeon Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jongmin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Myungho Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. JungHyun Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to JungHyun Han.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, K., Ahn, J., Lee, J. et al. Collaborative SLAM and AR-guided navigation for floor layout inspection. Vis Comput 36, 2051–2063 (2020). https://doi.org/10.1007/s00371-020-01911-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01911-8

Keywords

Search

Navigation