Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Affine Correspondences Between Multi-camera Systems for 6DOF Relative Pose Estimation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13692))

Included in the following conference series:

Abstract

We present a novel method to compute the 6DOF relative pose of multi-camera systems using two affine correspondences (ACs). Existing solutions to the multi-camera relative pose estimation are either restricted to special cases of motion, have too high computational complexity, or require too many point correspondences (PCs). Thus, these solvers impede an efficient or accurate relative pose estimation when applying RANSAC as a robust estimator. This paper shows that the relative pose estimation problem using ACs permits a feasible minimal solution, when exploiting the geometric constraints between ACs and multi-camera systems using a special parameterization. We present a problem formulation based on two ACs that encompass two common types of ACs across two views, i.e., inter-camera and intra-camera. Experiments on both virtual and real multi-camera systems prove that the proposed solvers are more efficient than the state-of-the-art algorithms, while resulting in a better relative pose accuracy. Source code is available at https://github.com/jizhaox/relpose-mcs-depth.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alyousefi, K., Ventura, J.: Multi-camera motion estimation with affine correspondences. In: International Conference on Image Analysis and Recognition, pp. 417–431 (2020)

    Google Scholar 

  2. Barath, D., Hajder, L.: Efficient recovery of essential matrix from two affine correspondences. IEEE Trans. Image Process. 27(11), 5328–5337 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  3. Barath, D., Kukelova, Z.: Homography from two orientation-and scale-covariant features. In: IEEE International Conference on Computer Vision, pp. 1091–1099 (2019)

    Google Scholar 

  4. Barath, D., Polic, M., Förstner, W., Sattler, T., Pajdla, T., Kukelova, Z.: Making affine correspondences work in camera geometry computation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 723–740. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_42

    Chapter  Google Scholar 

  5. Bentolila, J., Francos, J.M.: Conic epipolar constraints from affine correspondences. Comput. Vis. Image Underst. 122, 105–114 (2014)

    Article  Google Scholar 

  6. Burri, M., et al.: The EuRoC micro aerial vehicle datasets. Int. J. Robot. Res. 35(10), 1157–1163 (2016)

    Article  Google Scholar 

  7. Byröd, M., Josephson, K., Aström, K.: Fast and stable polynomial equation solving and its application to computer vision. Int. J. Comput. Vision 84(3), 237–256 (2009)

    Article  MATH  Google Scholar 

  8. Caesar, H., et al.: nuScenes: a multimodal dataset for autonomous driving. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11621–11631 (2020)

    Google Scholar 

  9. Clipp, B., Kim, J.H., Frahm, J.M., Pollefeys, M., Hartley, R.: Robust 6dof motion estimation for non-overlapping, multi-camera systems. In: IEEE Workshop on Applications of Computer Vision, pp. 1–8. IEEE (2008)

    Google Scholar 

  10. Cox, D.A., Little, J., O’Shea, D.: Using algebraic geometry. Springer Science & Business Media (2006)

    Google Scholar 

  11. Eichhardt, I., Barath, D.: Relative pose from deep learned depth and a single affine correspondence. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 627–644. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_37

    Chapter  Google Scholar 

  12. Eichhardt, I., Chetverikov, D.: Affine correspondences between central cameras for rapid relative pose estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 488–503. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_30

    Chapter  Google Scholar 

  13. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  14. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)

    Article  Google Scholar 

  15. Grayson, D.R., Stillman, M.E.: Macaulay 2, a software system for research in algebraic geometry (2002). https://faculty.math.illinois.edu/Macaulay2/

  16. Grossberg, M.D., Nayar, S.K.: A general imaging model and a method for finding its parameters. In: IEEE International Conference on Computer Vision, vol. 2, pp. 108–115. IEEE (2001)

    Google Scholar 

  17. Guan, B., Zhao, J., Barath, D., Fraundorfer, F.: Efficient recovery of multi-camera motion from two affine correspondences. In: IEEE International Conference on Robotics and Automation, pp. 1305–1311 (2021)

    Google Scholar 

  18. Guan, B., Zhao, J., Barath, D., Fraundorfer, F.: Minimal cases for computing the generalized relative pose using affine correspondences. In: IEEE International Conference on Computer Vision, pp. 6068–6077 (2021)

    Google Scholar 

  19. Hajder, L., Barath, D.: Relative planar motion for vehicle-mounted cameras from a single affine correspondence. In: IEEE International Conference on Robotics and Automation, pp. 8651–8657 (2020)

    Google Scholar 

  20. Häne, C., et al.: 3D visual perception for self-driving cars using a multi-camera system: calibration, mapping, localization, and obstacle detection. Image Vis. Comput. 68, 14–27 (2017)

    Google Scholar 

  21. Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2003)

    Google Scholar 

  22. Heng, L., et al.: Project AutoVision: localization and 3D scene perception for an autonomous vehicle with a multi-camera system. In: IEEE International Conference on Robotics and Automation, pp. 4695–4702 (2019)

    Google Scholar 

  23. Kim, J.H., Hartley, R., Frahm, J.M., Pollefeys, M.: Visual odometry for non-overlapping views using second-order cone programming. In: Asian Conference on Computer Vision. pp. 353–362 (2007)

    Google Scholar 

  24. Kim, J.H., Li, H., Hartley, R.: Motion estimation for nonoverlapping multicamera rigs: Linear algebraic and \(L_{\infty }\) geometric solutions. IEEE Trans. Pattern Anal. Mach. Intell. 32(6), 1044–1059 (2009)

    Google Scholar 

  25. Kneip, L., Furgale, P.: OpenGV: A unified and generalized approach to real-time calibrated geometric vision. In: IEEE International Conference on Robotics and Automation, pp. 12034–12043 (2014)

    Google Scholar 

  26. Kneip, L., Li, H.: Efficient computation of relative pose for multi-camera systems. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 446–453 (2014)

    Google Scholar 

  27. Kneip, L., Sweeney, C., Hartley, R.: The generalized relative pose and scale problem: View-graph fusion via 2D–2D registration. In: IEEE Winter Conference on Applications of Computer Vision, pp. 1–9 (2016)

    Google Scholar 

  28. Larsson, V., Aström, K., Oskarsson, M.: Efficient solvers for minimal problems by syzygy-based reduction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 820–828 (2017)

    Google Scholar 

  29. Lee, G.H., Faundorfer, F., Pollefeys, M.: Motion estimation for self-driving cars with a generalized camera. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2746–2753 (2013)

    Google Scholar 

  30. Lee, G.H., Pollefeys, M., Fraundorfer, F.: Relative pose estimation for a multi-camera system with known vertical direction. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 540–547 (2014)

    Google Scholar 

  31. Li, H., Hartley, R., Kim, J.H.: A linear approach to motion estimation using generalized camera models. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

    Google Scholar 

  32. Lidl, R., Niederreiter, H.: Finite Fields. Cambridge University Press (1997)

    Google Scholar 

  33. Lim, J., Barnes, N., Li, H.: Estimating relative camera motion from the antipodal-epipolar constraint. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1907–1914 (2010)

    Article  Google Scholar 

  34. Liu, L., Li, H., Dai, Y., Pan, Q.: Robust and efficient relative pose with a multi-camera system for autonomous driving in highly dynamic environments. IEEE Trans. Intell. Transp. Syst. 19(8), 2432–2444 (2017)

    Article  Google Scholar 

  35. Martyushev, E., Li, B.: Efficient relative pose estimation for cameras and generalized cameras in case of known relative rotation angle. J. Math. Imaging Vis. 62, 1076–1086 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  36. Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10), 761–767 (2004)

    Article  Google Scholar 

  37. Miraldo, P., Araujo, H., Queiró, J.: Point-based calibration using a parametric representation of the general imaging model. In: IEEE International Conference on Computer Vision, pp. 2304–2311. IEEE (2011)

    Google Scholar 

  38. Morel, J.M., Yu, G.: ASIFT: a new framework for fully affine invariant image comparison. SIAM J. Imag. Sci. 2(2), 438–469 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  39. Nistér, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 756–777 (2004)

    Article  Google Scholar 

  40. Pless, R.: Using many cameras as one. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2003)

    Google Scholar 

  41. Raposo, C., Barreto, J.P.: Theory and practice of structure-from-motion using affine correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5470–5478 (2016)

    Google Scholar 

  42. Scaramuzza, D., Fraundorfer, F.: Visual odometry: the first 30 years and fundamentals. IEEE Robot. Autom. Mag. 18(4), 80–92 (2011)

    Article  Google Scholar 

  43. Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4104–4113 (2016)

    Google Scholar 

  44. Stewénius, H., Nistér, D., Kahl, F., Schaffalitzky, F.: A minimal solution for relative pose with unknown focal length. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 789–794 (2005)

    Google Scholar 

  45. Stewénius, H., Oskarsson, M., Aström, K., Nistér, D.: Solutions to minimal generalized relative pose problems. In: Workshop on Omnidirectional Vision in conjunction with ICCV, pp. 1–8 (2005)

    Google Scholar 

  46. Sturm, P., Ramalingam, S.: A generic concept for camera calibration. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3022, pp. 1–13. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24671-8_1

    Chapter  Google Scholar 

  47. Sweeney, C., Flynn, J., Turk, M.: Solving for relative pose with a partially known rotation is a quadratic eigenvalue problem. In: International Conference on 3D Vision, pp. 483–490 (2014)

    Google Scholar 

  48. Ventura, J., Arth, C., Lepetit, V.: An efficient minimal solution for multi-camera motion. In: IEEE International Conference on Computer Vision, pp. 747–755 (2015)

    Google Scholar 

  49. Zhao, J., Kneip, L., He, Y., Ma, J.: Minimal case relative pose computation using ray-point-ray features. IEEE Trans. Pattern Anal. Mach. Intell. 42(5), 1176–1190 (2020)

    Google Scholar 

  50. Zhao, J., Xu, W., Kneip, L.: A certifiably globally optimal solution to generalized essential matrix estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 12034–12043 (2020)

    Google Scholar 

  51. Zheng, E., Wu, C.: Structure from motion using structure-less resection. In: IEEE International Conference on Computer Vision, pp. 2075–2083 (2015)

    Google Scholar 

Download references

Acknowledgment

This work has been partially funded by the National Natural Science Foundation of China (Grant Nos. 11902349 and 11727804).

Author information

Authors and Affiliations

  1. National University of Defense Technology, Changsha, 410073, China

    Banglei Guan

  2. Beijing, China

    Ji Zhao

Authors
  1. Banglei Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ji Zhao .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 7221 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Guan, B., Zhao, J. (2022). Affine Correspondences Between Multi-camera Systems for 6DOF Relative Pose Estimation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13692. Springer, Cham. https://doi.org/10.1007/978-3-031-19824-3_37

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19824-3_37

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19823-6

  • Online ISBN: 978-3-031-19824-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation