Abstract
Recently, multiple homography estimation is preferable for image stitching to handle the parallax problem, by estimating homographies from the feature correspondence in each local region. However, correspondence outliers and insufficient feature coverage will lead to unreliable local homography fitting. In this paper, we propose a novel method of multi-homography estimation and inference, driven by contour alignment. Our method uses explicit structural verification through contour alignment to eliminate incorrectly fitted homographies in some regions, and to select a better homography from other regions if current homography is rejected or with worse accuracy. With the guidance of the contour alignment, dense image alignment result is obtained by further inferring the local homography per superpixel. Quantitative and qualitative comparisons demonstrate the effectiveness of our method, especially for scenes with large parallax and viewpoint changes.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vision 74(1), 59–73 (2007)
Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3254–3261 (2014)
Chen, S.E.: Quicktime vr: an image-based approach to virtual environment navigation. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques, pp. 29–38 (1995)
Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_12
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vision 70(1), 41–54 (2006)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: CVPR 2011, pp. 49–56. IEEE (2011)
Herrmann, C., et al.: Robust image stitching with multiple registrations. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 53–67 (2018)
Herrmann, C., Wang, C., Strong Bowen, R., Keyder, E., Zabih, R.: Object-centered image stitching. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 821–835 (2018)
Jia, J., Tang, C.K.: Image stitching using structure deformation. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 617–631 (2008)
Lafferty, J.D., McCallum, A., Pereira, F.C.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)
Lee, K.Y., Sim, J.Y.: Warping residual based image stitching for large parallax. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8198–8206 (2020)
Li, J., Deng, B., Tang, R., Wang, Z., Yan, Y.: Local-adaptive image alignment based on triangular facet approximation. IEEE Trans. Image Process. 29, 2356–2369 (2019)
Li, J., Wang, Z., Lai, S., Zhai, Y., Zhang, M.: Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimedia 20(7), 1672–1687 (2017)
Li, N., Xu, Y., Wang, C.: Quasi-homography warps in image stitching. IEEE Trans. Multimedia 20(6), 1365–1375 (2017)
Li, S., Yuan, L., Sun, J., Quan, L.: Dual-feature warping-based motion model estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4283–4291 (2015)
Liao, T., Li, N.: Single-perspective warps in natural image stitching. IEEE Trans. Image Process. 29, 724–735 (2019)
Lin, C.C., Pankanti, S.U., Natesan Ramamurthy, K., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1163 (2015)
Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_23
Lin, K., Jiang, N., Liu, S., Cheong, L.F., Do, M., Lu, J.: Direct photometric alignment by mesh deformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2405–2413 (2017)
Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., Cheong, L.F.: Smoothly varying affine stitching. In: CVPR 2011, pp. 345–352. IEEE (2011)
Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3d video stabilization. ACM Trans. Graphics (TOG) 28(3), 1–9 (2009)
Lou, Z., Gevers, T.: Image alignment by piecewise planar region matching. IEEE Trans. Multimedia 16(7), 2052–2061 (2014)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Deepmatching: hierarchical deformable dense matching. Int. J. Comput. Vision 120(3), 300–323 (2016)
Silva, R.M., Feijó, B., Gomes, P.B., Frensh, T., Monteiro, D.: Real time 360 video stitching and streaming. In: ACM SIGGRAPH 2016 Posters, pp. 1–2 (2016)
Szeliski, R., Shum, H.Y., Shum, H.Y., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 251–258. ACM Press/Addison-Wesley Publishing Co. (1997)
Szeliski, R., et al.: Image alignment and stitching: a tutorial. Found. Trends® Comput. Graph. Vis. 2(1), 1–104 (2007)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Xiang, T.-Z., Xia, G.-S., Zhang, L.: Image stitching using smoothly planar homography. In: Lai, J.-H., et al. (eds.) PRCV 2018. LNCS, vol. 11256, pp. 524–536. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03398-9_45
Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: Advances in Neural Information Processing Systems, pp. 689–695 (2001)
Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving dlt. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2339–2346 (2013)
Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269 (2014)
Zheng, J., Wang, Y., Wang, H., Li, B., Hu, H.M.: A novel projective-consistent plane based image stitching method. IEEE Trans. Multimedia 21(10), 2561–2575 (2019)
Acknowledgment
This work was supported by the Natural Science Foundation of China (NSFC) under Grant No. 61773062.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cai, T., Jia, Y., Di, H., Wu, Y. (2021). Multi-homography Estimation and Inference Driven by Contour Alignment. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12888. Springer, Cham. https://doi.org/10.1007/978-3-030-87355-4_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-87355-4_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87354-7
Online ISBN: 978-3-030-87355-4
eBook Packages: Computer ScienceComputer Science (R0)