short-paper

Fusing Geometric and Scene Information for Cross-View Geo-Localization

Authors:

Shuigeng ZhouAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 3978 - 3982

https://doi.org/10.1145/3511808.3557633

Published: 17 October 2022 Publication History

Abstract

Cross-view geo-localization is to match scene images (e.g. ground-view images) with geo-tagged aerial images, which is crucial to a wide range of applications such as autonomous driving and street view navigation. Existing methods can neither address the perspective difference well nor effectively capture the scene information. In this work, we propose a Geometric and Scene Information Fusion (GSIF) model for more accurate cross-view geo-localization. GSIF first learns the geometric information of scene images and aerial images via log-polar transformation and spatial-attention aggregation to alleviate the perspective difference. Then, it mines the scene information of scene images via Sky View Factor (SVF) extraction. Finally, both geometric information and scene information are fused for image matching, and a balanced loss function is introduced to boost the matching accuracy. Experimental results on two real datasets show that our model can significantly outperforms the existing methods.

References

[1]

Lee Chapman, JE Thornes, and AV Bradley. 2001. Rapid determination of canyon geometry parameters for use in surface radiation budgets. Theoretical and applied climatology, Vol. 69, 1 (2001), 81--89.

[2]

Sixing Hu, Mengdan Feng, Rang MH Nguyen, and Gim Hee Lee. 2018. Cvm-net: Cross-view matching network for image-based ground-to-aerial geo-localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7258--7267.

[3]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[4]

Liu Liu and Hongdong Li. 2019. Lending orientation to neural networks for cross-view geo-localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5624--5633.

[5]

Colin McManus, Winston Churchill, Will Maddern, Alexander D Stewart, and Paul Newman. 2014. Shady dealings: Robust, long-term visual localisation using illumination invariance. In 2014 IEEE international conference on robotics and automation (ICRA). IEEE, 901--906.

[6]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).

[7]

Krishna Regmi and Ali Borji. 2018. Cross-view image synthesis using conditional gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3501--3510.

[8]

Krishna Regmi and Mubarak Shah. 2019. Bridging the domain gap for ground-to-aerial image matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 470--479.

[9]

Yujiao Shi, Liu Liu, Xin Yu, and Hongdong Li. 2019. Spatial-aware feature aggregation for image based cross-view geo-localization. Advances in Neural Information Processing Systems, Vol. 32 (2019).

[10]

Yujiao Shi, Xin Yu, Dylan Campbell, and Hongdong Li. 2020. Where am i looking at? joint location and orientation estimation by cross-view matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4064--4072.

[11]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[12]

Aparna Taneja, Luca Ballan, and Marc Pollefeys. 2014. Never get lost again: Vision based navigation using streetview images. In Asian Conference on Computer Vision. Springer, 99--114.

[13]

Nam N Vo and James Hays. 2016. Localizing and orienting street views using overhead imagery. In European conference on computer vision. Springer, 494--509.

[14]

Scott Workman, Richard Souvenir, and Nathan Jacobs. 2015. Wide-area image geolocalization with aerial reference imagery. In Proceedings of the IEEE International Conference on Computer Vision. 3961--3969.

Digital Library

[15]

Menghua Zhai, Zachary Bessinger, Scott Workman, and Nathan Jacobs. 2017. Predicting ground-level scene layout from aerial imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 867--875.

[16]

Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2881--2890.

[17]

Sijie Zhu, Taojiannan Yang, and Chen Chen. 2021. Revisiting street-to-aerial view image geo-localization and orientation estimation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 756--765.

Cited By

Li JYang CQi BZhu MWu N(2024)4SCIG: A Four-Branch Framework to Reduce the Interference of Sky Area in Cross-View Image Geo-LocalizationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.337937662(1-18)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3379376
Sun YYe YKang JFernandez-Beltran RFeng SLi XLuo CZhang PPlaza A(2023)Cross-View Object Geo-Localization in a Local Region With Satellite ImageryIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.330750861(1-16)Online publication date: 2023
https://doi.org/10.1109/TGRS.2023.3307508

Index Terms

Fusing Geometric and Scene Information for Cross-View Geo-Localization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Matching

Recommendations

Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Street-view imagery provides us with novel experiences to explore different places remotely. Carefully calibrated street-view images (e.g., Google Street View) can be used for different downstream tasks, e.g., navigation, map features extraction. As ...
Geo-localization of street views with aerial image databases
MM '11: Proceedings of the 19th ACM international conference on Multimedia

We study the feasibility of solving the challenging problem of geolocalizing ground level images in urban areas with respect to a database of images captured from the air such as satellite and oblique aerial images. We observe that comprehensive aerial ...
Cross-View Image Retrieval - Ground to Aerial Image Retrieval Through Deep Learning
Neural Information Processing
Abstract
Cross-modal retrieval aims to measure the content similarity between different types of data. The idea has been previously applied to visual, text, and speech data. In this paper, we present a novel cross-modal retrieval method specifically for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Open Research Projects of Zhejiang Lab
National Natural Science Foundation of China
Shanghai Pujiang Program

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)5

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li JYang CQi BZhu MWu N(2024)4SCIG: A Four-Branch Framework to Reduce the Interference of Sky Area in Cross-View Image Geo-LocalizationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.337937662(1-18)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3379376
Sun YYe YKang JFernandez-Beltran RFeng SLi XLuo CZhang PPlaza A(2023)Cross-View Object Geo-Localization in a Local Region With Satellite ImageryIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.330750861(1-16)Online publication date: 2023
https://doi.org/10.1109/TGRS.2023.3307508

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents