research-article

Efficiently locating photographs in many panoramas

Authors:

Michael Kroepfl,

Yonatan Wexler,

Eyal OfekAuthors Info & Claims

GIS '10: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems

Pages 119 - 128

https://doi.org/10.1145/1869790.1869810

Published: 02 November 2010 Publication History

Abstract

We present a method for efficient and reliable geo-positioning of images. It relies on image-based matching of the query images onto a trellis of existing images that provides accurate 5-DOF calibration (camera position and orientation without scale). As such it can handle any image input, including old historical images, matched against a whole city. On such a scale, care needs to be taken with the size of the database. We deviate from previous work by using 360° panoramas to simultaneously reduce the database size and increase the coverage. To reduce the likelihood of false matches, we restrict the range of angles for matched features. Furthermore, we enhance the RANSAC procedure to include two phases. The second phase includes guided feature matching to increase the likelihood of positive matches. Hence, we devise a matching confidence score that separates between true and false matches. We demonstrate the algorithm on a large scale database covering a whole city in order to show its usefulness for a vision-based augmented reality system.

References

[1]

Digital Photography, http://en.wikipedia.org/wiki/Digital_photography#Applications_and_considerations

[2]

Flickr#8482; http://www.flickr.com/

[3]

Facebook#8482; http://www.facebook.com/

[4]

Panoramio#8482; http://www.panoramio.com/

[5]

Photobucket#8482; http://www.photobucket.com/

[6]

Bing Maps#8482; http://www.bing.com/maps/explore/

[7]

Google Maps#8482; http://maps.google.com/

[8]

Photosynth#8482; htp://www.photosynth.net/

[9]

J. L. Bentley, Multidimensional binary search trees used for associative searching, Communications of the ACM, Volume 18, Issue 9, September 1975

Digital Library

[10]

M. Fischler and R. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381--395, June 1981.

Digital Library

[11]

J. A. Canny, Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679--714, 1986

Digital Library

[12]

C. Harris and M. J. Stephens. A combined corner and edge detector. In Alvey Vision Conference, pages 147--152, 1988.

[13]

D. G. Lowe. Object recognition from local scale invariant features. In Proc. of the International Conference on Computer Vision ICCV, Corfu, pages 1150--1157, 1999.

Digital Library

[14]

M. A. Lourakis, S. V. Tzurbakis, A. A. Argyros and S. C. Orphanoudakis, Using Geometric Constraints for Matching Disparate Stereo Views 3D Scenes Containing Planes. In Proc. of the International Conf. on Pat. Recogn. (ICPR'00), Vol. 1, Barcelona, Spain, Sep. 3--8, 2000.

[15]

A. Baumberg. Reliable feature matching across widely separated views. In CVPR, pages 774--781, 2000.

[16]

K. Mikolajczyk and C. Schmid: Indexing based on scale invariant interest points. In International Conference on Computer Vision, 525--531, 2001

[17]

B. Johansson and R. Cipolla. A system for automatic pose-estimation from a single image in a city scene. In International Conference on Signal Processing, Pattern Recognition and Applications, 2002.

[18]

K. Mikolajczyk and C. Schmid: An affine invariant interest point detector. In: ECCV. (2002) 128--142

Digital Library

[19]

J. Matas, O. Chum, M. Urban, and T. Pajdla. Robust wide baseline stereo from maximally stable extremal regions. In Proceedings of the British Machine Vision Conference, volume 1, pages 384--393, September 2002.

[20]

J. Sivic and A. Zisserman. Video Google: A text retrieval approach to object matching in videos. In ICCV, 2003.

Digital Library

[21]

K. Mikolajczyk and C. Schmid: A performance evaluation of local descriptors. In: CVPR. Volume 2., 2003, 257--263

[22]

D. Wagner and D. Schmalstieg. First steps towards handheld augmented reality. In 7th Intl. Symposium on Wearable Computers (ISWC'03), pages 127--137, White Plains, NY, October 2003.

Digital Library

[23]

D. Lowe. Distinctive image features from scale-invariant key points. IJCV, 60(2):91--110, 2004.

Digital Library

[24]

R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.

Digital Library

[25]

D. Robertson and R. Cipolla. An image-based system for urban navigation. In BMVC, 2004.

[26]

F. Fraundorfer and H. Bischof. Evaluation of local detectors on non-planar scenes. In Proc. 28th workshop of the Austrian Association for Pattern Recognition, pages 125--132, 2004.

[27]

T. Tuytelaars and L. V. Gool. Matching widely separated views based on affine invariant regions. IJCV, 1(59):61--85, 2004.

Digital Library

[28]

K. Mikolajczyk and C. Schmid: A performance evaluation of local descriptors. PAMI, 27(10):1615--1630, 2005.

Digital Library

[29]

K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool. A comparison of affine region detectors. International Journal of Computer Vision, 65(1--2):43--72, 2005.

Digital Library

[30]

M. Brown, R. Szeliski, and S. Winder. Multi-image matching using multi-scale oriented patches. In CVPR, volume 1, pages 510--517, 2005.

Digital Library

[31]

D. Steedly, C. Pal, and R. Szeliski, "Efficiently Registering Video into Panoramic Mosaics," Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, IEEE, 2005, pp. 1300--1307.

Digital Library

[32]

H. Bay, T. Tuytelaars, L. V. Gool: SURF: Speeded up robust features. In: European Conference on Computer Vision (2006)

Digital Library

[33]

N. Snavely, S. Seitz, R. Szeliski. Photo tourism: exploring photo collections in 3d, SIGGRAPH, 2006.

Digital Library

[34]

D. Nister and H. Stewenius. Scalable recognition with a vocabulary tree. In CVPR, pages 2161--2168, 2006.

Digital Library

[35]

W. Zhang and J. Kosecka. Image based localization in urban environments. In International Symposium on 3D Data Processing, Visualization and Transmission, 2006.

Digital Library

[36]

A. J. Chavez, A FAST interest point detection algorithm, Master of Science Thesis, 2008

[37]

G. Reitmayr, T. Drummond: Going out: robust model-based tracking for outdoor augmented reality. In: ISMAR, IEEE 2006 109--118

Digital Library

[38]

G. Schindler, M. Brown, and R. Szeliski. City-scale location recognition. In CVPR, pages 1--7, 2007.

[39]

S. Winder and M. Brown. Learning local image descriptors. In CVPR, 2007.

[40]

G. Reitmayr, T. Drummond: Initialization for visual tracking in urban environments. In: Proc. ISMAR 2007. 161--160

Digital Library

[41]

N. Jacobs, S. Satkin, N. Roman, R. Speyer, R. Pless, Geolocating static cameras. In IEEE International Conference on Computer Vision (ICCV), October 2007

[42]

B. Epshtein, E. Ofek, Y. Wexler, P. Zhang: Hierarchical photo organization using geo-relevance, SIGGIS 2007

Digital Library

[43]

D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg. Pose tracking from natural features on mobile phones. In Proc. 7th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'08), Sept. 15--18 2008.

Digital Library

[44]

D. Wagner, T. Langlotz, and D. Schmalstieg. Robust and unobtrusive marker tracking on mobile phones. In Proc. 7th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'08), Sept. 15--18 2008.

Digital Library

[45]

C. Wu, B. Clipp, X. Li, J. M. Frahm, M. Pollefeys: 3D Model Matching with Viewpoint Invariant Patches (VIPs). In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (2008)

[46]

S. Lilja: Matching Image Pairs, Master of Science Thesis, Stockholm, Sweden, 2008

[47]

R. Castle, G. Klein, D. Murray: Video-rate Localization in Multiple Maps for Wearable Augmented Reality. ISWC 2008. 12th IEEE Symposium on Wearable Computers. Sept. 2008

Digital Library

[48]

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, R. Szeliski: Building Rome in a day. In: IEEE International Conference on Computer Vision (ICCV). 2009

[49]

A. Irschara, C. Zach, J-M. Frahm, H. Bischof: From Structure-from-Motion Point Clouds to Fast location Recognition, CVPR, 2009

[50]

M. Kroepfl, E. Ofek, Y. Wexler, D. Wysocki, G. Kimchi: Geocoding by Image Matching. Microsoft Patent Application #327328.01, 2009

[51]

S. Winder G. Hua, M. Brown: Picking the Best Daisy, IEEE Computer Society, June 2009

[52]

A. Gil, O. M. Mozos, M. Ballesta, O. Reinoso, A comparative evaluation of interest point detectors and local descriptors for visual SLAM, Machine Vision and Applications, Springer-Verlag, March 2009

Digital Library

[53]

M. Brown, G. Hua, S. Winder: Discriminant Learning of Local Image Descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence. February 2010

Digital Library

[54]

E. Ofek, M. Kroepfl, J. Walker, G. Ramos, B. Aguera y Arcas: Viewing Media in the Context of Street-Level Images. Microsoft Patent Application #328899.02, 2010

[55]

ARToolkit http://www.hitl.washington.edu/artoolkit/

[56]

Bing Maps#8482; Streetside Photos CTP http://www.bing.com/maps/explore/#/bqx21pyfpdn6h2ly

Cited By

Kim BPark J(2017)Estimating deformation factors of planar patterns in spherical panoramic imagesMultimedia Systems10.1007/s00530-016-0513-x23:5(607-625)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s00530-016-0513-x
Yin YWang GZimmermann RAli MNewsam SRenz MTrajcevski GRavada S(2016)Automatic geographic metadata correction for sensor-rich video sequencesProceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/2996913.2997015(1-10)Online publication date: 31-Oct-2016
https://dl.acm.org/doi/10.1145/2996913.2997015
Li YSnavely NHuttenlocher DFua P(2016)Worldwide Pose Estimation Using 3D Point CloudsLarge-Scale Visual Geo-Localization10.1007/978-3-319-25781-5_8(147-163)Online publication date: 6-Jul-2016
https://doi.org/10.1007/978-3-319-25781-5_8
Show More Cited By

Recommendations

Exploiting line metric reconstruction from non-central circular panoramas

Comparison among non-central systems for single-view line metric reconstruction.Non-Manhattan line metric reconstruction from single image in non-central panoramas.Automatic line-image extraction in non-central panoramas. In certain non-central imaging ...
Omnivergent Stereo

The notion of a virtual camera for optimal 3D reconstruction is introduced. Instead of planar perspective images that collect many rays at a fixed viewpoint, omnivergent cameras collect a small number of rays at many different viewpoints. The resulting ...
Noise-Resilient Reconstruction of Panoramas and 3D Scenes Using Robot-Mounted Unsynchronized Commodity RGB-D Cameras

We present a two-stage approach to first constructing 3D panoramas and then stitching them for noise-resilient reconstruction of large-scale indoor scenes. Our approach requires multiple unsynchronized RGB-D cameras, mounted on a robot platform, which ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GIS '10: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2010

566 pages

ISBN:9781450304283

DOI:10.1145/1869790

General Chairs:
Divyakant Agrawal
University of California at Santa Barbara
,
Pusheng Zhang
Microsoft Corporation
,
Program Chairs:
Amr El Abbadi
University of California, Santa Barbara
,
Mohamed Mokbel
University of Minnesota

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GIS '10

Sponsor:

SIGSPATIAL

GIS '10: 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems

November 2 - 5, 2010

California, San Jose

Acceptance Rates

Overall Acceptance Rate 257 of 1,238 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
287
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim BPark J(2017)Estimating deformation factors of planar patterns in spherical panoramic imagesMultimedia Systems10.1007/s00530-016-0513-x23:5(607-625)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s00530-016-0513-x
Yin YWang GZimmermann RAli MNewsam SRenz MTrajcevski GRavada S(2016)Automatic geographic metadata correction for sensor-rich video sequencesProceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/2996913.2997015(1-10)Online publication date: 31-Oct-2016
https://dl.acm.org/doi/10.1145/2996913.2997015
Li YSnavely NHuttenlocher DFua P(2016)Worldwide Pose Estimation Using 3D Point CloudsLarge-Scale Visual Geo-Localization10.1007/978-3-319-25781-5_8(147-163)Online publication date: 6-Jul-2016
https://doi.org/10.1007/978-3-319-25781-5_8
Liu HLi HMei TLuo J(2015)Accurate sensing of scene geo-context via mobile visual localizationMultimedia Systems10.1007/s00530-013-0344-y21:3(255-265)Online publication date: 1-Jun-2015
https://dl.acm.org/doi/10.1007/s00530-013-0344-y
Kim J(2014)Remediating Panorama on the Small Screen: Scale, Movement and Spectatorship in Software-Driven Panoramic PhotographyAnimation10.1177/17468477145266779:2(159-176)Online publication date: 23-Jun-2014
https://doi.org/10.1177/1746847714526677
Wang GZimmermann RDemiryurek USarwat M(2014)Spatial sensor data processing and analysis for mobile media applicationsProceedings of the 1st ACM SIGSPATIAL PhD Workshop10.1145/2694859.2694868(1-5)Online publication date: 4-Nov-2014
https://dl.acm.org/doi/10.1145/2694859.2694868
Wang GYin YSeo BZimmermann RShen ZKnoblock CSchneider MKröger PKrumm JWidmayer P(2013)Orientation data correction with georeferenced mobile videosProceedings of the 21st ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems10.1145/2525314.2525445(400-403)Online publication date: 5-Nov-2013
https://dl.acm.org/doi/10.1145/2525314.2525445
Tompkin JPece FShah RIzadi SKautz JTheobalt CIzadi SQuigley APoupyrev IIgarashi T(2013)Video collections in panoramic contextsProceedings of the 26th annual ACM symposium on User interface software and technology10.1145/2501988.2502013(131-140)Online publication date: 8-Oct-2013
https://dl.acm.org/doi/10.1145/2501988.2502013
Liu HMei TLi HLuo JLi S(2013)Robust and accurate mobile visual localization and its applicationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/24917359:1s(1-22)Online publication date: 17-Oct-2013
https://dl.acm.org/doi/10.1145/2491735
Shirai MHirota MYokoyama SFukuta NIshikawa HCruz IKnoblock CKröger PTanin EWidmayer P(2012)Discovering multiple HotSpots using geo-tagged photographsProceedings of the 20th International Conference on Advances in Geographic Information Systems10.1145/2424321.2424397(490-493)Online publication date: 6-Nov-2012
https://dl.acm.org/doi/10.1145/2424321.2424397
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents