research-article

Sketch-based shape retrieval

Authors:

Ronald Richter,

Tamy Boubekeur,

Kristian Hildebrand,

Marc AlexaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 4

Article No.: 31, Pages 1 - 10

https://doi.org/10.1145/2185520.2185527

Published: 01 July 2012 Publication History

Abstract

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.

Supplementary Material

JPG File (tp103_12.jpg)

Download
16.42 KB

ZIP File (a31-eitz.zip)

Supplemental material.

Download
102.95 MB

MP4 File (tp103_12.mp4)

Download
39.80 MB

References

[1]

Bay, H., Tuytelaars, T., and Gool, L. J. V. 2006. SURF: Speeded up robust features. In ECCV, 404--417.

Digital Library

[2]

Bülthoff, H., and Edelman, S. 1992. Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proc. National Academy of Sciences 89, 1, 60--64.

[3]

Candès, E. J., and Donoho, D. L. 1999. Curvelets -- a surprisingly effective nonadaptive representation for objects with edges. In Int'l. Conf. Curves and Surfaces, 105--120.

[4]

Canny, J. 1986. A computational approach to edge detection. IEEE TPAMI 8, 6, 679--698.

Digital Library

[5]

Chen, D.-Y., Tian, X.-P., Shen, Y.-T., and Ouhyoung, M. 2003. On visual similarity based 3d model retrieval. Comput. Graph. Forum (Proc. Eurographics) 22, 3, 223--232.

[6]

Chen, T., Cheng, M., Tan, P., Shamir, A., and Hu, S. 2009. Sketch2Photo: internet image montage. ACM TOG (Proc. SIGGRAPH ASIA) 28, 5, 124:1--124:10.

Digital Library

[7]

Cole, F., Golovinskiy, A., Limpaecher, A., Barros, H. S., Finkelstein, A., Funkhouser, T., and Rusinkiewicz, S. 2008. Where do people draw lines? ACM TOG (Proc. SIGGRAPH) 27, 3, 88:1--88:11.

Digital Library

[8]

Daras, P., and Axenopoulos, A. 2010. A 3D shape retrieval framework supporting multimodal queries. IJCV 89, 2, 229--247.

Digital Library

[9]

DeCarlo, D., Finkelstein, A., Rusinkiewicz, S., and Santella, A. 2003. Suggestive contours for conveying shape. ACM TOG (Proc. SIGGRAPH) 22, 3, 848--855.

Digital Library

[10]

Dutagaci, H., Cheung, C. P., and Godil, A. 2010. A benchmark for best view selection of 3D objects. In Proc. ACM workshop on 3D object retrieval, 45--50.

Digital Library

[11]

Eitz, M., Richter, R., Hildebrand, K., Boubekeur, T., and Alexa, M. 2011. Photosketcher: interactive sketch-based image synthesis. IEEE CG&A 31, 6, 56--66.

Digital Library

[12]

Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. 2003. A search engine for 3D models. ACM TOG 22, 1, 83--105.

Digital Library

[13]

Hou, S., and Ramani, K. 2006. Sketch-based 3D engineering part class browsing and retrieval. In Sketch-Based Interfaces and Modeling, 131--138.

Digital Library

[14]

Jacobs, C. E., Finkelstein, A., and Salesin, D. H. 1995. Fast multiresolution image querying. In Proc. SIGGRAPH 95, 277--286.

Digital Library

[15]

Lee, J., and Funkhouser, T. 2008. Sketch-based search and composition of 3D models. In Sketch-Based Interfaces and Modeling, 97--104.

Digital Library

[16]

Lee, Y., Zitnick, C., and Cohen, M. 2011. ShadowDraw: real-time user guidance for freehand drawing. ACM TOG (Proc. SIGGRAPH) 30, 4, 27:1--27:10.

Digital Library

[17]

Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Trans. Information Theory 28, 2, 129--137.

Digital Library

[18]

Löffler, J. 2000. Content-based retrieval of 3D models in distributed web databases by visual shape information. In Int'l. Conf. Information Visualization, 82--87.

Digital Library

[19]

Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV 60, 2, 91--110.

Digital Library

[20]

Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 3, 145--175.

Digital Library

[21]

Pu, J., Lou, K., and Ramani, K. 2005. A 2D sketch-based user interface for 3D CAD model retrieval. Computer-Aided Design and Applications 2, 6, 717--725.

[22]

Schölkopf, B., and Smola, A. 2002. Learning with Kernels. The MIT Press.

[23]

Secord, A., Lu, J., Finkelstein, A., Singh, M., and Nealen, A. 2011. Perceptual models of viewpoint preference. ACM TOG 30, 5, 109:1--109:12.

Digital Library

[24]

Shao, T., Xu, W., Yin, K., Wang, J., Zhou, K., and Guo, B. 2011. Discriminative sketch-based 3D model retrieval via robust shape matching. Computer Graphics Forum (Proc. Pacific Graphics) 30, 7, 2011--2020.

[25]

Shilane, P., Min, P., Kazhdan, M., and Funkhouser, T. 2004. The Princeton Shape Benchmark. In Proc. Shape Modeling International, 167--178.

Digital Library

[26]

Shin, H., and Igarashi, T. 2007. Magic canvas: interactive design of a 3-D scene prototype from freehand sketches. In Proc. Graphics Interface, 63--70.

Digital Library

[27]

Sivic, J., and Zisserman, A. 2003. Video Google: a text retrieval approach to object matching in videos. In ICCV, 1470--1477.

Digital Library

[28]

Squire, D., Mueller, W., Mueller, H., and Raki, J. 1999. Content-based query of image databases. In Scand. Conf. on Image Analysis, 143--149.

[29]

Tangelder, J., and Veltkamp, R. 2008. A survey of content based 3D shape retrieval methods. Multimedia Tools and Applications 39, 441--471.

Digital Library

[30]

Witten, I., Moffat, A., and Bell, T. 1999. Managing gigabytes: compressing and indexing documents and images. Morgan Kaufmann.

Digital Library

[31]

Yoon, S., Scherer, M., Schreck, T., and Kuijper, A. 2010. Sketch-based 3d model retrieval using diffusion tensor fields of suggestive contours. In Int'l. Conf. Multimedia, 193--200.

Digital Library

Cited By

Binninger AHertz ASorkine‐Hornung OCohen‐Or DGiryes R(2024)SENS: Part‐Aware Sketch‐based Implicit Neural Shape ModelingComputer Graphics Forum10.1111/cgf.1501543:2Online publication date: 23-Apr-2024
https://doi.org/10.1111/cgf.15015
Luo ZDu DZhu HYu YFu HHan X(2024)SketchMetaFace: A Learning-Based Sketching Interface for High-Fidelity 3D Character Face ModelingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329170330:8(5260-5275)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3291703
Chen TDing CZhu LZang YLiao YLi ZSun L(2024)Reality3DSketch: Rapid 3D Modeling of Objects From Single Freehand SketchesIEEE Transactions on Multimedia10.1109/TMM.2023.332753326(4859-4870)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3327533
Show More Cited By

Index Terms

Sketch-based shape retrieval
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
    2. Shape modeling
      1. Parametric curve and surface models
2. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Sketch-based image retrieval using keyshapes

Although sketch based image retrieval (SBIR) is still a young research area, there are many applications capable of exploiting this retrieval paradigm, such as web searching and pattern detection. Moreover, nowadays drawing a simple sketch query turns ...
Utilizing shape retrieval in sketch synthesis

This article describes SR-Sketch, a sketch creation tool that can act both as a front-end visual query module to visual information retrieval systems and as an aid tool for fast image composition. The system allows the user to draw shapes on the ...
3D sketch-based 3D shape retrieval
3DOR '16: Proceedings of the Eurographics 2016 Workshop on 3D Object Retrieval

Sketch-based 3D shape retrieval has unique representation availability of the queries and vast applications. Therefore, it has received more and more attentions in the research community of content-based 3D object retrieval. However, sketch-based 3D ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 4

July 2012

935 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2185520

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2012

Published in TOG Volume 31, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

182
Total Citations
View Citations
2,127
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)5

Reflects downloads up to 14 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Binninger AHertz ASorkine‐Hornung OCohen‐Or DGiryes R(2024)SENS: Part‐Aware Sketch‐based Implicit Neural Shape ModelingComputer Graphics Forum10.1111/cgf.1501543:2Online publication date: 23-Apr-2024
https://doi.org/10.1111/cgf.15015
Luo ZDu DZhu HYu YFu HHan X(2024)SketchMetaFace: A Learning-Based Sketching Interface for High-Fidelity 3D Character Face ModelingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329170330:8(5260-5275)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3291703
Chen TDing CZhu LZang YLiao YLi ZSun L(2024)Reality3DSketch: Rapid 3D Modeling of Objects From Single Freehand SketchesIEEE Transactions on Multimedia10.1109/TMM.2023.332753326(4859-4870)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3327533
Du XZhang TXie H(2024)DualShape: Sketch-Based 3D Shape Design With Part Generation and RetrievalIEEE Access10.1109/ACCESS.2024.336165912(18888-18900)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3361659
Liang SDai WCai YXie C(2024)Sketch-based 3D shape retrieval via teacher–student learningComputer Vision and Image Understanding10.1016/j.cviu.2023.103903239(103903)Online publication date: Feb-2024
https://doi.org/10.1016/j.cviu.2023.103903
Shibuya HNagai Y(2023)Automatic Parts Correspondence Determination for Transforming Assemblies via Local and Global Geometry ProcessingInternational Journal of Automation Technology10.20965/ijat.2023.p017617:2(176-182)Online publication date: 5-Mar-2023
https://doi.org/10.20965/ijat.2023.p0176
Ge CWang JQi QSun HXu TLiao JWilliams BChen YNeville J(2023)Semi-transductive learning for generalized zero-shot sketch-based image retrievalProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i6.25931(7678-7686)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i6.25931
Yating LJing XLiang LZheng WWenyi CMi W(2023)A review of disentangled representation learning for visual data processing and analysisJournal of Image and Graphics10.11834/jig.21126128:4(903-934)Online publication date: 2023
https://doi.org/10.11834/jig.211261
Zhang STam HLi YMu TZhang S(2023)SceneViewer: Automating Residential Photography in Virtual EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.321483629:12(5523-5537)Online publication date: Dec-2023
https://doi.org/10.1109/TVCG.2022.3214836
Xu PHospedales TYin QSong YXiang TWang L(2023)Deep Learning for Free-Hand Sketch: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.314885345:1(285-312)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TPAMI.2022.3148853
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents