research-article

Patent image retrieval: a survey

Authors:

Roland MörzingerAuthors Info & Claims

PaIR '11: Proceedings of the 4th workshop on Patent information retrieval

Pages 3 - 8

https://doi.org/10.1145/2064975.2064979

Published: 24 October 2011 Publication History

Abstract

Drawings are an important component of patents, and many search tasks in the intellectual property domain rely on the comparison of patent drawings. In this paper, we begin with a review of algorithms developed for the automated retrieval of similar images in the patent domain. There is however a larger body of research dedicated to analysis and retrieval of images found in technical documents that is also applicable to the images found in patents. We present an overview of this research for technical drawings, diagrams, charts, plots and chemical structures. Finally, we discuss the evaluation of image retrieval for patents, including short descriptions of the new patent image evaluation tasks in the CLEF-IP and TREC-CHEM evaluation campaigns in 2011.

References

[1]

S. Adams. Electronic non-text material in patent applications - some questions for patent offices, applicants and searchers. World Patent Information, 27(2):99--103, 2005.

[2]

A. Antonacopoulos and D. Kennedy. Information extraction from complex circular charts. Document Analysis and Recognition, International Conference on, 0:784--787, 2001.

Digital Library

[3]

D. V. Bakergem. Image collections in the design studio. The electronic design studio, pages 261--271, 1990.

Digital Library

[4]

A. Barta and I. Vajk. Document image analysis by probabilistic network and circuit diagram extraction. Informatica, An International Journal of Computing and Informatics, 29:291--301, 2005.

[5]

D. Blostein, E. Lank, and R. Zanibbi. Treatment of diagrams in document image analysis. In Proceedings of the First International Conference on Theory and Application of Diagrams, pages 330--344, London, UK, 2000. Springer-Verlag.

Digital Library

[6]

W. Browuer, S. Kataria, S. Das, P. Mitra, and C. L. Giles. Segregating and extracting overlapping data points in two-dimensional plots. In JCDL, pages 276--279, 2008.

Digital Library

[7]

W. H. Clayton M. Enhancing the sketchbook. Proceedings of the Association for Computer Aided Design in Architecture (ACADIA 91), 1991.

[8]

M. Das, R. Manmatha, and E. M. Riseman. Indexing flower patent images using domain knowledge. IEEE Intelligent Systems, 14(5):24--33, 1999.

Digital Library

[9]

R. Datta, D. Joshi, J. Li, and J. Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40(2):1--60, 2008.

Digital Library

[10]

G. Feng, C. Viard-Gaudin, and Z. Sun. On-line hand-drawn electric circuit diagram recognition using 2d dynamic programming. Pattern Recognition, 42(12):3215--3223, 2009.

Digital Library

[11]

I. V. Filippov and M. C. Nicklaus. Optical structure recognition software to recover chemical information: Osra, an open source solution. Journal of Chemical Information and Modeling, 49(3):740--743, 2009.

[12]

L. A. Fletcher and R. Kasturi. A robust algorithm for text string separation from mixed text/graphics images. IEEE Trans. Pattern Anal. Mach. Intell., 10(6):910--918, 1988.

Digital Library

[13]

W. Huang, C. L. Tan, and W. K. Leow. Model-based chart image recognition. In GREC, pages 87--99, 2003.

[14]

B. Huet, G. Guarascio, N. J. Kern, and B. Mérialdo. Relational skeletons for retrieval in patent drawings. In ICIP (2), pages 737--740, 2001.

[15]

S. Kataria, W. Browuer, P. Mitra, and C. L. Giles. Automatic extraction of data points and text blocks from 2-dimensional plots in digital documents. In AAAI, pages 1169--1174, 2008.

Digital Library

[16]

J. List. How drawings could enhance retrieval in mechanical and device patent searching. World Patent Information, 29(3):210--218, 2007.

[17]

X. Lu, S. Kataria, W. J. Brouwer, J. Z. Wang, P. Mitra, and C. L. Giles. Automated analysis of images in documents for intelligent document search. Int. J. Doc. Anal. Recognit.(IJDAR), 12(2):65--81, 2009.

Digital Library

[18]

X. Lu, P. Mitra, J. Z. Wang, and C. L. Giles. Automatic categorization of figures in scientific documents. In JCDL, pages 129--138. ACM, 2006.

Digital Library

[19]

M. Lupu, J. Huang, J. Zhu, and J. Tait. TREC Chemical Information Retrieval - An Evaluation Effort for Chemical IR Systems. World Patent Information, 2011.

[20]

L. Najman, O. Gibot, and S. Berche. Indexing technical drawings using title block structure recognition. In Proc. International Conference on Document Analysis and Recognition (ICDAR), pages 587--591, 2001.

Digital Library

[21]

J. R. Osborn. Theory pictures as trails: diagrams and the navigation of theoretical narratives. Cognitive Science Online, 3.2:14--44, 2005.

[22]

P. Sidiropoulos, S. Vrochidis, and I. Kompatsiaris. Content-based binary image retrieval using the adaptive hierarchical density histogram. Pattern Recognition, 44(4):739--750, 2011.

Digital Library

[23]

A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000.

Digital Library

[24]

A. Tiwari and V. Bansal. Patseek: Content based image retrieval system for patent database. In ICEB, pages 1167--1171, 2004.

[25]

K. Tombre. Analysis of engineering drawings: State of the art and challenges. In GREC, pages 257--264, 1998.

Digital Library

[26]

S. Vrochidis, S. Papadopoulos, A. Moumtzidou, P. Sidiropoulos, E. Pianta, and I. Kompatsiaris. Towards content-based patent image retrieval: A framework perspective. World Patent Information, 32(2):94--106, 2010.

[27]

N. Weber and A. Henrich. Retrieval of technical drawings in dxf format - concepts and problems. In LWA, pages 213--220, 2007.

[28]

Z. Zhiyuan, Z. Juan, and X. Bin. An outward-appearance patent-image retrieval approach based on the contour-description matrix. In FCST, pages 86--89, Washington, DC, USA, 2007. IEEE Computer Society.

Digital Library

[29]

Y. Zhou and C. L. Tan. Chart analysis and recognition in document images. In ICDAR, page 1055, Washington, DC, USA, 2001. IEEE Computer Society.

Digital Library

[30]

Y. P. Zhou and C. L. Tan. Hough technique for bar charts detection and recognition in document images. In International Conference on Image Processing, ICIP 2000, pages 494--497, 2000.

Cited By

An LMatsuura YOshima N(2024)Literature Review: Advanced Computational Tools for Patent AnalysisTechnology and Business Model Innovation: Challenges and Opportunities10.1007/978-3-031-55911-2_47(483-494)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-3-031-55911-2_47
Wang HZhang Y(2023)Learning Efficient Representations for Image-Based Patent RetrievalPattern Recognition and Computer Vision10.1007/978-981-99-8540-1_2(15-26)Online publication date: 25-Dec-2023
https://doi.org/10.1007/978-981-99-8540-1_2
Ghauri JMüller-Budack EEwerth R(2023)Classification of Visualization Types and Perspectives in PatentsLinking Theory and Practice of Digital Libraries10.1007/978-3-031-43849-3_16(182-191)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43849-3_16
Show More Cited By

Index Terms

Patent image retrieval: a survey
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
2. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

The role of utility models in patent filing strategies: evidence from European countries

We examine the role of utility models (UM) in patent filing strategies. With an extensive patent family data from European countries, we explore the structures and characteristics of patent families, which include UMs. A simple typology of patent ...
Binary Patent Image Retrieval Using the Hierarchical Oriented Gradient Histogram
ICSS '15: Proceedings of the 2015 International Conference on Service Science

To confirm the ingenuity of a patent, the drawings appeared in the patent document play a great role in the comparison of similar patent and can further combine with text-based image retrieval for accurate search. Considerable work has been done in ...
Reducing Noises for Recall-Oriented Patent Retrieval
BDCLOUD '14: Proceedings of the 2014 IEEE Fourth International Conference on Big Data and Cloud Computing

Patents have been considered as key enablers for many knowledge- and information-centric companies as well as institutes. The higher the required patent capability, the more important is the need for an effective and efficient patent search system. Many ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PaIR '11: Proceedings of the 4th workshop on Patent information retrieval

October 2011

46 pages

ISBN:9781450309554

DOI:10.1145/2064975

Program Chairs:
Mihai Lupu
Information Retrieval Facility
,
Andreas Rauber
Vienna University of Technology
,
Allan Hanbury
Vienna University of Technology

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '11

Sponsor:

CIKM '11: International Conference on Information and Knowledge Management

October 24, 2011

Glasgow, Scotland, UK

Acceptance Rates

Overall Acceptance Rate 7 of 13 submissions, 54%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
539
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

An LMatsuura YOshima N(2024)Literature Review: Advanced Computational Tools for Patent AnalysisTechnology and Business Model Innovation: Challenges and Opportunities10.1007/978-3-031-55911-2_47(483-494)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-3-031-55911-2_47
Wang HZhang Y(2023)Learning Efficient Representations for Image-Based Patent RetrievalPattern Recognition and Computer Vision10.1007/978-981-99-8540-1_2(15-26)Online publication date: 25-Dec-2023
https://doi.org/10.1007/978-981-99-8540-1_2
Ghauri JMüller-Budack EEwerth R(2023)Classification of Visualization Types and Perspectives in PatentsLinking Theory and Practice of Digital Libraries10.1007/978-3-031-43849-3_16(182-191)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43849-3_16
Wei XWu JAjayi KOyen DAizawa AMandl TCarevic ZHinze AMayr PSchaer P(2022)Visual descriptor extraction from patent figure captionsProceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries10.1145/3529372.3533299(1-5)Online publication date: 20-Jun-2022
https://dl.acm.org/doi/10.1145/3529372.3533299
Yang LGong MAsari V(2020)Diagram Image Retrieval and Analysis: Challenges and Opportunities2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW50498.2020.00098(685-698)Online publication date: Jun-2020
https://doi.org/10.1109/CVPRW50498.2020.00098
Bhatti NHanbury AStottinger J(2018)Contextual local primitives for binary patent image retrievalMultimedia Tools and Applications10.1007/s11042-017-4808-577:7(9111-9151)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1007/s11042-017-4808-5
Bhatti NHanbury A(2018)Image search in patentsInternational Journal on Document Analysis and Recognition10.1007/s10032-012-0197-516:4(309-329)Online publication date: 21-Dec-2018
https://dl.acm.org/doi/10.1007/s10032-012-0197-5
Singh GSingh ANaik DSeema S(2018)Exploration and Implementation of Classification Algorithms for Patent ClassificationInternational Conference on Intelligent Data Communication Technologies and Internet of Things (ICICI) 201810.1007/978-3-030-03146-6_62(565-572)Online publication date: 21-Dec-2018
https://doi.org/10.1007/978-3-030-03146-6_62
Rusiñol MLladós J(2017)Flowchart Recognition in Patent Information RetrievalCurrent Challenges in Patent Information Retrieval10.1007/978-3-662-53817-3_13(351-368)Online publication date: 26-Mar-2017
https://doi.org/10.1007/978-3-662-53817-3_13
Ni HGuo ZHuang B(2015)Patent Image Classification Using Local-Constrained Linear Coding and Spatial Pyramid MatchingProceedings of the 2015 International Conference on Service Science10.1109/ICSS.2015.13(28-31)Online publication date: 8-May-2015
https://dl.acm.org/doi/10.1109/ICSS.2015.13
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents