research-article

A comprehensive evaluation methodology for noisy historical document recognition techniques

Authors:

Nikolaos Stamatopoulos,

Georgios Louloudis,

Basilis GatosAuthors Info & Claims

AND '09: Proceedings of The Third Workshop on Analytics for Noisy Unstructured Text Data

Pages 47 - 54

https://doi.org/10.1145/1568296.1568306

Published: 23 July 2009 Publication History

Abstract

In this paper, we propose a new comprehensive methodology in order to evaluate the performance of noisy historical document recognition techniques. We aim to evaluate not only the final noisy recognition result but also the main intermediate stages of text line, word and character segmentation. For this purpose, we efficiently create the text line, word and character segmentation ground truth guided by the transcription of the historical documents. The proposed methodology consists of (i) a semiautomatic procedure in order to detect the text line, word and character segmentation ground truth regions making use of the correct document transcription, (ii) calculation of proper evaluation metrics in order to measure the performance of the final OCR result as well as of the intermediate segmentation stages. The semi-automatic procedure for detecting the ground truth regions has been evaluated and proved efficient and time saving. Experimental results prove that using the proposed technique, the percentage of time saved for the text line, word and character segmentation ground truth creation is more than 90%. An analytic experiment using a commercial OCR engine applied to a historical book is also presented.

References

[1]

ABBYY FineReader OCR: http://finereader.abbyy.com/

[2]

Adamek, T., O'Connor, N. E. and Smeaton, A. F. 2007. Word Matching Using Single-Closed Contours for Indexing Handwritten Historical Documents. International Journal on Document Analysis and Recognition (IJDAR). Vol. 9, No 2--4 (2007), 153--165.

Digital Library

[3]

Chang F., Chen C.-J. and Lu C.-J. A linear-time component-labeling algorithm using contour tracing technique. Comput. Vision Image Understanding. 93 (2) (2004), 206--220.

Digital Library

[4]

Choudhury, G. S., DiLauro, T., Ferguson R., Droettboom, M. and Fujinaga I. 2006. Document Recognition for a Million Books. D-Lib Magazine. Vol. 12, No. 3 (2006).

[5]

Gatos B., Antonacopoulos, A. and Stamatopoulos, N. 2007. Handwriting Segmentation Contest. 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, (September 2007), 1284--1288.

Digital Library

[6]

Gatos B., Pratikakis I. and Perantonis S. J. 2006. Adaptive Degraded Document Image Binarization, Pattern Recognition, 39, (2006), 317--327.

Digital Library

[7]

Gezerlis, V. G. and Theodoridis, S. 2002. Optical Character Recognition for the Orthodox Hellenic Byzantine music notation. Pattern Recognition 35 (2002), 895--914.

[8]

Kanai, J. Nartker, T. A. Rice, S. and Nagy, G. 1993. Performance metrics for document understanding systems. Proceedings of the Second International Conference on Document Analysis and Recognition, Tsukuba Science City, Japan, (1993), 424--427.

[9]

Konidaris, T., Gatos, B., Ntzios, K., Pratikakis, I. And Perantonis S. J. 2007. Keyword - Guided Word Spotting in Historical Printed Documents Using Synthetic Data and User feedback. International Journal on Document Analysis and Recognition (IJDAR), Vol. 9, No 2 (2007) 167--177.

Digital Library

[10]

Kornfield, E. M, Manmatha, R. and Allan J. 2004. Text Alignment with Handwritten Documents. In Proceedings of the 1st International Workshop on Document Image Analysis for Libraries (DIAL 2004), 195.

Digital Library

[11]

Laskov, L. 2006. Classification and Recognition of Neume Note Notation in Historical Documents. In Proceedings of the International Conference of Computer Systems and Technologies (CompSysTech 2006), V. 4. 1--4.

[12]

Lavrenko, V., Rath, T. M. and Manmatha R. 2004. Holistic Word Recognition for Handwritten Historical Documents. In Proceedings of the First International Workshop on Document Image Analysis for Libraries (DIAL 2004), 278--287.

Digital Library

[13]

Louloudis G., Gatos, B., Pratikakis, I. and Halatsis, C. 2008. Text line detection in handwritten documents, Pattern Recognition (41), No. 12, (December 2008), 3758--3772.

Digital Library

[14]

Louloudis G., Gatos, B. and Pratikakis, I. 2008. Line and word segmentation of handwritten documents. In International Conference on Frontiers in Handwriting Recognition (ICFHR'08), Montreal, Canada, (August 2008), 247--252.

[15]

Ntzios, K., Gatos, B., Pratikakis, I., Konidaris, T. and Perantonis S. J. 2007. An Old Greek Handwritten OCR System based on an Efficient Segmentation-free Approach. International Journal on Document Analysis and Recognition (IJDAR), Vol. 9, No. 2 (2007) 179--192.

Digital Library

[16]

Phillips, I. and Chhabra, A. 1999. Empirical Performance Evaluation of Graphics Recognition Systems. IEEE Transactions Pattern Analysis and Machine Intelligence, Vol. 21, No. 9, (1999), 849--870.

Digital Library

[17]

Rath, T. M. and Manmatha, R. 2007. Word Spotting for Historical Documents. International Journal on Document Analysis and Recognition (IJDAR), Vol. 9, No 2 (2007), 139--152.

Digital Library

[18]

Stamatopoulos, N., Gatos, B. and Kesidis, A. 2007. Automatic Borders Detection of Camera Document Images. In 2nd Int. Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, (2007), 71--78.

[19]

Vamvakas, G., Gatos, B., Stamatopoulos, N., Louloudis G. and Perantonis, S. J. 2008. A Complete Optical Character Recognition Methodology for Historical Documents. In Proceedings of the DAS, (2008), 525--532.

Digital Library

[20]

Wahl F. M., Wong K. Y. and Casey R. G. Block Segmentation and Text Extraction in Mixed Text/Image Documents. Computer Graphics and Image Processing, 20, (1982), 375--390.

[21]

Zimmermann, M. and Bunke H. 2002. Automatic Segmentation of the IAM Off-line Database for Handwritten English Text. In Proceedings of the 16^th International Conference on Pattern Recognition (ICPR 2002), 35--39.

Digital Library

Cited By

Wu F(2020)Applying Machine Learning in Optical Music Recognition of Numbered Music NotationCognitive Analytics10.4018/978-1-7998-2460-2.ch098(1915-1937)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-2460-2.ch098
Wu F(2017)Applying Machine Learning in Optical Music Recognition of Numbered Music NotationInternational Journal of Multimedia Data Engineering & Management10.4018/IJMDEM.20170701028:3(21-41)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.4018/IJMDEM.2017070102
Wu F(2016)An Evaluation Framework of Optical Music Recognition in Numbered Music Notation2016 IEEE International Symposium on Multimedia (ISM)10.1109/ISM.2016.0134(626-631)Online publication date: Dec-2016
https://doi.org/10.1109/ISM.2016.0134
Show More Cited By

Index Terms

A comprehensive evaluation methodology for noisy historical document recognition techniques
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Document analysis
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
        Video segmentation
      2. Computer vision tasks

Recommendations

Handwritten ZIP code recognition using lexicon free word recognition algorithm
ICDAR '95: Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 2) - Volume 2

The paper describes a new approach to ZIP code recognition using a word recognition algorithm, where a numeral string is recognized as a word. The paper also describes an end to end ZIP code recognition system consisting of tilt/slant correction, line ...
Gujarati Script Recognition
Abstract
Character recognition is the extraction of printed or handwritten text from images into machine-readable format. The extracted text can be easily edited, modified and efficiently stored. While there are several Optical Character Recognition (OCR) ...
Survey on Segmentation and Recognition of Handwritten Arabic Script
Abstract
The issue of handwritten recognition in Arabic script nature has attracted many researchers from both academic and industrial fields. But their efforts have not reached satisfying outcomes till now. In this paper, a survey has been done in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AND '09: Proceedings of The Third Workshop on Analytics for Noisy Unstructured Text Data

July 2009

127 pages

ISBN:9781605584966

DOI:10.1145/1568296

Program Chairs:
Daniel Lopresti
Lehigh University
,
Shourya Roy
Xerox India Innovation Hub
,
Klaus Schulz
University of Munich
,
L. Venkata Subramaniam
IBM India Research Lab

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

AND '09

AND '09: Third Workshop on Analytics for Noisy Unstructured Text Data

July 23 - 24, 2009

Barcelona, Spain

Acceptance Rates

AND '09 Paper Acceptance Rate 15 of 22 submissions, 68%;

Overall Acceptance Rate 15 of 22 submissions, 68%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
282
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu F(2020)Applying Machine Learning in Optical Music Recognition of Numbered Music NotationCognitive Analytics10.4018/978-1-7998-2460-2.ch098(1915-1937)Online publication date: 2020
https://doi.org/10.4018/978-1-7998-2460-2.ch098
Wu F(2017)Applying Machine Learning in Optical Music Recognition of Numbered Music NotationInternational Journal of Multimedia Data Engineering & Management10.4018/IJMDEM.20170701028:3(21-41)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.4018/IJMDEM.2017070102
Wu F(2016)An Evaluation Framework of Optical Music Recognition in Numbered Music Notation2016 IEEE International Symposium on Multimedia (ISM)10.1109/ISM.2016.0134(626-631)Online publication date: Dec-2016
https://doi.org/10.1109/ISM.2016.0134
Carrasco RAntonacopoulos ASchulz K(2014)An open-source OCR evaluation toolProceedings of the First International Conference on Digital Access to Textual Cultural Heritage10.1145/2595188.2595221(179-184)Online publication date: 19-May-2014
https://dl.acm.org/doi/10.1145/2595188.2595221
Blanke TBryant MHedges M(2011)Ocropodium: open source OCR for small-scale historical archivesJournal of Information Science10.1177/016555151142941838:1(76-86)Online publication date: 21-Nov-2011
https://doi.org/10.1177/0165551511429418
Shima TTerasawa KKawashima TBarrett BBrown MManmatha RGehring J(2011)Image processing for historical newspaper archivesProceedings of the 2011 Workshop on Historical Document Imaging and Processing10.1145/2037342.2037363(127-132)Online publication date: 16-Sep-2011
https://dl.acm.org/doi/10.1145/2037342.2037363

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents