Abstract
The transcription of written text images is one of the most challenging tasks in document analysis since it has to cope with the variability and ambiguity encountered in handwritten data. Only in a very restricted setting, as encountered in postal addresses or bank checks, transcription works well enough for commercial applications. In the case of unconstrained modern handwritten text, recent advances have pushed the field towards becoming interesting for practical applications. For historic data, however, recognition accuracies are still far too low for automatic systems. Instead, recent efforts aim at interactive solutions in which the computer merely assists an expert creating a transcription. In this chapter, an overview of the field is given and the steps along the processing chain from the text line image to the final output are explained, starting with image normalization and feature representation. Two recognition approaches, based on hidden Markov models and neural networks, are introduced in more detail. Finally, databases and software toolkits are presented, and hints to further material are provided.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arica N, Yarmal-Vural FT (2002) Optical character recognition for cursive handwriting. IEEE Trans Pattern Anal Mach Intell 24(6):801–814
Ahmed P, Suen CY (1987) Computer recognition of totally unconstrained handwritten zip codes. Int J Pattern Recognit Artif Intell 1(1):1–15
Baum LE, Petrie T, Soules G, Weiss N (1970) A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat 41(1):164–171
Bazzi I, Schwartz R, Makhoul J (1999) An omnifont open-vocabulary OCR system for English and Arabic. IEEE Trans Pattern Anal Mach Intell 21(6):495–505
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient decent is difficult. IEEE Trans Neural Netw 5(2):157–166
Boz̆inović RM, Srihari S (1989) Off-line cursive script word recognition. IEEE Trans Pattern Anal Mach Intell 11(1):68–83
Brakensiek A, Rigoll G (2004) Handwritten address recognition using hidden Markov models. In: Dengel A et al (eds) Reading and learning. Volume 2956 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 103–122
Caillault É, Viard-Gaudin C (2007) Mixed discriminant training of hybrid ANN/HMM. Int J Pattern Recognit Artif Intell 21(1):117–134
Chen J, Cao H, Prasad R, Bhardwaj A, Natarajan P (2010) Gabor features for offline Arabic handwriting recognition. In: 9th IARP international workshop on document analysis systems, Boston, pp 53–58
Collobert R, Bengio S, Mariéthoz J (2002) TORCH: a modular machine learning software library. Technical report IDIAP-RR 02-46, IDIAP Research Institute
Dietterich T (2009) Machine learning for sequential data: a review. In: Caelli T, Amin A, Duin R, de Ridder D, Kamel M (eds) Structural, syntactic, and statistical pattern recognition. Volume 2396 of lecture notes in computer science. Springer, Berlin/Heidelberg, pp 227–246
Do TMT, Artieres T (2006) Conditional random fields for online handwriting recognition. In: International workshop of frontiers in handwriting recognition, La Baule, pp 197–204
Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley-Interscience, New York
España-Boquera S, Castro-Bleda MJ, Gorbe-Moya J, Zamora-Martinez F (2010) Improving offline handwritten text recognition with hybrid HMM/ANN models. IEEE Trans Pattern Anal Mach Intell 33(4):767–779
Fischer A, Bunke H (2009) Kernel PCA for HMM-based cursive handwriting recognition. In: 13th international conference on computer analysis of images and pattern, Münster, pp 181–188
Fischer A, Riesen K, Bunke H (2010) Graph similarity features for HMM-based handwriting recognition in historical documents. In: 12th international conference on frontiers in handwriting recognition, Kolkata. pp 253–258
Fischer A, Keller A, Frinken V, Bunke H (2011, submitted) Lexicon-free handwritten word spotting using character HMMs pattern recognition letters 33(7):934–942
Frinken V, Fischer A, Manmatha R, Bunke H (2012) A novel word spotting method based on recurrent neural networks. IEEE Trans Pattern Anal Mach Intell 34(2), pp 211, 224
Gader PD, Mohamed MA, Chiang J-H (1995) Comparison of crisp and fuzzy character neural network in handwritten word recognition. IEEE Trans Fuzzy Syst 3(3):357–364
Gader PD, Mohamed MA, Chiang J-H (1997) Handwritten word recognition with character and inter-character neural networks. IEEE Trans Syst Man Cybern 27(1):158–164
Goodman JT (2001) A bit of progress in language modeling – extended version. Technical report MSR-TR-2001-72, Microsoft Research, One Microsoft Way Redmond, WA 98052, 8
Gorski N, Anisimov V, Augustin E, Baret O, Maximov S (2001) Industrial bank check processing: the A2iA CheckReader(tm). Int J Doc Anal Recognit 3(4):196–206
Graves A (2012) Supervised sequence labelling with recurrent neural networks. Springer, Heidelberg/New York/Dordrecht/London
Graves A, Schmidhuber J (2009) Offline handwriting recognition with multidimensional recurrent neural networks. In: Koller D et al (eds) Advances in neural information processing systems 21. MIT Press, Cambridge, pp 545–552
Graves A, Liwicki M, Fernández S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intell 31(5):855–868
Guyon I, Schomaker L, Plamondon R, Liberman M, Janet S (1994) UNIPEN project of on-line data exchange and recognizer benchmarks. In: 12th international conference on pattern recognition, Jerusalem, pp 29–33
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. SIGKDD Explor 11(1):10–18
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Jiang H (2005) Confidence measures for speech recognition: a survey. Speech Commun 45:455–470
Johanson S, Leech GN, Goodluck H (1978) Manual of information to accompany the Lancaster-Oslo/Bergen corpus of British English, for use with digital computers. Technical report, Department of English, University of Oslo, Norway
Kavallieratou E, Fakotakis N, Kokkinakis GK (2002) An unconstrained handwriting recognition system. Int J Doc Anal Recognit 4(4):226–242
Knerr S, Augustin E, Baret O, Price D (1998) Hidden Markov model based word recognition and its application to legal amount reading on French checks. Comput Vis Image Underst 70(3):404–419
Koerich AL, Sabourin R, Suen CY (2003) Lexicon-driven HMM decoding for large vocabulary handwriting recognition with multiple character models. Int J Doc Anal Recognit 6:126–144
Lee S-W (ed) (1999) Advances in handwriting recognition. World Scientific, Singapore/River Edge/London
Liwicki M, Graves A, Bunke H (2012) Neural networks for handwriting recognition. In: Ogiela MR, Jain LC (eds) Computation cation, vol 386/2012. Springer, Berlin/Heidelberg, pp 5–24
Lorigo LM, Govindaraju V (2006) Offline Arabic handwriting recognition: a survey. IEEE Trans Pattern Anal Mach Intell 28(5):712–725
Marti U-V (2000) Off-line recognition of handwritten texts. PhD thesis, University of Bern, Bern
Marti U-V, Bunke H (2001) Using a statistical language model to improve the performance of an HMM-based cursive handwriting recognition system. Int J Pattern Recognit Artif Intell 15:65–90
Marti U-V, Bunke H (2002) The IAM-database: an English sentence database for offline handwriting recognition. Int J Doc Anal Recognit 5:39–46
Ogawa A, Takeda K, Itakura F (1998) Balancing acoustic and linguistic probabilities. In: International conference on acoustic, speech, and signal processing, Seattle, pp 181–184
O’Reilly RC, Frank MJ (2003) Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Ics-03-03, ICS, Mar 2003
Pechwitz M, Maergner V (2003) HMM based approach for handwritten Arabic word recognition using the IFN/ENIT-database. In: 7th international conference on document analysis and recognition, Edinburgh, pp 890–894
Plötz T, Fink G (2011) Markov models handwriting recognition. Springer, London/Dordrecht/ Heidelberg/New York
Pramod Sankar K, Ambati V, Pratha L, Jawahar CV (2006) Digitizing a million books: challenges for document analysis. In: 7th IAPR workshop on document analysis systems, Nelson, pp 425–436
Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286
Rath TM, Manmatha R (2007) Word spotting for historical documents. Int J Doc Anal Recognit 9:139–152
RodrÃguez JA, Perronnin F (2008) Local gradient histogram features for word spotting in unconstrained handwritten documents. In: 11th international conference frontiers in handwriting recognition, Montréal, pp 7–12
RodrÃguez-Serrano JA, Perronnin F, Sánchez G, Lladós J (2010) Unsupervised writer adaptation of whole-word HMMs with application to word-spotting. Pattern Recognit Lett 31(8):742–749
Rutovitz D (1966) Pattern recognition. J R Stat Soc A (General) 129(4):504–530
Sayre KM (1973) Machine recognition of handwritten words: a project report. Pattern Recognit 3(3):213–228
Seiler R, Schenkel M, Eggimann F (1996) Off-line cursive handwriting recognition compared with on-line recognition. In: 13th international conference on pattern recognition, Vienna, vol 4, pp 505–509
Senior AW, Robinson AJ (1998) An off-line cursive handwriting recognition system. IEEE Trans Pattern Anal Mach Intell 20(3):309–321
Srihari SN, Srinivasan H, Huang C, Shetty S (2006) Spotting words in Latin, Devanagari and Arabic scripts. Indian J Artif Intell 16(3):2–9
Stolke A (2002) SRILM – an extensible language modeling toolkit. In: International conference on spoken language processing, Denver, pp 901–904
Taira E, Uchida S, Sakoe H (2004) Nonuniform slant correction for handwritten word recognition. IEICE Trans Inf Syst E87-D(5):1247–1253
Terasawa K, Tanaka Y (2009) Slit style HOG features for document image word spotting. In: 10th international conference on document analysis and recognition, Barcelona, vol 1, pp 116–120
Toselli AH, Juan A, González J, Salvador I, Vidal E, Casacuberta F, Keysers D, Ney H (2004) Integrated handwritten recognition and interpretation using finite-state models. Int J Pattern Recognit Artif Intell 18(4):519–539
Toselli AH, Vidal E, Casacuberta F (2011) Multimodal interactive pattern recognition and applications. Springer, London/New York
van der Zant T, Schomaker L, Haak K (2008) Handwritten-word spotting using biologically inspired features. IEEE Trans Pattern Anal Mach Intell 30(11):1945–1957
Vinciarelli A (2002) A survey on off-line cursive word recognition. Pattern Recognit 35(7):1433–1446
Vinciarelli A (2003) Offline cursive handwriting: from word to text recognition. Technical report IDIAP-PP 03-24, Institut Dalle Molle Intelligance Artificielle Perceptive (IDIAP), Martigny
Vinciarelli A, Bengio S, Bunke H (2004) Offline recognition of unconstrained handwritten texts using HMMs and statistical language models. IEEE Trans Pattern Anal Mach Intell 26(6): 709–720
Viterbi A (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory 13:260–269
Werbos PJ (1988) Generalization of backpropagation to a recurrent gas model. Neural Netw 1:339–356
Wienecke M, Fink GA, Gerhard S (2005) Toward automatic video-based whiteboard reading. Int J Doc Anal Recognit 7:188–200
Young S, Evermann G, Gales M, Hain T, Kershaw D, Liu X, Moore G, Odell J, Ollason D, Povey D, Valtchev V, Woodland P (2006) The HTK book. Technical report, Cambridge University Engineering Department, Dec 2006
Further Reading
Fischer A, Keller A, Frinken V, Bunke H (2011, submitted) Lexicon-free handwritten word spotting using character HMMs
Frinken V, Fischer A, Manmatha R, Bunke H (2012) A novel word spotting method based on recurrent neural networks. IEEE Trans Pattern Anal Mach Intell. Accepted for publication
Goodman JT (2001) A bit of progress in language modeling – extended version. Technical report MSR-TR-2001-72, Microsoft Research, One Microsoft Way Redmond, WA 98052, 8
Graves A (2012) Supervised sequence labelling with recurrent neural networks. Springer, Heidelberg/New York/Dordrecht/London
Jiang H (2005) Confidence measures for speech recognition: a survey. Speech Commun 45:455–470
Liwicki M, Graves A, Bunke H (2012) Neural networks for handwriting recognition. In: Ogiela MR, Jain LC (eds) Computational intelligence paradigms in advanced pattern classification, vol 386/2012. Springer, Berlin/Heidelberg, pp 5–24
Plötz T, Fink G (2011) Markov models handwriting recognition. Springer, London/Dordrecht/ Heidelberg/New York
Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286
Toselli AH, Vidal E, Casacuberta F (2011) Multimodal interactive pattern recognition and applications. Springer, London/New York
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag London
About this entry
Cite this entry
Frinken, V., Bunke, H. (2014). Continuous Handwritten Script Recognition. In: Doermann, D., Tombre, K. (eds) Handbook of Document Image Processing and Recognition. Springer, London. https://doi.org/10.1007/978-0-85729-859-1_12
Download citation
DOI: https://doi.org/10.1007/978-0-85729-859-1_12
Published:
Publisher Name: Springer, London
Print ISBN: 978-0-85729-858-4
Online ISBN: 978-0-85729-859-1
eBook Packages: Computer ScienceReference Module Computer Science and Engineering