Abstract
Today's digital libraries increasingly include not only printed text but also scanned handwritten pages and other multimedia material. There are, however, few tools available for manipulating handwritten pages. Here, we extend our algorithm from [5]based on dynamic time warping (DTW) for a word by word alignment of handwritten documents with (ASCII) transcripts. We specifically attempt to incorporate language modelling and parameter training into our algorithm. In addition, we take a critical look at our evaluation metrics. We see at least three uses for such alignment algorithms. First, alignment algorithms allow us to produce displays (for example on the web) that allow a person to easily find their place in the manuscript when reading a transcript. Second, such alignment algorithms will allow us to produce large quantities of ground truth data for evaluating handwriting recognition algorithms. Third, such algorithms allow us to produce indices in a straightforward manner for handwriting material. We provide experimental results of our algorithm on a set of 100 pages of historical handwritten material–specifically the writings of George Washington. Our method achieves average F-measure values of 68.3 online by line alignment and 57.8 accuracy when aligning whole pages at time.
Similar content being viewed by others
References
Ho, T., Nagy, G.: OCR with no shape training. In: Proceedings of 15th ICPR, pp. 27–30. Barcelona (2000)
Hobby, J.D.: Matching document images with ground truth. Int. J. Doc Anal. Recognit. 1(1), 52–61 (1998)
Kane, S., Lehman, A., Partridge, E.: Indexing George Washington's handwritten manuscripts. Technical Report MM-34, Center for Intelligent Information Retrieval. University of Massachusetts Amherst (2001)
Kay, M., Röscheisen, M.: Text-translation alignment. Comput. Linguist. 19(1), 121–142 (1993)
Kornfield, E.M., Manmatha, R., Allan, J.: Text alignment with handwritten documents. In: Proceedings of DIAL, pp.195–211. Palo Alto, California (2004)
Levenshtein, V.I.: Binary codes capable of correcting spurious insertions and deletions of ones. Russian Problemy Peredachi Informatsii 1, 12–25 (1965) (Original in Russian. English translation in Problems of Information Transmission 1, 8–17 (1965))
Manmatha, R., Rothfeder, J.: A scale space approach for automatically segmenting words from historical handwritten documents. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1212–1225 (2005)
Manmatha, R., Srimal, N.: Scale space technique for word segmentation in handwritten documents. In: Scale-Space Theories in Computer Vision pp. 22–33 (1999)
Press, W., Teukolsky, S., Vetterling, W., Flannery, B.: Numerical Recipes in C. Cambridge University Press, Cambridge, UK (1993)
Rath, T., Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of CVPR-03, vol. 2, pp. 521–527. Madison, WI (2003)
Rath, T.M., Lavrenko, V., Manmatha, R.: A statistical approach to retrieving historical manuscript images without recognition. Technical Report MM-42, Center for Intelligent Information Retrieval, University of Massachusetts Amherst (2003)
Roy, D.K., Malamud, C.: Speaker identification based text to audio alignment for an audio retrieval system. In: Proceedings of ICASSP '97, pp. 1099–1102. Munich, Germany (1997)
Sakoe, H., Chiba, S.: Dynamic programming optimization for spoken work recognition. IEEE Trans. Acoust. Speeh Signal Process. 26, 623–625 (1980)
Tomai, C., Zhang, B., Govindaraju, V.: Transcript mapping for historic handwritten document images. In: Proceedings of the 8th International Workshop on Frontiers in Handwriting Recognition, pp. 413–418. Niagara-on-the-Lake, ON (2002)
Triebel, R.: Automatische erkennung von handgeschriebenen worten mithilfe des level-building algorithmus. Master's thesis, Institut fur Informatik, alber-Ludwigs-Universtät Freiburg (1999) (in German)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kornfield, E.M., Manmatha, R. & Allan, J. Further explorations in text alignment with handwritten documents. IJDAR 10, 39–52 (2007). https://doi.org/10.1007/s10032-006-0019-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10032-006-0019-8