TexRGAN: a deep adversarial framework for text restoration from deformed handwritten documents

Free form handwritten document images commonly contains deformed text images such as struck-out and underlined words. The deformed text images drastically degrades the performance of intensely used document processing applications like optical character recognition (OCR). Here we propose an end-to-end text image restoration system based on generative adversarial network (GAN). The proposed model TexRGAN is perhaps the first attempt to restore deformed handwritten texts like struck-out and underlined text images using GAN model and it simultaneously handles strikeout and underline words both with a single deep network model. The proposed GAN model uses spatial as well as structural loss to generate restored text images from a given deformed text image input as condition. The proposed network is trained in weakly supervised approach to avoid unavailability of training data and the cost and error of manual annotations. We evaluate the performance of the proposed TexRGAN on various types of deformation and shapes of strike-through-strokes such as slanted strokes, nearly straight strokes, multiple strokes, underlines, crossed strokes etc. The TexRGAN is also evaluated directly in terms the OCR performance. The evaluation metrics show robustness and applicability in real-world scenario.