Article

Document Dewarping with Control Points

Authors:

Cheng-Lin LiuAuthors Info & Claims

Document Analysis and Recognition – ICDAR 2021: 16th International Conference, Lausanne, Switzerland, September 5–10, 2021, Proceedings, Part I

Pages 466 - 480

https://doi.org/10.1007/978-3-030-86549-8_30

Published: 05 September 2021 Publication History

Abstract

Document images are now widely captured by handheld devices such as mobile phones. The OCR performance on these images are largely affected due to geometric distortion of the document paper, diverse camera positions and complex backgrounds. In this paper, we propose a simple yet effective approach to rectify distorted document image by estimating control points and reference points. After that, we use interpolation method between control points and reference points to convert sparse mappings to backward mapping, and remap the original distorted document image to the rectified image. Furthermore, control points are controllable to facilitate interaction or subsequent adjustment. We can flexibly select post-processing methods and the number of vertices according to different application scenarios. Experiments show that our approach can rectify document images with various distortion types, and yield state-of-the-art performance on real-world dataset. This paper also provides a training dataset based on control points for document dewarping. Both the code and the dataset are released at https://github.com/gwxie/Document-Dewarping-with-Control-Points.

References

[1]

Brown MS and Tsoi YC Geometric and shading correction for images of printed materials using boundary IEEE Trans. Image Process. 2006 15 6 1544-1554

Digital Library

[2]

Courteille F, Crouzil A, Durou JD, and Gurdjos P Shape from shading for the digitization of curved documents Mach. Vision Appl. 2007 18 5 301-316

Digital Library

[3]

Das, S., Ma, K., Shu, Z., Samaras, D., Shilkrot, R.: DewarpNet: single-image document unwarping with stacked 3D and 2D regression networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 131–140 (2019)

[4]

He, Y., Pan, P., Xie, S., Sun, J., Naoi, S.: A book dewarping system by boundary-based 3D surface reconstruction. In: 2013 12th International Conference on Document Analysis and Recognition, pp. 403–407. IEEE (2013)

[5]

Li X, Zhang B, Liao J, and Sander PV Document rectification and illumination correction using a patch-based CNN ACM Trans. Graphics 2019 38 6 1-11

Digital Library

[6]

Liu C, Zhang Y, Wang B, and Ding X Restoring camera-captured distorted document images Int. J. Doc. Anal. Recogn. 2015 18 2 111-124

[7]

Liu W et al. Leibe B, Matas J, Sebe N, Welling M, et al. SSD: single shot multibox detector Computer Vision – ECCV 2016 2016 Cham Springer 21-37

[8]

Liu, X., Meng, G., Fan, B., Xiang, S., Pan, C.: Geometric rectification of document images using adversarial gated unwarping network. Pattern Recogn. 108, 107576 (2020)

[9]

Ma, K., Shu, Z., Bai, X., Wang, J., Samaras, D.: DocUNet: document image unwarping via a stacked U-Net. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4709 (2018)

[10]

Markovitz A, Lavi I, Perel O, Mazor S, and Litman R Vedaldi A, Bischof H, Brox T, and Frahm J-M Can you read me now? Content aware rectification using angle supervision Computer Vision – ECCV 2020 2020 Cham Springer 208-223

Digital Library

[11]

Meijering E A chronology of interpolation: from ancient astronomy to modern signal and image processing Proc. IEEE 2002 90 3 319-342

[12]

Ramanna, V., Bukhari, S.S., Dengel, A.: Document image dewarping using deep learning. In: International Conference on Pattern Recognition Applications and Methods (2019)

[13]

Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Neural Information Processing Systems (2015)

[14]

Sorkine, O.: Laplacian mesh processing. In: Eurographics (STARs), p. 29 (2005)

[15]

Stamatopoulos N, Gatos B, Pratikakis I, and Perantonis SJ Goal-oriented rectification of camera-based document images IEEE Trans. Image Process. 2010 20 4 910-920

Digital Library

[16]

Tian, Y., Narasimhan, S.G.: Rectification and 3D reconstruction of curved document images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 377–384. IEEE (2011)

[17]

Tsoi, Y.C., Brown, M.S.: Multi-view document rectification using boundary. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

[18]

Wada T, Ukida H, and Matsuyama T Shape from shading with interreflections under a proximal light source: Distortion-free copying of an unfolded book Int. J. Comput. Vision 1997 24 2 125-135

Digital Library

[19]

Wang N, Zhang Y, Li Z, Fu Y, Liu W, and Jiang Y-G Ferrari V, Hebert M, Sminchisescu C, and Weiss Y Pixel2Mesh: generating 3D mesh models from single RGB images Computer Vision – ECCV 2018 2018 Cham Springer 55-71

Digital Library

[20]

Wang, Z., Simoncelli, E.P., Bovik, A.C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, pp. 1398–1402. IEEE (2003)

[21]

Xie G-W, Yin F, Zhang X-Y, and Liu C-L Bai X, Karatzas D, and Lopresti D Dewarping document image by displacement flow estimation with fully convolutional network Document Analysis Systems 2020 Cham Springer 131-144

Digital Library

[22]

You S, Matsushita Y, Sinha S, Bou Y, and Ikeuchi K Multiview rectification of folded documents IEEE Trans. Pattern Anal. Mach. Intell. 2017 40 2 505-511

Digital Library

[23]

Zhang L, Yip AM, Brown MS, and Tan CL A unified framework for document restoration using inpainting and shape-from-shading Pattern Recogn. 2009 42 11 2961-2978

Digital Library

Cited By

Wan XZhang CLyu PFan SNi ZYao KDing EWang JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Towards unified multi-granularity text detection with interactive attentionProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694116(50012-50025)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694116
Wigington C(2024)Handheld Video Document Scanning: A Robust On-Device Model for Multi-Page Document ScanningProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685662(1-9)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685662
Zhu ZTang ZGao L(2024)Table image dewarping with key element segmentationInternational Journal on Document Analysis and Recognition10.1007/s10032-024-00480-z27:3(349-362)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10032-024-00480-z
Show More Cited By

Recommendations

Dewarping Document Image in Complex Scene by Geometric Control Points
Pattern Recognition
Abstract
In the process of document digitization, document images captured by mobile devices suffer from physical distortion, which is detrimental to subsequent document processing. Geometric information of the distorted document images provide global and ...
Dewarping Document Image by Displacement Flow Estimation with Fully Convolutional Network
Document Analysis Systems
Abstract
As camera-based documents are increasingly used, the rectification of distorted document images becomes a need to improve the recognition performance. In this paper, we propose a novel framework for both rectifying distorted document image and ...
Automatic Corresponding Control Points Selection for Historical Document Image Registration
ICDAR '09: Proceedings of the 2009 10th International Conference on Document Analysis and Recognition

Image registration is crucial for various image analysis tasks. In particular, most approaches to correction of bleed-through distortion on handwritten document images require the recto image and the verso image to be precisely registered. In this paper,...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

Document Analysis and Recognition – ICDAR 2021: 16th International Conference, Lausanne, Switzerland, September 5–10, 2021, Proceedings, Part I

Sep 2021

652 pages

ISBN:978-3-030-86548-1

DOI:10.1007/978-3-030-86549-8

Editors:
Josep Lladós
Universitat Autònoma de Barcelona, Barcelona, Spain
,
Daniel Lopresti
Lehigh University, Bethlehem, PA, USA
,
Seiichi Uchida
Kyushu University, Fukuoka-shi, Japan

© Springer Nature Switzerland AG 2021.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 September 2021

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wan XZhang CLyu PFan SNi ZYao KDing EWang JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Towards unified multi-granularity text detection with interactive attentionProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694116(50012-50025)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3694116
Wigington C(2024)Handheld Video Document Scanning: A Robust On-Device Model for Multi-Page Document ScanningProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685662(1-9)Online publication date: 20-Aug-2024
https://dl.acm.org/doi/10.1145/3685650.3685662
Zhu ZTang ZGao L(2024)Table image dewarping with key element segmentationInternational Journal on Document Analysis and Recognition10.1007/s10032-024-00480-z27:3(349-362)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10032-024-00480-z
Zhang WWang QHuang KGu XGuo F(2024)Coarse-to-Fine Document Image Registration for DewarpingDocument Analysis and Recognition - ICDAR 202410.1007/978-3-031-70546-5_20(343-358)Online publication date: 30-Aug-2024
https://dl.acm.org/doi/10.1007/978-3-031-70546-5_20
Li PQuan WGuo JYan D(2023)Layout-aware Single-image Document FlatteningACM Transactions on Graphics10.1145/362781843:1(1-17)Online publication date: 2-Nov-2023
https://dl.acm.org/doi/10.1145/3627818
Verhoeven FMagne TSorkine-Hornung O(2023)UVDoc: Neural Grid-based Document UnwarpingSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618174(1-11)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.1145/3610548.3618174
Li RYin FHuang L(2023)Dewarping Document Image in Complex Scene by Geometric Control PointsPattern Recognition10.1007/978-3-031-47665-5_22(265-278)Online publication date: 5-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-47665-5_22
Feng HZhou WDeng JWang YLi H(2022)Geometric Representation Learning for Document Image RectificationComputer Vision – ECCV 202210.1007/978-3-031-19836-6_27(475-492)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1007/978-3-031-19836-6_27

View Options

View options

Figures

Tables

Media

View Table of Conten