Improved Learning for Online Handwritten Chinese Text Recognition with Convolutional Prototype Network
Authors: 
Yi Chen, Heng Zhang, Cheng-Lin LiuAuthors Info & Claims
Yi Chen, Heng Zhang, Cheng-Lin LiuAuthors Info & ClaimsPages 38 - 53
Abstract
Segmentation-based handwritten text recognition has the advantage of character interpretability but needs a character classifier with high classification accuracy and non-character rejection capability. The classifier can be trained on both character samples and string samples but real string samples are usually insufficient. In this paper, we proposed a learning method for segmentation-based online handwritten Chinese text recognition with a convolutional prototype network as the underlying classifier. The prototype classifier is inherently resistant to non-characters, and so, can be trained with character and string samples without the need of data augmentation. The learning has two stages: pre-training on character samples with a modified loss function for improving non-character resistance, and weakly supervised learning on both character and string samples for improving recognition performance. Experimental results on the CASIA-OLHWDB and ICDAR2013-Online datasets show that the proposed method can achieve promising recognition performance without training data augmentation.
References
[1]
Liu C-L, Jäger S, and Nakagawa M Online recognition of Chinese characters: the state-of-the-art IEEE Trans. Pattern Anal. Mach. Intell. 2004 26 2 198-213
[2]
Liu C-L and Nakagawa M Evaluation of prototype learning algorithms for nearest-neighbor classifier in application to handwritten character recognition Pattern Recognit. 2001 34 3 601-615
[3]
Wang D-H, Liu C-L, and Zhou X-D An approach for real-time recognition of online Chinese handwritten sentences Pattern Recognit. 2012 45 10 3661-3675
[4]
Zhou X-D, Wang D-H, Tian F, Liu C-L, and Nakagawa M Handwritten Chinese/Japanese text recognition using semi-Markov conditional random fields IEEE Trans. Pattern Anal. Mach. Intell. 2013 35 10 2413-2426
[5]
Wu, Y.-C., Fei, Y., Liu, C.-L.: Improving handwritten Chinese text recognition using neural network language models and convolutional neural network shape models. Pattern Recognit. 65, 251–264 (2017)
[6]
Wang, Z.-X., Wang, Q.-F., Yin, F., Liu, C.-L.: Weakly supervised learning for over-segmentation based handwritten Chinese text recognition. In: ICFHR 2020, pp. 157–162 (2020)
[7]
Su, T.-H., Zhang, T., Guan, D.-J., Huang, H.-J.: Off-line recognition of realistic Chinese handwriting using segmentation-free strategy. Pattern Recognit. 42(1), 167–182 (2009)
[8]
Jiang, Z.-W., Ding, X.-Q., Liu, C., Wang, Y.-W.: A novel short merged off-line handwritten Chinese character string segmentation algorithm using hidden Markov model. In: ICDAR 2011, pp. 668–672 (2011)
[9]
Jayech K, Mahjoub MA, and Amara NEB Synchronous multi-stream hidden Markov model for offline Arabic handwriting recognition without explicit segmentation Neurocomputing 2016 214 958-971
[10]
Peng, D.-Z., et al.: Recognition of handwritten Chinese text by segmentation: a segment-annotation-free approach. IEEE Trans. Multimed. (2022)
[11]
Wang Z-R, Jun D, and Wang J-M Writer-aware CNN for parsimonious HMM-based offline handwritten Chinese text recognition Pattern Recognit. 2020 100
[12]
Liu C-L, Yin F, Wang D-H, and Wang Q-F Online and offline handwritten Chinese character recognition: benchmarking on new databases Pattern Recognit. 2013 46 1 155-162
[13]
Yang H-M, Zhang X-Y, Yin F, Yang Q, and Liu C-L Convolutional prototype network for open set recognition IEEE Trans. Pattern Anal. Mach. Intell. 2022 44 5 2358-2370
[14]
Liu C-L, Sako H, and Fujisawa H Effects of classifier structures and training regimes on integrated segmentation and recognition of handwritten numeral strings IEEE Trans. Pattern Anal. Mach. Intell. 2004 26 11 1395-1407
[15]
Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: CVPR 2012, pp. 3642–3649
[16]
Zhang X-Y, Bengio Y, and Liu C-L Online and offline handwritten Chinese character recognition: a comprehensive study and new benchmark Pattern Recognit. 2017 61 348-360
[17]
Liu C-L Normalization-cooperated gradient feature extraction for handwritten character recognition IEEE Trans. Pattern Anal. Mach. Intell. 2007 29 8 1465-1469
[18]
Liu, X., Hu, B.-T., Chen, Q.-C., Xiang-Ping, W., You, J.-H.: Stroke sequence-dependent deep convolutional neural network for online handwritten chinese character recognition. IEEE Trans. Neural Networks Learn. Syst. 31(11), 4637–4648 (2020)
[19]
Ren H-Q, Wang W-Q, Xi-Wen Q, and Cai Y-Q A new hybrid-parameter recurrent neural network for online handwritten Chinese character recognition Pattern Recognit. Lett. 2019 128 400-406
[20]
Xie Z-C, Sun Z-H, Jin L-W, Ni H, and Lyons TJ Learning spatial-semantic context with fully convolutional recurrent network for online handwritten Chinese text recognition IEEE Trans. Pattern Anal. Mach. Intell. 2018 40 8 1903-1917
[21]
Chen, K., et al.: A compact CNN-DBLSTM based character model for online handwritten Chinese text recognition. In: ICDAR 2017, pp. 1068–1073
[22]
Qin, X.-H., Zhang, H.-Y., Ke, X., Shen, Z.-H., Qi, S.-M., Liu, K.: Progressive multitask learning network for online Chinese signature segmentation and recognition. In: ICFHR 2022, pp. 153–167
[23]
Lee S-W and Song H-H Optimal design of reference models for large-set handwritten character recognition Pattern Recognit. 1994 27 9 1267-1274
[24]
Niu, S.-C., et al.: Towards stable test-time adaptation in dynamic wild world. ICLR (2023)
[25]
Liu, C.-L., Kim, I.-J., Kim, J.H.: High accuracy handwritten Chinese character recognition by improved feature matching method. In: ICDAR 1997, pp. 1033–1037
[26]
Raghavendra, B.S., Narayanan, C.K., Sita, G., Ramakrishnan, A.G., Sriganesh, M.: Prototype learning methods for online handwriting recognition. In: ICDAR 2005, pp. 287–291
[27]
Liu, C.-L.: One-vs-all training of prototype classifier for pattern classification and retrieval. In: ICPR 2010, pp. 3328–3331
[28]
Impedovo S, Mangini FM, and Barbuzzi D A novel prototype generation technique for handwriting digit recognition Pattern Recognit. 2014 47 3 1002-1010
[29]
Ao X, Zhang X-Y, and Liu C-L Cross-modal prototype learning for zero-shot handwritten character recognition Pattern Recognit. 2022 131
[30]
Yang, H.-M., Zhang, X.-Y., Yin, F., Liu, C.-L.: Robust classification with convolutional prototype learning. In: CVPR 2018, pp. 3474–3482
[31]
Gao, L.-K., Zhang, H., Liu, C-L.: Handwritten text recognition with convolutional prototype network and most aligned frame based CTC training. ICDAR (1), pp. 205–220 (2021)
[32]
Yu, M.-M., Zhang, H., Yin, F., Liu, C.-L.: An efficient prototype-based model for handwritten text recognition with multi-loss fusion. In: ICFHR 2022, pp. 404–418
[33]
Heng Zhang, Cheng-Lin Liu: A Lattice-Based Method for Keyword Spotting in Online Chinese Handwriting. ICDAR 2011: 1064–1068
[34]
Liu C-L and Nakagawa M Precise Candidate Selection for Large Character Set Recognition by Confidence Evaluation IEEE Trans. Pattern Anal. Mach. Intell. 2000 22 6 636-642
[35]
Jeffrey A. Barnett: Computational Methods for A Mathematical Theory of Evidence. Classic Works of the Dempster-Shafer Theory of Belief Functions 2008: 197–216
[36]
Liu, C.-L., Yin, F., Wang, D.-H., Wang, Q.-F.: CASIA online and offline Chinese handwriting databases. In: ICDAR 2011, pp. 37–41 (2011)
[37]
Yin, F., Wang, Q.-F., Zhang, X.-Y., Liu, C.-L.: ICDAR 2013 Chinese handwriting recognition competition. In: ICDAR 2013, pp. 1464–1470
[38]
Shi B-G, Bai X, and Yao C An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition IEEE Trans. Pattern Anal. Mach. Intell. 2017 39 11 2298-2304
[39]
Zhou X-D, Zhang Y-M, Tian F, Wang H-A, and Liu C-L Minimum-risk training for semi-Markov conditional random fields with application to handwritten Chinese/Japanese text recognition Pattern Recognit. 2014 47 5 1904-1916
[40]
Peng, D.-Z., Jin, L.-W., Wu, Y.-Q., Wang, Z.-P., Cai, M.-X.: A fast and accurate fully convolutional network for end-to-end handwritten Chinese text segmentation and recognition. In: ICDAR 2019, pp. 25–30
Recommendations
Radical aggregation network for few-shot offline handwritten Chinese character recognition
Highlights- A radical aggregation network for radical-level Chinese character recognition.
- ...
Graphical abstractOverall architecture of our radical aggregation network (RAN). The RAN consists of three components, a radical mapping encoder (RME), a radical aggregation module (RAM), and a character analysis decoder (CAD). The RME module ...
AbstractOffline handwritten Chinese character recognition has attracted much interest due to its various applications. The most cutting-edge methods treat Chinese character as a whole, ignoring the structures and radicals that compose ...
Handwritten Chinese text editing and recognition system
This paper describes a handwritten Chinese text editing and recognition system that can edit handwritten text and recognize it with a client-server mode. First, the client end samples and redisplays the handwritten text by using digital ink technics, ...
Online and offline handwritten Chinese character recognition: Benchmarking on new databases
Recently, the Institute of Automation of Chinese Academy of Sciences (CASIA) released the unconstrained online and offline Chinese handwriting databases CASIA-OLHWDB and CASIA-HWDB, which contain isolated character samples and handwritten texts produced ...
Comments
Information & Contributors
Information
Published In
Aug 2023
482 pages
ISBN:978-3-031-41684-2
DOI:10.1007/978-3-031-41685-9
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 21 August 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 27 Jul 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in