research-article

Feature learning and encoding for multi-script writer identification

Authors:

Abdelillah Semma,

Yaâcoub Hannad,

Mohamed El Youssfi El KettaniAuthors Info & Claims

International Journal on Document Analysis and Recognition (IJDAR), Volume 25, Issue 2

Pages 79 - 93

https://doi.org/10.1007/s10032-022-00394-8

Published: 01 June 2022 Publication History

Abstract

Writer identification from handwriting samples has been an interesting research problem for the pattern recognition community in general and handwriting recognition community in particular. In most cases, however, it is assumed that writers produce writing samples in a single script only. A more challenging scenario is the multi-script writer identification where the training and test samples of writers belong to different scripts. This paper presents a deep learning-based solution for writer identification in a multi-script scenario. The technique relies on identifying keypoints in handwriting and extracting small patches around these keypoints. These patches are aimed to capture the writing gestures of individuals which are likely to be common across multiple scripts. Robust feature representations are learned from these patches using a deep convolutional neural network and the features are encoded using a newly proposed variant of the Vector of Locally Aggregated Descriptors (VLAD). Experiments on three bilingual handwriting datasets including writing samples in Arabic, English, French, Chinese and Farsi report promising identification rates and significantly outperform the current state-of-the-art on this problem.

References

[1]

Abbas Faycel, Gattal Abdeljalil, Djeddi Chawki, Siddiqi Imran, Bensefia Ameur, and Saoudi Kamel Texture feature column scheme for single-and multi-script writer identification IET Biometr. 2021 10 2 179-193

[2]

Gattal Abdeljalil, Chawki Djeddi, Imran Siddiqi, and Somaya Al-Maadeed. Writer identification on historical documents using oriented basic image features. In 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 369–373. IEEE, 2018

[3]

Mohamed Nidhal Abdi and Maher Khemakhem A model-based approach to offline text-independent arabic writer identification and verification Pattern Recognit. 2015 48 5 1890-1903

[4]

Félix Abecassis. Opencv-morphological skeleton. Retrieved from Félix Abecassis Projects and Experiments: International Journal of Remote Sensinghttp://felix.abecassis.me/2011/09/opencv-morphological-skeleton/geological mapping at Cuprite Nevada:a rule-based system, 31:7, 2011

[5]

Somaya Al-Maadeed, Abdelaali Hassaine, Ahmed Bouridane, and Muhammad Atif Tahir. Novel geometric features for off-line writer identification. Pattern Analysis and Applications, 19(3):699–708, 2016

[6]

Bennour Akram, Djeddi Chawki, Gattal Abdeljalil, Siddiqi Imran, and Mekhaznia Tahar Handwriting based writer recognition using implicit shape codebook Forensic Sci. Int. 2019 301 91-100

[7]

Ameur Bensefia, Ali Nosary, Thierry Paquet, and Laurent Heutte. Writer identification by writer’s invariants. In: Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition, pages 274–279. IEEE, 2002

[8]

Bensefia Ameur, Paquet Thierry, and Heutte Laurent A writer identification and verification system Pattern Recogonit Lett. 2005 26 13 2080-2092

[9]

Bertolini Diego, Oliveira Luiz S, Justino E, and Sabourin Robert Texture-based descriptors for writer identification and verification Expert Syst. with Appl. 2013 40 6 2069-2080

[10]

Bulacu Marius and Schomaker Lambert Text-independent writer identification and verification using textural and allographic features Pattern Anal. Mach. Intell. IEEE Trans 2007 29 4 701-717

[11]

Djeddi Chawki and Souici-Meslati Labiba. A texture based approach for arabic writer identification and verification. In: 2010 International Conference on Machine and Web Intelligence, pages 115–120. IEEE, 2010

[12]

Vincent Christlein, David Bernecker, and Elli Angelopoulou. Writer identification using vlad encoded contour-zernike moments. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pages 906–910. IEEE, 2015

[13]

Vincent Christlein, David Bernecker, Andreas Maier, and Elli Angelopoulou. Offline writer identification using convolutional neural network activation features. In: German Conference on Pattern Recognition, pages 540–552. Springer, 2015

[14]

Vincent Christlein, Martin Gropp, Stefan Fiel, and Andreas Maier. Unsupervised feature learning for writer identification and writer retrieval. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 1, pages 991–997. IEEE, 2017

[15]

Vincent Christlein and Andreas Maier. Encoding cnn activations for writer recognition. In:D 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pages 169–174. IEEE, 2018

[16]

Jonathan Delhumeau, Philippe-Henri Gosselin, Hervé Jégou, and Patrick Pérez. Revisiting the vlad image representation. In: Proceedings of the 21st ACM international conference on Multimedia, pages 653–656, 2013

[17]

Chawki Djeddi, Somaya Al-Maadeed, Abdeljalil Gattal, Imran Siddiqi, Abdellatif Ennaji, and Haikal El Abed. Icfhr2016 competition on multi-script writer demographics classification using” quwi” database. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 602–606. IEEE, 2016

[18]

Chawki Djeddi, Somaya Al-Maadeed, Abdeljalil Gattal, Imran Siddiqi, Labiba Souici-Meslati, and Haikal El Abed. Icdar2015 competition on multi-script writer identification and gender classification using ‘quwi’database. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pages 1191–1195. IEEE, 2015

[19]

Chawki Djeddi, Somaya Al-Maadeed, Imran Siddiqi, Gattal Abdeljalil, Sheng He, and Younes Akbari. Icfhr 2018 competition on multi-script writer identification. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 506–510. IEEE, 2018

[20]

Chawki Djeddi, Abdeljalil Gattal, Labiba Souici-Meslati, Imran Siddiqi, Youcef Chibani, and Haikal El Abed. Lamis-mshd: a multi-script offline handwriting database. In: 2014 14th International Conference on Frontiers in Handwriting Recognition, pages 93–97. IEEE, 2014

[21]

Chawki Djeddi, Imran Siddiqi, Labiba Souici-Meslati, and Abdellatif Ennaji. Multi-script writer identification optimized with retrieval mechanism. In: 2012 International Conference on Frontiers in Handwriting Recognition, pages 509–514. IEEE, 2012

[22]

Djeddi Chawki, Siddiqi Imran, Souici-Meslati Labiba, and Ennaji Abdellatif Text-independent writer recognition using multi-script handwritten texts Pattern Recognit. Lett. 2013 34 10 1196-1202

[23]

bibitemfecker2014writer D Fecker, A Asit, Volker Märgner, Jihad El-Sana, and Tim Fingscheidt. Writer identification for historical arabic documents. In: 2014 22nd International Conference on Pattern Recognition, pages 3050–3055. IEEE, 2014

[24]

Stefan Fiel and Robert Sablatnig. Writer identification and retrieval using a convolutional neural network. In: International Conference on Computer Analysis of Images and Patterns, pages 26–37. Springer, 2015

[25]

Utpal Garain and Thierry Paquet. Off-line multi-script writer identification using ar coefficients. In: 2009 10th International Conference on Document Analysis and Recognition, pages 991–995. IEEE, 2009

[26]

Ghiasi Golnaz and Safabakhsh Reza Offline text-independent writer identification using codebook and efficient code extraction methods Image Vision Comput. 2013 31 5 379-391

[27]

Tara Gilliam, Richard C Wilson, and John A Clark. Scribe identification in medieval english manuscripts. In: 2010 20th International Conference on Pattern Recognition, pages 1880–1883. IEEE, 2010

[28]

Guo Zhenhua, Zhang Lei, and Zhang David A completed modeling of local binary pattern operator for texture classification IEEE Trans. Image Process. 2010 19 6 1657-1663

[29]

Yaâcoub Hannad, Imran Siddiqi, Chawki Djeddi, and Mohamed El-Youssfi El-Kettani. Improving arabic writer identification using score-level fusion of textural descriptors. IET Biometrics, 8(3):221–229, 2019

[30]

Hannad Yaacoub, Siddiqi Imran, El Youssfi Mohamed, and Kettani El Writer identification using texture descriptors of handwritten fragments Expert Syst. Appl. 2016 47 14-22

[31]

Christopher G Harris, Mike Stephens, et al. A combined corner and edge detector. In: Alvey vision conference, volume 15, pages 10–5244. Citeseer, 1988

[32]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

[33]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In: European conference on computer vision, pages 630–645. Springer, 2016

[34]

He Sheng, Wiering Marco, and Schomaker Lambert Junction detection in handwritten documents and its application to writer identification Pattern Recognit. 2015 48 12 4036-4048

[35]

Zhenyu He, Xinge You, and Yuan Yan Tang. Writer identification using global wavelet-based features. Neurocomputing, 71(10-2):1832–1841, 2008

[36]

Rajiv Jain and David Doermann. Offline writer identification using k-adjacent segments. In: 2011 International Conference on Document Analysis and Recognition, pages 769–773. IEEE, 2011

[37]

Hervé Jégou, Matthijs Douze, and Cordelia Schmid. On the burstiness of visual elements. In: 2009 IEEE conference on computer vision and pattern recognition, pages 1169–1176. IEEE, 2009

[38]

Jegou Herve, Perronnin Florent, Douze Matthijs, Sánchez Jorge, Perez Patrick, and Schmid Cordelia Aggregating local image descriptors into compact codes IEEE Trans. Pattern Anal. Mach. Intell. 2011 34 9 1704-1716

[39]

Tak-Eun Kim and Myoung Ho Kim Improving the search accuracy of the vlad through weighted aggregation of local descriptors J. Visual Comm. Image Represent. 2015 31 237-252

[40]

Neeraj Kumar, Li Zhang, and Shree Nayar. What is a good nearest neighbors algorithm for finding similar patches in images? In:D European conference on computer vision, pages 364–378. Springer, 2008

[41]

Lai Songxuan, Zhu Yecheng, and Jin Lianwen Encoding pathlet and sift features with bagged vlad for historical writer identification IEEE Trans. Inf. Forensics Secur. 2020 15 3553-3566

[42]

Georgios Louloudis, Basilis Gatos, and Nikolaos Stamatopoulos. Icfhr 2012 competition on writer identification challenge 1: Latin/greek documents. In: 2012 International Conference on Frontiers in Handwriting Recognition, pages 829–834. IEEE, 2012

[43]

Alieh Masomi, Hamid Reza Ghafari, Kazem Nouri, Younes Akbari, Walid Bouamra, and Chawki Djeddi. A new database for writer demographics attributes detection based on off-line persian and english handwriting. In: Proceedings of the Mediterranean Conference on Pattern Recognition and Artificial Intelligence, pages 125–130, 2016

[44]

Andrew J Newell and Lewis D Griffin. Writer identification using oriented basic image features and the delta encoding. Pattern Recognit., 47(6):2255–2265, 2014

[45]

Nguyen Hung Tuan, Nguyen Cuong Tuan, Ino Takeya, Indurkhya Bipin, and Nakagawa Masaki Text-independent writer identification using convolutional neural network Pattern Recognit. Lett. 2019 121 104-112

[46]

Stephen M Omohundro. Five balltree construction algorithms. International Computer Science Institute Berkeley, 1989

[47]

Florent Perronnin, Jorge Sánchez, and Thomas Mensink. Improving the fisher kernel for large-scale image classification. In: European conference on computer vision, pages 143–156. Springer, 2010

[48]

Arshia Rehman, Saeeda Naz, Muhammad Imran Razzak, and Ibrahim A Hameed. Automatic visual features for writer identification: A deep learning approach. IEEE access, 7:17149–17157, 2019

[49]

Huwida ES Said, Tienniu N Tan, and Keith D Baker. Personal identification based on handwriting. Pattern Recognition, 33(1):149–160, 2000

[50]

Lambert Schomaker. Advances in writer identification and verification. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), volume 2, pages 1268–1273. IEEE, 2007

[51]

Schomaker Lambert and Bulacu Marius Automatic writer identification using connected-component contours and edge-based features of uppercase western script IEEE Transactions on Pattern Analysis and Machine Intelligence 2004 26 6 787-798

[52]

Abdelillah Semma, Yaâcoub Hannad, and Mohamed El Youssfi El Kettani. Impact of the cnn patch size in the writer identification. In: Networking, Intelligent Systems and Security, pages 103–114. Springer, 2022

[53]

Semma, Abdelillah, Hannad, Yaâcoub., Siddiqi, Imran, Djeddi, Chawki, El Youssfi, Mohamed, Kettani, El (2021)Writer identification using deep learning with fast keypoints and harris corner detector. Expert Syst. Appl. 184, 115473

[54]

Semma Abdelillah, Lazrak Said, Hannad Yaâcoub, Boukhani Mohamed, and El Kettani Youssfi Writer identification: The effect of image resizing on cnn performance The Int. Archives . Photogramm. Remote Sens. Spatial Inf. Sci 2021 46 501-507

[55]

Sheng Biyun, Shen Chunhua, Lin Guosheng, Li Jun, Yang Wankou, and Sun Changyin Crowd counting via weighted vlad on a dense attribute feature map IEEE Trans. Circuits Syst. Video Techno. 2016 28 8 1788-1797

[56]

Imran Siddiqi and Nicole Vincent. Writer identification in handwritten documents. In: Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), volume 1, pages 108–112. IEEE, 2007

[57]

Siddiqi Imran and Vincent Nicole Text independent writer recognition using redundant writing patterns with contour-based orientation and curvature features Pattern Recognit. 2010 43 11 3853-3865

[58]

Sargur N Srihari, Sung-Hyuk Cha, Hina Arora, and Sangjik Lee. Individuality of handwriting. J. Forensic Sci., 47(4):856–872, 2002

[59]

Guo Xian Tan, Christian Viard-Gaudin, and Alex C Kot. Individuality of alphabet knowledge in online writer identification. In: International Journal on Document Analysis and Recognition (IJDAR), 13(2):147–157, 2010

[60]

Yanhong Wang, Yigang Cen, Liequan Liang, Linna Zhang, Viacheslav Voronin, and Vladimir Mladenovic. Fusion of deep features and weighted vlad vectors based on multiple features for image retrieval. In MATEC Web of Conferences, 2017

[61]

Xiangqian Wu, Tang Youbao, and Wei Bu Offline text-independent writer identification based on scale invariant feature transform IEEE Transactions on Information Forensics and Security 2014 9 3 526-536

[62]

Linjie Xing and Yu Qiao. Deepwriter: A multi-stream deep cnn for text-independent writer identification. I:n 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), pages 584–589. IEEE, 2016

[63]

Yu-Jie Xiong, Ying Wen, Patrick SP Wang, and Yue Lu. Text-independent writer identification using sift descriptor and contour-directional feature. In 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pages 91–95. IEEE, 2015

[64]

Yang Weixin, Jin Lianwen, and Liu Manfei Deepwriterid: an end-to-end online text-independent writer identification system IEEE Intell. Syst. 2016 31 2 45-53

[65]

Zhang Xu-Yao, Xie Guo-Sen, Liu Cheng-Lin, and Bengio Yoshua End-to-end online writer identification with recurrent neural network IEEE Trans. Human–Mach. Syst. 2016 47 2 285-292

[66]

Yong Zhu, Tieniu Tan, and Yunhong Wang. Biometric personal identification based on handwriting. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, volume 2, pages 797–800. IEEE, 2000

Cited By

Kumar VSundaram S(2024)Siamese-based offline word level writer identification in a reduced subspaceEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107720130:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107720
Patel RPatel NFataniya BShah D(2024)Automated Digitization of Student’s Marks from the Answer-Book Images Using a Lightweight CNN ModelSN Computer Science10.1007/s42979-024-02693-95:4Online publication date: 29-Mar-2024
https://dl.acm.org/doi/10.1007/s42979-024-02693-9
Pach J(2023)Database of Fragments of Medieval Codices of the 11th–12th Centuries – The Uniqueness of Requirements and DataComputational Science – ICCS 202310.1007/978-3-031-36027-5_8(104-112)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-36027-5_8

Index Terms

Feature learning and encoding for multi-script writer identification
1. Applied computing
  1. Document management and text processing
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Writer Identification using Deep Learning with FAST Keypoints and Harris corner detector
Highlights
- Writer identification from handwriting samples using machine-learned features.
- ...
Abstract
Writer identification from offline images of handwriting is an interesting pattern classification problem that has been investigated for many decades now. Despite significant research endeavors, the problem still remains challenging ...
WriterINet: a multi-path deep CNN for offline text-independent writer identification
Abstract
Handwriting-based identification is a fundamental pattern recognition problem that has attracted considerable interest in recent years. Writer identification is a major challenge, considering how diverse the written content is and how much ...
Text-independent writer recognition using multi-script handwritten texts

This paper presents a text-independent writer recognition method in a multi-script environment. Handwritten texts in Greek and English are considered in this study. The objective is to recognize the writer of a handwritten text in one script from the ...

Comments

Information & Contributors

Information

Published In

cover image International Journal on Document Analysis and Recognition

International Journal on Document Analysis and Recognition Volume 25, Issue 2

Jun 2022

92 pages

ISSN:1433-2833

EISSN:1433-2825

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2022.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 June 2022

Accepted: 16 January 2022

Revision received: 13 January 2022

Received: 06 October 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kumar VSundaram S(2024)Siamese-based offline word level writer identification in a reduced subspaceEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107720130:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107720
Patel RPatel NFataniya BShah D(2024)Automated Digitization of Student’s Marks from the Answer-Book Images Using a Lightweight CNN ModelSN Computer Science10.1007/s42979-024-02693-95:4Online publication date: 29-Mar-2024
https://dl.acm.org/doi/10.1007/s42979-024-02693-9
Pach J(2023)Database of Fragments of Medieval Codices of the 11th–12th Centuries – The Uniqueness of Requirements and DataComputational Science – ICCS 202310.1007/978-3-031-36027-5_8(104-112)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-36027-5_8

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents