research-article

ASSV: Handwritten Signature Verification Using Acoustic Signals

Authors:

Run ZhaoAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 3, Issue 3

Article No.: 80, Pages 1 - 22

https://doi.org/10.1145/3351238

Published: 09 September 2019 Publication History

Abstract

As one kind of biological characteristics of people, handwritten signature has been widely used in the banking industry, government and education. Verifying handwritten signatures manually causes too much human cost, and its high probability of errors can threaten the property safety and even society stability. Therefore, the need for an automatic verification system is emphasized. This paper proposes a device-free on-line handwritten signature verification system ASSV, providing paper-based handwritten signature verification service. As far as we know, ASSV is the first system which uses the changes of acoustic signals to realize signature verification. ASSV differs from previous on-line signature verification work in two aspects: 1. It requires neither a special sensor-instrumented pen nor a tablet; 2. People do not need to wear a device such as a smartwatch on the dominant hand for hand tracking. Differing from previous acoustic-based sensing systems, ASSV uses a novel chord-based method to estimate phase-related changes caused by tiny actions. Then based on the estimation, frequency-domain features are extracted by a discrete cosine transform (DCT). Moreover, a deep convolutional neural network (CNN) model fed with distance matrices is designed to verify signatures. Extensive experiments show that ASSV is a robust, efficient and secure system achieving an AUC of 98.7% and an EER of 5.5% with a low latency.

Supplementary Material

ding (ding.zip)

Supplemental movie, appendix, image and software files for, ASSV: Handwritten Signature Verification Using Acoustic Signals

Download
35.14 MB

References

[1]

2013. 2013 ABA Deposit Account Fraud Survey. Retrieved Jan. 4, 2019 from https://www.aba.com/Products/Surveys/Pages/2013DepositAccount.aspx

[2]

2015. 2015 AFP Payments Fraud and Control Survey. Retrieved Jan. 4, 2019 from http://wwtug.org/instmem.html

[3]

Md Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, and Shwetak Patel. 2013. DopLink: using the doppler effect for multi-device interaction. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, 583--586.

Digital Library

[4]

Indrajit Bhattacharya, Prabir Ghosh, and Swarup Biswas. 2013. Offline signature verification using pixel matching technique. Procedia Technology 10 (2013), 970--977.

[5]

Horst Bunke, Markus Roth, and Ernst Günter Schukat-Talamazzini. 1995. Off-line cursive handwriting recognition using hidden Markov models. Pattern recognition 28, 9 (1995), 1399--1413.

[6]

Subhash Chandra and Sushila Maheskar. 2016. Offline signature verification based on geometric feature extraction using artificial neural network. In Recent Advances in Information Technology (RAIT), 2016 3rd International Conference on. IEEE, 410--414.

[7]

Robert B Cleveland, William S Cleveland, Jean E McRae, and Irma Terpenning. 1990. STL: A Seasonal-Trend Decomposition. Journal of Official Statistics 6, 1 (1990), 3--73.

[8]

Marcos Faundez-Zanuy. 2007. On-line signature recognition based on VQ-DTW. Pattern Recognition 40, 3 (2007), 981--992.

Digital Library

[9]

Julian Fierrez, Javier Ortega-Garcia, Daniel Ramos, and Joaquin Gonzalez-Rodriguez. 2007. HMM-based on-line signature verification: Feature extraction and signature modeling. Pattern recognition letters 28, 16 (2007), 2325--2334.

Digital Library

[10]

Andreas Fischer, Moises Diaz, Rejean Plamondon, and Miguel A Ferrer. 2015. Robust score normalization for DTW-based on-line signature verification. In Document Analysis and Recognition (ICDAR), 2015 13th International Conference on. IEEE, 241--245.

Digital Library

[11]

Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning. Vol. 1. MIT press Cambridge.

Digital Library

[12]

Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1911--1914.

Digital Library

[13]

Luiz G Hafemann, Robert Sabourin, and Luiz S Oliveira. 2017. Learning features for offline handwritten signature verification using deep convolutional neural networks. Pattern Recognition 70 (2017), 163--176.

Digital Library

[14]

Luiz G Hafemann, Robert Sabourin, and Luiz S Oliveira. 2017. Offline handwritten signature verificationâĂŤliterature review. In Image Processing Theory, Tools and Applications (IPTA), 2017 Seventh International Conference on. IEEE, 1--8.

[15]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780.

Digital Library

[16]

Donato Impedovo and Giuseppe Pirlo. 2008. Automatic signature verification: The state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38, 5 (2008), 609--635.

Digital Library

[17]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).

[18]

Anil K Jain, Arun Ross, and Salil Prabhakar. 2004. An introduction to biometric recognition. IEEE Transactions on circuits and systems for video technology 14, 1 (2004), 4--20.

Digital Library

[19]

Ramanujan S Kashi, Jianying Hu, Winston L Nelson, and William Turin. 1997. On-line handwritten signature verification using hidden Markov model features. In icdar. IEEE, 253.

Digital Library

[20]

Alisher Kholmatov and Berrin Yanikoglu. 2005. Identity authentication using improved online signature verification method. Pattern recognition letters 26, 15 (2005), 2400--2408.

Digital Library

[21]

Franck Leclerc and Rejean Plamondon. 1994. Automatic signature verification: The state of the artâĂŤ1989-1993. International journal of pattern recognition and artificial intelligence 8, 03 (1994), 643--660.

[22]

Alona Levy, Ben Nassi, Yuval Elovici, and Erez Shmueli. 2018. Handwritten Signature Verification Using Wrist-Worn Devices. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 119.

Digital Library

[23]

Kang Ling, Haipeng Dai, Yuntang Liu, and Alex X Liu. 2018. UltraGesture: Fine-Grained Gesture Sensing and Recognition. In 2018 15th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). IEEE, 1--9.

[24]

Li Lu, Jiadi Yu, Yingying Chen, Hongbo Liu, Yanmin Zhu, Yunfei Liu, and Minglu Li. 2018. LipPass: Lip Reading-based User Authentication on Smartphones Leveraging Acoustic Signals. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 1466--1474.

[25]

Wenguang Mao, Jian He, and Lili Qiu. 2016. CAT: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 69--81.

Digital Library

[26]

Mario E. Munich and Pietro Perona. 2003. Visual identification by signature tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 2 (2003), 200--217.

Digital Library

[27]

Vishvjit S Nalwa. 1997. Automatic on-line signature verification. Proc. IEEE 85, 2 (1997), 215--239.

[28]

Srikanta Pal, Michael Blumenstein, and Umapada Pal. 2011. Off-line signature verification systems: a survey. In Proceedings of the International Conference & Workshop on Emerging Trends in Technology. ACM, 652--657.

Digital Library

[29]

Rejean Plamondon and Guy Lorette. 1989. Automatic signature verification and writer identificationâĂŤthe state of the art. Pattern recognition 22, 2 (1989), 107--131.

[30]

Réjean Plamondon and Sargur N Srihari. 2000. Online and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on pattern analysis and machine intelligence 22, 1 (2000), 63--84.

Digital Library

[31]

Lawrence R Rabiner and Biing-Hwang Juang. 1986. An introduction to hidden Markov models. ieee assp magazine 3, 1 (1986), 4--16.

[32]

A Rodríguez Valiente, A Trinidad, JR García Berrocal, C Górriz, and R Ramírez Camacho. 2014. Extended high-frequency (9-20 kHz) audiometry reference thresholds in 645 healthy subjects. International journal of audiology 53, 8 (2014), 531--545.

[33]

Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 474--485.

Digital Library

[34]

Robert Sabourin, Réjean Plamondon, and Guy Lorette. 1992. Off-line identification with handwritten signature images: Survey and perspectives. In Structured Document Image Analysis. Springer, 219--234.

[35]

MC Schoeman-Malan. 2015. Fraud and forgery of the testator's will or signature: the flight from formalities to no formalities. JS Afr. L. (2015), 125.

[36]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929--1958.

Digital Library

[37]

Harry Thornburg. 2016. Real-Time Decomposition of Time Series. Retrieved Jan. 1, 2019 from https://developer.ibm.com/streamsdev/docs/real-time-decomposition-of-time-series/

[38]

David Tse and Pramod Viswanath. 2005. Fundamentals of wireless communication. Cambridge university press.

Digital Library

[39]

Tianben Wang, Daqing Zhang, Leye Wang, Yuanqing Zheng, Tao Gu, Bernadette Dorizzi, and Xingshe Zhou. 2018. Contactless Respiration Monitoring using Ultrasound Signal with Off-the-shelf Audio Devices. IEEE Internet of Things Journal (2018).

[40]

Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, Xingshe Zhou, and Bernadette Dorizzi. 2018. C-FMCW based contactless respiration detection using acoustic signal. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 170.

Digital Library

[41]

Wei Wang, Alex X Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 82--94.

Digital Library

[42]

Teng Wei and Xinyu Zhang. 2015. mtrack: High-precision passive tracking using millimeter wave radios. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 117--129.

Digital Library

[43]

Mei Wang Wenguang Mao and Lili Qiu. 2018. AIM: Acoustic Imaging on a Mobile. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM.

Digital Library

[44]

Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a mobile device into a mouse in the air. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 15--29.

Digital Library

[45]

Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 15--28.

Digital Library

[46]

Jie Zou and Zengfu Wang. 2013. Application of HMM to online signature verification based on segment differences. In Biometric Recognition. Springer, 425--432.

Cited By

Han MYang HNi TDuan DRuan MChen YZhang JXu W(2024)mmSign: mmWave-based Few-Shot Online Handwritten Signature VerificationACM Transactions on Sensor Networks10.1145/360594520:4(1-31)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3605945
Bi HZhang WLi SChen YZhou CZhou T(2024)SmartSit: Sitting Posture Recognition Through Acoustic Sensing on SmartphonesIEEE Transactions on Multimedia10.1109/TMM.2024.337576126(8119-8130)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3375761
Akbar ASheng ZZhang QWang D(2023)Cross-Domain Gesture Sequence Recognition for Two-Player Exergames using COTS mmWave RadarProceedings of the ACM on Human-Computer Interaction10.1145/36264777:ISS(327-356)Online publication date: Nov-2023
https://doi.org/10.1145/3626477
Show More Cited By

Index Terms

ASSV: Handwritten Signature Verification Using Acoustic Signals
1. Human-centered computing
  1. Ubiquitous and mobile computing
2. Security and privacy
  1. Human and societal aspects of security and privacy

Recommendations

ASSV: handwritten signature verification using acoustic signals
UbiComp/ISWC '19 Adjunct: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers

As one kind of biological characteristics of people, handwritten signature has been widely used in the banking industry, government and education. Verifying handwritten signatures manually causes too much human cost, and its high probability of errors ...
Recognition of Nastalique Urdu ligatures
MOCR '13: Proceedings of the 4th International Workshop on Multilingual OCR

There has been considerable work on Arabic OCR. However, all that work is based on Naskh style. Urdu script is based on Arabic alphabet, but uses Nastalique style. The Nastalique style makes OCR in general and character segmentation in particular, a ...
Performance analysis of chinese cursive character recognition based on convolutional neural network
RACS '19: Proceedings of the Conference on Research in Adaptive and Convergent Systems

Chinese cursive characters written in old books are more difficult to recognize than other Chinese Characters such as handwritten Chinese character because they have many various styles. For this reason, it needs a software-based recognition model or ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 3, Issue 3

September 2019

1415 pages

EISSN:2474-9567

DOI:10.1145/3361560

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Published in IMWUT Volume 3, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
565
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)4

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Han MYang HNi TDuan DRuan MChen YZhang JXu W(2024)mmSign: mmWave-based Few-Shot Online Handwritten Signature VerificationACM Transactions on Sensor Networks10.1145/360594520:4(1-31)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3605945
Bi HZhang WLi SChen YZhou CZhou T(2024)SmartSit: Sitting Posture Recognition Through Acoustic Sensing on SmartphonesIEEE Transactions on Multimedia10.1109/TMM.2024.337576126(8119-8130)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3375761
Akbar ASheng ZZhang QWang D(2023)Cross-Domain Gesture Sequence Recognition for Two-Player Exergames using COTS mmWave RadarProceedings of the ACM on Human-Computer Interaction10.1145/36264777:ISS(327-356)Online publication date: Nov-2023
https://doi.org/10.1145/3626477
Li WHe TJing NWang L(2023)mmHSV: In-Air Handwritten Signature Verification via Millimeter-Wave RadarACM Transactions on Internet of Things10.1145/36144434:4(1-22)Online publication date: 22-Nov-2023
https://dl.acm.org/doi/10.1145/3614443
Liu WDu HLi BLi JChang ZWang L(2023) RFPad: Enabling Device-Free Handwriting Recognition With a Tag Square IEEE Transactions on Human-Machine Systems10.1109/THMS.2023.323660553:2(325-334)Online publication date: Apr-2023
https://doi.org/10.1109/THMS.2023.3236605
Wu YBi HFan JXu GChen H(2023)DMHC: Device-free multi-modal handwritten character recognition system with acoustic signalKnowledge-Based Systems10.1016/j.knosys.2023.110314264(110314)Online publication date: Mar-2023
https://doi.org/10.1016/j.knosys.2023.110314
Huang JWang YZou YWu KNi L(2023)Ubiquitous WiFi and Acoustic Sensing: Principles, Technologies, and ApplicationsJournal of Computer Science and Technology10.1007/s11390-023-3073-538:1(25-63)Online publication date: 31-Jan-2023
https://doi.org/10.1007/s11390-023-3073-5
Chen MLin JZou YWu K(2022)Acoustic Sensing Based on Online Handwritten Signature VerificationSensors10.3390/s2223934322:23(9343)Online publication date: 30-Nov-2022
https://doi.org/10.3390/s22239343
Schrapel MGrannemann DRohs M(2022)Sign H3re: Symbol and X-Mark Writer Identification Using Audio and Motion Data from a Digital PenProceedings of Mensch und Computer 202210.1145/3543758.3543764(209-218)Online publication date: 4-Sep-2022
https://dl.acm.org/doi/10.1145/3543758.3543764
Sadak MKahraman NUludağ U(2022)Dynamic and static feature fusion for increased accuracy in signature verificationSignal Processing: Image Communication10.1016/j.image.2022.116823108(116823)Online publication date: Oct-2022
https://doi.org/10.1016/j.image.2022.116823
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents