research-article

Effective 2D Stroke-based Gesture Augmentation for RNNs

Authors:

Mykola Maslych,

Eugene Matthew Taranta,

Mostafa Aldilati,

Joseph J. LaviolaAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 282, Pages 1 - 13

https://doi.org/10.1145/3544548.3581358

Published: 19 April 2023 Publication History

Abstract

Recurrent neural networks (RNN) require large training datasets from which they learn new class models. This limitation prohibits their use in custom gesture applications where only one or two end user samples are given per gesture class. One common way to enhance sparse datasets is to use data augmentation to synthesize new samples. Although there are numerous known techniques, they are often treated as standalone approaches when in reality they are often complementary. We show that by intelligently chaining augmentation techniques together that simulate different gesture production variability types, such as those affecting the temporal and spatial qualities of a gesture, we can significantly increase RNN accuracy without sacrificing training time. Through experimentation on four public stroke-based 2D gesture datasets, we show that RNNs trained with our data augmentation chaining technique achieves state-of-the-art recognition accuracy in both writer-dependent and writer-independent test scenarios.

Supplementary Material

Supplemental Materials (3544548.3581358-supplemental-materials.zip)

Download
245.43 KB

MP4 File (3544548.3581358-video-preview.mp4)

Video Preview

Download
3.13 MB

MP4 File (3544548.3581358-talk-video.mp4)

Pre-recorded Video Presentation

Download
97.57 MB

References

[1]

Lisa Anthony and Jacob O. Wobbrock. 2010. A lightweight multistroke recognizer for user interface prototypes. In Proceedings of Graphics Interface 2010(GI ’10). Canadian Information Processing Society, CAN, 245–252.

Digital Library

[2]

Lisa Anthony and Jacob O Wobbrock. 2012. $ n-protractor: A fast and accurate multistroke recognizer. In Proceedings of Graphics Interface 2012(GI ’12). Graphics Interface Conference 2012, Toronto, Ontario, Canada, 117–120.

[3]

Ahmed Sabbir Arif and Wolfgang Stuerzlinger. 2014. User adaptation to a faulty unistroke-based text entry technique by switching to an alternative gesture set. In Proceedings of Graphics Interface 2014(GI ’14). Canadian Information Processing Society, CAN, 183–192.

Digital Library

[4]

Ujjwal Bhattacharya, Réjean Plamondon, Souvik Dutta Chowdhury, Pankaj Goyal, and Swapan K. Parui. 2017. A Sigma-Lognormal Model-Based Approach to Generating Large Synthetic Online Handwriting Sample Databases. International Journal on Document Analysis and Recognition (IJDAR) 20, 3 (Sept. 2017), 155–171. https://doi.org/10.1007/s10032-017-0287-5

Digital Library

[5]

Rachel Blagojevic, Samuel Hsiao-Heng Chang, and Beryl Plimmer. 2010. The Power of Automatic Feature Selection: Rubine on Steroids.SBIM 10(2010), 79–86.

[6]

Ariel Caputo, Andrea Giachetti, Simone Soso, Deborah Pintani, Andrea D’Eusanio, Stefano Pini, Guido Borghi, Alessandro Simoni, Roberto Vezzani, Rita Cucchiara, Andrea Ranieri, Franca Giannini, Katia Lupinetti, Marina Monti, Mehran Maghoumi, Joseph J. LaViola Jr, Minh-Quan Le, Hai-Dang Nguyen, and Minh-Triet Tran. 2021. SHREC 2021: Skeleton-based Hand Gesture Recognition in the Wild. Computers & Graphics 99 (Oct. 2021), 201–211. https://doi.org/10.1016/j.cag.2021.07.007

Digital Library

[7]

F. M. Caputo, S. Burato, G. Pavan, T. Voillemin, H. Wannous, J. P. Vandeborre, M. Maghoumi, E. M. Taranta II, A. Razmjoo, J. J. LaViola Jr., F. Manganaro, S. Pini, G. Borghi, R. Vezzani, R. Cucchiara, H. Nguyen, M. T. Tran, and A. Giachetti. 2019. Online Gesture Recognition. In Eurographics Workshop on 3D Object Retrieval. The Eurographics Association, Department of Computer Science, University of Verona, Italy, 10 pages. https://doi.org/10.2312/3dor.20191067

[8]

Yineng Chen, Xiaojun Su, Feng Tian, Jin Huang, Xiaolong (Luke) Zhang, Guozhong Dai, and Hongan Wang. 2016. Pactolus: A Method for Mid-Air Gesture Segmentation within EMG. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’16). Association for Computing Machinery, New York, NY, USA, 1760–1765. https://doi.org/10.1145/2851581.2892492

Digital Library

[9]

Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. CoRR abs/1406.1078(2014), 15 pages. arXiv:1406.1078http://arxiv.org/abs/1406.1078

[10]

Sung-Jung Cho, Eunseok Choi, Won-Chul Bang, Jing Yang, Junil Sohn, Dong Yoon Kim, Young-Bum Lee, and Sangryong Kim. 2006. Two-stage Recognition of Raw Acceleration Signals for 3-D Gesture-Understanding Cell Phones. In Tenth International Workshop on Frontiers in Handwriting Recognition, Guy Lorette (Ed.). Université de Rennes 1, Suvisoft, La Baule (France). https://hal.inria.fr/inria-00103854http://www.suvisoft.com.

[11]

Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le. 2019. RandAugment: Practical automated data augmentation with a reduced search space. https://doi.org/10.48550/ARXIV.1909.13719

[12]

Kenny Davila, Stephanie Ludi, and Richard Zanibbi. 2014. Using Off-Line Features and Synthetic Data for On-Line Handwritten Math Symbol Recognition. In 2014 14th International Conference on Frontiers in Handwriting Recognition. IEEE, Hersonissos, Greece, 323–328. https://doi.org/10.1109/ICFHR.2014.61

[13]

Yousef Elarian, Radwan Abdel-Aal, Irfan Ahmad, Mohammad Tanvir Parvez, and Abdelmalek Zidouri. 2014. Handwriting Synthesis: Classifications and Techniques. International Journal on Document Analysis and Recognition (IJDAR) 17, 4 (Dec. 2014), 455–469. https://doi.org/10.1007/s10032-014-0231-x

Digital Library

[14]

Raul Fernandez, Andrew Rosenberg, Alexander Sorin, Bhuvana Ramabhadran, and Ron Hoory. 2017. Voice-Transformation-Based Data Augmentation for Prosodic Classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 5530–5534. https://doi.org/10.1109/ICASSP.2017.7953214

Digital Library

[15]

Germain Forestier, François Petitjean, Hoang Anh Dau, Geoffrey I. Webb, and Eamonn Keogh. 2017. Generating Synthetic Time Series to Augment Sparse Datasets. In 2017 IEEE International Conference on Data Mining (ICDM). 865–870. https://doi.org/10.1109/ICDM.2017.106 ISSN: 2374-8486.

[16]

Donatien Grolaux, Jean Vanderdonckt, Thanh-Diane Nguyen, and Iyad Khaddam. 2020. SketchADoodle: Touch-surface Multi-stroke Gesture Handling by Bézier Curves. Proceedings of the ACM on Human-Computer Interaction 4, EICS (June 2020), 1–30. https://doi.org/10.1145/3397875

Digital Library

[17]

T.M. Ha and H. Bunke. 1997. Off-Line, Handwritten Numeral Recognition by Perturbation Method. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 5 (May 1997), 535–539. https://doi.org/10.1109/34.589216

Digital Library

[18]

Shangchen Han, Beibei Liu, Randi Cabezas, Christopher D. Twigg, Peizhao Zhang, Jeff Petkau, Tsz-Ho Yu, Chun-Jung Tai, Muzaffer Akbay, Zheng Wang, Asaf Nitzan, Gang Dong, Yuting Ye, Lingling Tao, Chengde Wan, and Robert Wang. 2020. MEgATrack: Monochrome Egocentric Articulated Hand-Tracking for Virtual Reality. ACM Transactions on Graphics 39, 4 (July 2020), 87:87:1–87:87:13. https://doi.org/10.1145/3386569.3392452

Digital Library

[19]

Taihei Hayashi, Keiji Gyohten, Hidehiro Ohki, and Toshiya Takami. 2018. A Study of Data Augmentation for Handwritten Character Recognition Using Deep Learning. In 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). 552–557. https://doi.org/10.1109/ICFHR-2018.2018.00102

[20]

Samitha Herath, Mehrtash Harandi, and Fatih Porikli. 2017. Going deeper into action recognition: A survey. Image and Vision Computing 60 (April 2017), 4–21. https://doi.org/10.1016/j.imavis.2017.01.010

Digital Library

[21]

Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580(2012), 18 pages. arXiv:1207.0580http://arxiv.org/abs/1207.0580

[22]

Alexander Hoelzemann, Nimish Sorathiya, and Kristof Van Laerhoven. 2021. Data Augmentation Strategies for Human Activity Data Using Generative Adversarial Neural Networks. 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) (2021). https://doi.org/10.1109/PerComWorkshops51409.2021.9431046

[23]

Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, In ICML’15: Proceedings of the 32nd International Conference on International Conference on Machine Learning. CoRR 37, 448–456. arXiv:1502.03167http://arxiv.org/abs/1502.03167

[24]

Brian Kenji Iwana and Seiichi Uchida. 2021. An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks. PLoS ONE 16, 7 (July 2021), e0254841. https://doi.org/10.1371/journal.pone.0254841

[25]

Maria Karam and m. c. schraefel. 2006. Investigating user tolerance for errors in vision-enabled gesture-based interactions. In Proceedings of the working conference on Advanced visual interfaces(AVI ’06). Association for Computing Machinery, New York, NY, USA, 225–232. https://doi.org/10.1145/1133265.1133309

Digital Library

[26]

Jungsoo Kim, Jiasheng He, Kent Lyons, and Thad Starner. 2007. The Gesture Watch: A Wireless Contact-free Gesture based Wrist Interface. In 2007 11th IEEE International Symposium on Wearable Computers. IEEE, Boston, MA, USA, 15–22. https://doi.org/10.1109/ISWC.2007.4373770 ISSN: 2376-8541.

Digital Library

[27]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization, In 3rd International Conference on Learning Representations, ICLR, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. arXiv preprint arXiv:1412.6980 3, 1, 15 pages. http://arxiv.org/abs/1412.6980

[28]

Emanuele Ledda and Lucio Davide Spano. 2021. Applying Long-Short Term Memory Recurrent Neural Networks for Real-Time Stroke Recognition. In Companion of the 2021 ACM SIGCHI Symposium on Engineering Interactive Computing Systems(EICS ’21). Association for Computing Machinery, New York, NY, USA, 50–55. https://doi.org/10.1145/3459926.3464754

Digital Library

[29]

Hyeon-Kyu Lee and J.H. Kim. 1999. An HMM-based threshold model approach for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 10 (Oct. 1999), 961–973. https://doi.org/10.1109/34.799904 Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.

Digital Library

[30]

Luis A. Leiva, Daniel Martín-Albo, and Réjean Plamondon. 2016. Gestures à Go Go: Authoring Synthetic Human-Like Stroke Gestures Using the Kinematic Theory of Rapid Movements. ACM Transactions on Intelligent Systems and Technology 7, 2 (Jan. 2016), 1–29. https://doi.org/10.1145/2799648

Digital Library

[31]

Jiajun Li, Jianguo Tao, Liang Ding, Haibo Gao, Zongquan Deng, Yang Luo, and Zhandong Li. 2018. A New Iterative Synthetic Data Generation Method for CNN Based Stroke Gesture Recognition. Multimedia Tools and Applications 77, 13 (July 2018), 17181–17205. https://doi.org/10.1007/s11042-017-5285-6

Digital Library

[32]

Yang Li. 2010. Protractor: A Fast and Accurate Gesture Recognizer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 2169–2172. https://doi.org/10.1145/1753326.1753654

Digital Library

[33]

Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. 2016. Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition. In Computer Vision – ECCV 2016(Lecture Notes in Computer Science), Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 816–833. https://doi.org/10.1007/978-3-319-46487-9_50 REVIEW FOR SURE.

[34]

Xiao-Hui Liu and Chin-Seng Chua. 2010. Rejection of non-meaningful activities for HMM-based activity recognition system. Image and Vision Computing 28, 6 (June 2010), 865–871. https://doi.org/10.1016/j.imavis.2009.11.001

Digital Library

[35]

Mehran Maghoumi and Joseph J. LaViola. 2019. DeepGRU: Deep Gesture Recognition Utility. In Advances in Visual Computing. Vol. 11844. Springer International Publishing, Cham, 16–31. https://doi.org/10.1007/978-3-030-33720-9_2

Digital Library

[36]

Mehran Maghoumi, Eugene Matthew Taranta, and Joseph LaViola. 2021. DeepNAG: Deep Non-Adversarial Gesture Generation. In 26th International Conference on Intelligent User Interfaces. ACM, College Station TX USA, 213–223. https://doi.org/10.1145/3397481.3450675

Digital Library

[37]

Nathan Magrofuoco, Paolo Roselli, and Jean Vanderdonckt. 2022. Two-dimensional Stroke Gesture Recognition: A Survey. Comput. Surveys 54, 7 (Sept. 2022), 1–36. https://doi.org/10.1145/3465400

Digital Library

[38]

Ross Messing, Chris Pal, and Henry Kautz. 2009. Activity Recognition Using the Velocity Histories of Tracked Keypoints. In 2009 IEEE 12th International Conference on Computer Vision. 104–111. https://doi.org/10.1109/ICCV.2009.5459154

[39]

Agnieszka Mikołajczyk and Michał Grochowski. 2018. Data augmentation for improving deep learning in image classification problem. In 2018 International Interdisciplinary PhD Workshop (IIPhDW). 117–122. https://doi.org/10.1109/IIPHDW.2018.8388338

[40]

Francisco J. Moreno-Barea, José M. Jerez, and Leonardo Franco. 2020. Improving Classification Accuracy Using Data Augmentation on Small Data Sets. Expert Systems with Applications 161 (Dec. 2020), 113696. https://doi.org/10.1016/j.eswa.2020.113696

[41]

Miguel A. Nacenta, Yemliha Kamber, Yizhou Qiang, and Per Ola Kristensson. 2013. Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’13). Association for Computing Machinery, New York, NY, USA, 1099–1108. https://doi.org/10.1145/2470654.2466142

Digital Library

[42]

Vinod Nair and Geoffrey E. Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10). Omnipress, Haifa, Israel, 807–814. https://icml.cc/Conferences/2010/papers/432.pdf

Digital Library

[43]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., Vancouver, Canada, 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

Digital Library

[44]

Réjean Plamondon. 1995. A kinematic theory of rapid human movements: Part I. Movement representation and generation. Biological cybernetics 72, 4 (1995), 295–307.

Digital Library

[45]

Réjean Plamondon and Moussa Djioua. 2006. A multi-level representation paradigm for handwriting stroke generation. Human movement science 25, 4-5 (2006), 586–607.

[46]

Robert Powalka. 1993. Experiments With Applying Slant Counteraction to Script Recognition.

[47]

Hossein Rahmani and Ajmal Mian. 2016. 3D Action Recognition from Novel Viewpoints. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Las Vegas, NV, USA, 1506–1515. https://doi.org/10.1109/CVPR.2016.167

[48]

Dean Rubine. 1991. Specifying Gestures by Example. ACM SIGGRAPH Computer Graphics 25, 4 (July 1991), 329–337. https://doi.org/10.1145/127719.122753

Digital Library

[49]

Yasushi Sakurai, Christos Faloutsos, and Masashi Yamamuro. 2007. Stream Monitoring under the Time Warping Distance. In 2007 IEEE 23rd International Conference on Data Engineering. IEEE, Istanbul, 1046–1055. https://doi.org/10.1109/ICDE.2007.368963

[50]

Jan Schlüter and Thomas Grill. 2015. Exploring Data Augmentation for Improved Singing Voice Detection with Neural Networks. In ISMIR. 121–126.

[51]

Jia Sheng. 2003. A study of adaboost in 3d gesture recognition. Department of Computer Science, University of Toronto 1 (2003), 7 pages.

[52]

Arash Shilandari, H. Marvi, and H. Khosravi. 2021. Speech Emotion Recognition Using Data Augmentation Method by Cycle-Generative Adversarial Networks. https://doi.org/10.20944/PREPRINTS202104.0651.V1

[53]

Connor Shorten and Taghi M. Khoshgoftaar. 2019. A Survey on Image Data Augmentation for Deep Learning. Journal of Big Data 6, 1 (July 2019), 60. https://doi.org/10.1186/s40537-019-0197-0

[54]

Clifford K. F. So and George Baciu. 2006. Hypercube sweeping algorithm for subsequence motion matching in large motion databases. In Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications(VRCIA ’06). Association for Computing Machinery, New York, NY, USA, 221–228. https://doi.org/10.1145/1128923.1128960

Digital Library

[55]

Sijie Song, Cuiling Lan, Junliang Xing, Wenjun Zeng, and Jiaying Liu. 2016. An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data. arXiv:1611.06067 [cs] (Nov. 2016). http://arxiv.org/abs/1611.06067 arXiv:1611.06067.

[56]

Odongo Steven Eyobu and Dong Seog Han. 2018. Feature Representation and Data Augmentation for Human Activity Classification Based on Wearable IMU Sensor Data Using a Deep LSTM Neural Network. Sensors 18, 9 (Sept. 2018), 2892. https://doi.org/10.3390/s18092892

[57]

Jingren Tang, Hong Cheng, Yang Zhao, and Hongliang Guo. 2018. Structured dynamic time warping for continuous hand trajectory gesture recognition. Pattern Recognition 80 (Aug. 2018), 21–31. https://doi.org/10.1016/j.patcog.2018.02.011

[58]

Eugene M. Taranta and Joseph J. LaViola. 2015. Penny pincher: a blazing fast, highly accurate $-family recognizer. In Proceedings of the 41st Graphics Interface Conference(GI ’15). Canadian Information Processing Society, CAN, 195–202.

[59]

Eugene M. Taranta, Mehran Maghoumi, Corey R. Pittman, and Joseph J. LaViola. 2016. A Rapid Prototyping Approach to Synthetic Data Generation for Improved 2D Gesture Recognition. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST ’16). Association for Computing Machinery, New York, NY, USA, 873–885. https://doi.org/10.1145/2984511.2984525

Digital Library

[60]

Eugene M. Taranta, Andrés N. Vargas, and Joseph J. LaViola. 2016. Streamlined and accurate gesture recognition with Penny Pincher. Computers & Graphics 55 (April 2016), 130–142. https://doi.org/10.1016/j.cag.2015.10.011

Digital Library

[61]

Eugene M. Taranta II, Corey R. Pittman, Mehran Maghoumi, Mykola Maslych, Yasmine M. Moolenaar, and Joseph J. Laviola Jr. 2021. Machete: Easy, Efficient, and Precise Continuous Custom Gesture Segmentation. ACM Transactions on Computer-Human Interaction 28, 1 (Jan. 2021), 5:1–5:46. https://doi.org/10.1145/3428068

Digital Library

[62]

Eugene M. Taranta II, Amirreza Samiei, Mehran Maghoumi, Pooya Khaloo, Corey R. Pittman, and Joseph J. LaViola Jr.2017. Jackknife: A Reliable Recognizer with Few Samples and Many Modalities. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 5850–5861. https://doi.org/10.1145/3025453.3026002

Digital Library

[63]

Eugene M. Taranta II, Thaddeus K. Simons, Rahul Sukthankar, and Joseph J. Laviola Jr.2015. Exploring the Benefits of Context in 3D Gesture Recognition for Game-Based Virtual Environments. ACM Transactions on Interactive Intelligent Systems 5, 1 (March 2015), 1:1–1:34. https://doi.org/10.1145/2656345

Digital Library

[64]

Jean Vanderdonckt, Paolo Roselli, and Jorge Luis Pérez-Medina. 2018. !FTL, an Articulation-Invariant Stroke Gesture Recognizer with Controllable Position, Scale, and Rotation Invariances. In Proceedings of the 20th ACM International Conference on Multimodal Interaction(ICMI ’18). Association for Computing Machinery, New York, NY, USA, 125–134. https://doi.org/10.1145/3242969.3243032

Digital Library

[65]

Radu-Daniel Vatavu. 2017. Improving Gesture Recognition Accuracy on Touch Screens for Users with Low Vision. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 4667–4679. https://doi.org/10.1145/3025453.3025941

Digital Library

[66]

Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2012. Gestures as Point Clouds: A $P Recognizer for User Interface Prototypes. In Proceedings of the 14th ACM International Conference on Multimodal Interaction(ICMI ’12). Association for Computing Machinery, New York, NY, USA, 273–280. https://doi.org/10.1145/2388676.2388732

Digital Library

[67]

Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2013. Relative accuracy measures for stroke gestures. In Proceedings of the 15th ACM on International conference on multimodal interaction(ICMI ’13). Association for Computing Machinery, New York, NY, USA, 279–286. https://doi.org/10.1145/2522848.2522875

Digital Library

[68]

Radu-Daniel Vatavu, Lisa Anthony, and Jacob O. Wobbrock. 2018. $Q: A Super-Quick, Articulation-Invariant Stroke-Gesture Recognizer for Low-Resource Devices. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services(MobileHCI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3229434.3229465

Digital Library

[69]

R. D. Vatavu, S. G. Pentiuc, L. Grisoni, and C. Chaillou. 2008. Modeling Shapes for Pattern Recognition: A Simple Low-Cost Spline-based Approach. Advances in Electrical and Computer Engineering 8, 1(2008), 67–71. https://doi.org/10.4316/aece.2008.01012

[70]

Radu-Daniel Vatavu, Daniel Vogel, Géry Casiez, and Laurent Grisoni. 2011. Estimating the Perceived Difficulty of Pen Gestures. In Human-Computer Interaction – INTERACT 2011(Lecture Notes in Computer Science), Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer, Berlin, Heidelberg, 89–106. https://doi.org/10.1007/978-3-642-23771-3_9

[71]

Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. 2014. Human Action Recognition by Representing 3D Skeletons as Points in a Lie Group. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Columbus, OH, USA, 588–595. https://doi.org/10.1109/CVPR.2014.82

Digital Library

[72]

Qingsong Wen, Liang Sun, Xiaomin Song, Jing Gao, Xue Wang, and Huan Xu. 2021. Time Series Data Augmentation for Deep Learning: A Survey. In IJCAI. https://doi.org/10.24963/ijcai.2021/631 Augmentation, Time Series.

[73]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology(UIST ’07). Association for Computing Machinery, New York, NY, USA, 159–168. https://doi.org/10.1145/1294211.1294238

Digital Library

[74]

Sebastien C Wong, Adam Gatt, Victor Stamatescu, and Mark D McDonnell. 2016. Understanding data augmentation for classification: when to warp?. In 2016 international conference on digital image computing: techniques and applications (DICTA). IEEE, 1–6.

[75]

Xinyu Yang, Zhenguo Zhang, Xu Cui, and Rong-yi Cui. 2021. A Time Series Data Augmentation Method Based on Dynamic Time Warping. 2021 International Conference on Computer Communication and Artificial Intelligence (CCAI) (2021). https://doi.org/10.1109/CCAI50917.2021.9447507

[76]

Yong Du, Wei Wang, and Liang Wang. 2015. Hierarchical recurrent neural network for skeleton based action recognition. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Boston, MA, USA, 1110–1118. https://doi.org/10.1109/CVPR.2015.7298714

[77]

Qian Yu, Yongxin Yang, Feng Liu, Yi-Zhe Song, Tao Xiang, and Timothy M. Hospedales. 2017. Sketch-a-Net: A Deep Neural Network that Beats Humans. International Journal of Computer Vision 122, 3 (May 2017), 411–425. https://doi.org/10.1007/s11263-016-0932-3

Digital Library

[78]

Pengfei Zhang, Cuiling Lan, Junliang Xing, Wenjun Zeng, Jianru Xue, and Nanning Zheng. 2017. View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Data. In 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, Venice, 2136–2145. https://doi.org/10.1109/ICCV.2017.233

[79]

Ying Zheng, Hongxun Yao, Xiaoshuai Sun, Shengping Zhang, Sicheng Zhao, and Fatih Porikli. 2021. Sketch-Specific Data Augmentation for Freehand Sketch Recognition. Neurocomputing 456 (Oct. 2021), 528–539. https://doi.org/10.1016/j.neucom.2020.05.124

Digital Library

[80]

Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. 2020. Random Erasing Data Augmentation. Proceedings of the AAAI Conference on Artificial Intelligence 34, 07 (April 2020), 13001–13008. https://doi.org/10.1609/aaai.v34i07.7000 Number: 07.

[81]

Guangming Zhu, Liang Zhang, Peiyi Shen, and Juan Song. 2017. Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM. IEEE Access 5(2017), 4517–4524. https://doi.org/10.1109/ACCESS.2017.2684186

Cited By

Harnett MLacy AYu PFriesen R(2025)Haptic Interaction Methods for Freehand Contour Generation on a Refreshable Pin DisplayJournal of Computing and Information Science in Engineering10.1115/1.406741725:3Online publication date: 27-Jan-2025
https://doi.org/10.1115/1.4067417
Hosseini-Kivanani NSalobrar-García EElvira-Hurtado LSalas MSchommer CLeiva L(2024)Blueprint of Tomorrow: Contrasting Off-Line and On-Line Drawing Tasks for Alzheimer’s Disease ScreeningIntelligent Data Engineering and Automated Learning – IDEAL 202410.1007/978-3-031-77731-8_38(422-433)Online publication date: 14-Nov-2024
https://doi.org/10.1007/978-3-031-77731-8_38
Yusuf OHabib MMoustafa M(2023)Transforming Hand Gesture Recognition Into Image Classification Using Data Level FusionGlobal Perspectives on Robotics and Autonomous Systems10.4018/978-1-6684-7791-5.ch003(39-78)Online publication date: 16-Jun-2023
https://doi.org/10.4018/978-1-6684-7791-5.ch003

Index Terms

Effective 2D Stroke-based Gesture Augmentation for RNNs
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
    2. Interactive systems and tools
      1. User interface programming
2. Theory of computation
  1. Design and analysis of algorithms
    1. Data structures design and analysis
      1. Pattern matching

Recommendations

Data augmentation for univariate time series forecasting with neural networks
Highlights
- Empirical evaluation of data augmentation techniques for univariate time series forecasting.
Abstract
Neural networks have been proven particularly accurate in univariate time series forecasting settings, requiring however a significant number of training samples to be effectively trained. In machine learning applications where ...
Illumination-based Augmentation for Cuneiform Deep Neural Sign Classification
Automated content-based search for arbitrary cuneiform signs in photographic reproductions is a challenging task in the analysis of ancient documents, a central component of which is a reliable cuneiform sign classification. We present an illumination-...
Towards building robust DNN applications: an industrial case study of evolutionary data augmentation
ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

Data augmentation techniques that increase the amount of training data by adding realistic transformations are used in machine learning to improve the level of accuracy. Recent studies have demonstrated that data augmentation techniques improve the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

14911 pages

ISBN:9781450394215

DOI:10.1145/3544548

Editors:
Albrecht Schmidt
LMU Munich, Germany60028717
,
Kaisa Väänänen
Tampere University, Finland60011170
,
Tesh Goyal
Google Research, USA60006191
,
Per Ola Kristensson
University of Cambridge, UK60031101
,
Anicia Peters
University of Namibia, Namibia60072704
,
Stefanie Mueller
Massachusetts Institute of Technology, USA60022195
,
Julie R. Williamson
University of Glasgow, UK60001490
,
Max L. Wilson
University of Nottingham, UK60015138

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
549
Total Downloads

Downloads (Last 12 months)214
Downloads (Last 6 weeks)11

Reflects downloads up to 04 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Harnett MLacy AYu PFriesen R(2025)Haptic Interaction Methods for Freehand Contour Generation on a Refreshable Pin DisplayJournal of Computing and Information Science in Engineering10.1115/1.406741725:3Online publication date: 27-Jan-2025
https://doi.org/10.1115/1.4067417
Hosseini-Kivanani NSalobrar-García EElvira-Hurtado LSalas MSchommer CLeiva L(2024)Blueprint of Tomorrow: Contrasting Off-Line and On-Line Drawing Tasks for Alzheimer’s Disease ScreeningIntelligent Data Engineering and Automated Learning – IDEAL 202410.1007/978-3-031-77731-8_38(422-433)Online publication date: 14-Nov-2024
https://doi.org/10.1007/978-3-031-77731-8_38
Yusuf OHabib MMoustafa M(2023)Transforming Hand Gesture Recognition Into Image Classification Using Data Level FusionGlobal Perspectives on Robotics and Autonomous Systems10.4018/978-1-6684-7791-5.ch003(39-78)Online publication date: 16-Jun-2023
https://doi.org/10.4018/978-1-6684-7791-5.ch003

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten