research-article

Dynamic Graffiti Stylisation with Stochastic Optimal Control

Authors:

Sylvain Calinon,

Frederic Fol LeymarieAuthors Info & Claims

MOCO '17: Proceedings of the 4th International Conference on Movement Computing

Article No.: 18, Pages 1 - 8

https://doi.org/10.1145/3077981.3078044

Published: 28 June 2017 Publication History

Abstract

We present a method for the interactive generation of stylised letters, curves and motion paths that are similar to the ones that can be observed in art forms such as graffiti and calligraphy. We define various stylisations of a letter form over a common geometrical structure, which is given by the spatial layout of a sparse sequence of targets. Different stylisations are then generated by optimising the trajectories of a dynamical system that tracks the target sequence. The evolution of the dynamical system is computed with a stochastic formulation of optimal control, in which each target is defined probabilistically as a multivariate Gaussian. The covariance of each Gaussian explicitly defines the variability as well as the curvilinear evolution of trajectory segments. Given this probabilistic formulation, the optimisation procedure results in a trajectory distribution rather than a single path. It is then possible to stochastically sample from the distribution an infinite number of dynamically and aesthetically consistent trajectories which mimic the variability that is typically observed in human drawing or writing. We further demonstrate how this system can be used together with a simple user interface in order to explore different stylisations of interactively or procedurally defined letters.

References

[1]

Z. AlMeraj, B. Wyvill, T. Isenberg, A.A. Gooch, and R. Guy. 2009. Automatically mimicking unique hand-drawn pencil lines. Computers & Graphics 33, 4 (2009).

Digital Library

[2]

F. Attneave. 1954. Some informational aspects of visual perception. Psychological review 61, 3 (1954), 183--93.

[3]

D Berio, M Akten, F. Fol Leymarie, M. Grierson, and R. Plamondon. 2017. Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks. In Proc. of 4th Int'l Conf. on Movement Computing (MOCO). London, UK.

Digital Library

[4]

D. Berio, S. Calinon, and F. Fol Leymarie. 2016. Learning dynamic graffiti strokes with a compliant robot. In Proc. of IEEE/RSJ Int'l Conf on Intelligent Robots and Systems (IROS). Daejeon, Korea, 3981--6.

[5]

D Berio, S. Calinon, and F. Fol Leymarie. 2017. Generating Calligraphic Trajectories with Model Predictive Control. In Proc. of Graphics Interface (GI). Canada.

[6]

D. Berio and F. Fol Leymarie. 2015. Computational Models for the Analysis and Synthesis of Graffiti Tag Strokes. In Workshop on Computational Aesthetics (CAe), P. Rosin (Ed.). Eurographics, Istanbul, Turkey, 35--47.

Digital Library

[7]

H. Bezine, Adel M. Alimi, and N. Sherkat. 2004. Generation and analysis of handwriting script with the beta-elliptic model. In Proc. of Int'l Workshop on Frontiers in Handwriting Recognition (IWFHR-9). IEEE, 515--20.

Digital Library

[8]

S. Calinon. 2016. Stochastic learning and control in multiple coordinate systems. In Intl Workshop on Human-Friendly Robotics. Genova, Italy, 1--5.

[9]

S. Calinon. 2016. A tutorial on task-parameterized movement learning and retrieval. Intelligent Service Robotics 9, 1 (2016), 1--29.

Digital Library

[10]

A. d'Avella, P. Saltiel, and E. Bizzi. 2003. Combinations of muscle synergies in the construction of a natural motor behavior. Nature neuroscience 6, 3 (2003), 300--308.

[11]

E.H. Dooijes. 1983. Analysis of handwriting movements. Acta Psychologica 54, 1 (1983), 99--114.

[12]

M. Egerstedt and Clyde Martin. 2009. Control Theoretic Splines: Optimal Control, Statistics, and Path Planning. Princeton University Press, Princeton Oxford.

[13]

J. Feldman and M. Singh. 2005. Information along contours and object boundaries. Psychological review 112, 1 (2005), 243.

[14]

A. Ferri. 2016. Teoria del writing, La ricerca dello stile. Professional Dreamers.

[15]

T. Flash. 1983. Organizing principles underlying the formation of arm trajectories. Ph.D. Dissertation. Massachusetts Institute of Technology.

[16]

T. Flash and A.A. Handzel. 2007. Affine differential geometry analysis of human arm movements. Biological cybernetics 96, 6 (2007), 577--601.

Digital Library

[17]

T. Flash and B. Hochner. 2005. Motor primitives in vertebrates and invertebrates. Current opinion in neurobiology 15, 6 (2005), 660--6.

[18]

T. Flash and N. Hogan. 1985. The coordination of arm movements. Journal of Neuroscience 5, 7 (1985), 1688--1703.

[19]

D. Freedberg and V. Gallese. 2007. Motion, emotion and empathy in esthetic experience. Trends in cognitive sciences 11, 5 (2007), 197--203.

[20]

F.N. Freeman. 1914. Experimental analysis of the writing movement. Psychological Monographs: General and Applied 17, 4 (1914), 1--57.

[21]

W.T. Freeman, J.B. Tenenbaum, and E.C. Pasztor. 2003. Learning style translation for the lines of a drawing. ACM Trans. on Graphics (TOG) 22, 1 (2003), 33--46.

Digital Library

[22]

D.E. Goldberg and R. Lingle, Jr. 1985. Alleles Loci and the TSP. In 1st Intl. Conf. on Genetic Algorithms. L. Erlbaum Associates Inc., 154--159.

Digital Library

[23]

P. Haeberli. 1989. Dynadraw: A dynamic drawing technique. (1989). www.graficaobscura.com/dyna/.

[24]

C.M. Harris and D.M. Wolpert. 1998. Signal-dependent noise determines motor planning. Nature 394, 6695 (1998), 780--784.

[25]

A. Hertzmann, N. Oliver, B. Curless, and S.M. Seitz. 2002. Curve Analogies. In Proc. 13th Eurographics Workshop on Rendering (EGRW). Pisa, Italy, 233--46.

Digital Library

[26]

D.H. House and M. Singh. 2007. Line Drawing as a Dynamic Process. In Proc. of 15th Pacific Conf. on Comp. Graphics & Applications. IEEE, Maui, USA, 351--60.

Digital Library

[27]

M.I. Jordan and D.M. Wolpert. 1999. Computational Motor Control. In The Cognitive Neurosciences (2nd ed.), M. Gazzaniga (Ed.). MIT Press.

[28]

J. Kimvall. 2014. The G-word. Dokument, Stockholm.

[29]

J.E. Kyprianidis, J. Collomosse, T. Wang, and T. Isenberg. 2013. State of the "Art": A Taxonomy of Artistic Stylization Techniques for Images and Video. IEEE Transactions on Visualization and Computer Graphics 19, 5 (2013), 866--885.

Digital Library

[30]

F. Lacquaniti, C. Terzuolo, and P. Viviani. 1983. The law relating the kinematic and figural aspects of drawing movements. Acta psychologica 54, 1 (1983), 115--130.

[31]

K. Lang and M. Alexa. 2015. The Markov pen: Online synthesis of free-hand drawing styles. In Proc. of Workshop on Non-Photorealistic Animation and Rendering (NPAR). Eurographics, Istanbul, Turkey, 203--15.

Digital Library

[32]

D. Llorens Piñana, F. Prat, Marzal Varo, and others. 2008. The UJIpenchars Database: A Pen-Based Database of Isolated Handwritten Characters. (2008).

[33]

P. Morasso. 1981. Spatial control of arm movements. Experimental Brain Research 42, 2 (1981), 223--7.

[34]

P. Morasso. 1986. Understanding Cursive Script as a Trajectory Formation Paradigm. Graphonomics 37 (1986), 137--67.

[35]

H. Nagasaki. 1989. Asymmetric velocity and acceleration profiles of human arm movements. Experimental Brain Research 74, 2 (1989), 319--26.

[36]

A. Pignocchi. 2010. How the Intentions of the Draftsman Shape Perception of a Drawing. Consciousness and Cognition 19, 4 (2010), 887--898.

[37]

R. Plamondon. 1995. A Kinematic Theory of Rapid Human Movements. Part I. Biological cybernetics 72, 4 (1995), 295--307.

Digital Library

[38]

R. Plamondon, C. O'Reilly, J. Galbally, A. Almaksour, and É. Anquetil. 2014. Recent developments in the study of rapid human movements with the kinematic theory. Pattern Recognition Letters 35 (2014), 225--35.

Digital Library

[39]

R. Plamondon, C. O'Reilly, C. Remi, and T. Duval. 2013. The Lognormal Hand-writer: Learning, Performing and Declining. Vol. 4.

[40]

B. Rohrer and N. Hogan. 2003. Avoiding spurious submovement decompositions: a globally optimal algorithm. Biological cybernetics 89, 3 (2003), 190--199.

Digital Library

[41]

R. Sosnik, B. Hauptmann, A. Karni, and T. Flash. 2004. When practice leads to co-articulation: the evolution of geometrically defined movement primitives. Experimental Brain Research 156, 4 (2004), 422--438.

[42]

A.K. Tanwani and S. Calinon. 2016. Learning Robot Manipulation Tasks With Task-Parameterized Semitied Hidden Semi-Markov Model. IEEE Robotics and Automation Letters 1, 1 (2016), 235--242.

[43]

H.L. Teulings and L. Schomaker. 1993. Invariant properties between stroke features in handwriting. Acta psychologica 82, 1 (1993), 69--88.

[44]

E. Todorov and M.I. Jordan. 2002. Optimal feedback control as a theory of motor coordination. Nature neuroscience 5, 11 (2002), 1226--1235.

[45]

Y. Uno, M. Kawato, and R. Suzuki. 1989. Formation and control of optimal trajectory in human multijoint arm movement. Biological cybernetics 61, 2 (1989), 89--101.

Digital Library

[46]

P. Viviani and R. Schneider. 1991. A developmental study of the relationship between geometry and kinematics in drawing movements. Journal of Experimental Psychology: Human Perception and Performance 17, 1 (1991), 198--218.

[47]

W.C. Watt. 1988. Canons of alphabetic change. In The Alphabet and the Brain: The Lateralization of Writing. Springer, 122--152.

[48]

D.M. Wolpert, J. Diedrichsen, and J.R. Flanagan. 2011. Principles of sensorimotor learning. Nature Reviews Neuroscience 12 (2011), 739--51.

[49]

M. Zeestraten, S. Calinon, and D. G. Caldwell. 2016. Variable Duration Movement Encoding with Minimal Intervention Control. In Proc. of Int'l Conf. on Robotics and Automation (ICRA). IEEE, Stockholm, Sweden, 497--503.

Cited By

Guo DMin HYan G(2023)SPSOC: Staged Pseudo-Spectral Optimal Control Optimization Model for Robotic Chinese CalligraphyIntelligent Robotics and Applications10.1007/978-981-99-6492-5_36(416-428)Online publication date: 16-Oct-2023
https://doi.org/10.1007/978-981-99-6492-5_36
Berio DLeymarie FAsente PEchevarria J(2022)StrokeStyles: Stroke-based Segmentation and Stylization of FontsACM Transactions on Graphics10.1145/350524641:3(1-21)Online publication date: 28-Apr-2022
https://dl.acm.org/doi/10.1145/3505246
Wu RChao FZhou CHuang YYang LLin CChang XShen QShang C(2022)A Developmental Evolutionary Learning Framework for Robotic Chinese Stroke WritingIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2021.309822914:3(1155-1169)Online publication date: Sep-2022
https://doi.org/10.1109/TCDS.2021.3098229
Show More Cited By

Index Terms

Dynamic Graffiti Stylisation with Stochastic Optimal Control
1. Computing methodologies
  1. Computer graphics
    1. Animation
      1. Procedural animation
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input

Recommendations

Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks
MOCO '17: Proceedings of the 4th International Conference on Movement Computing

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories ...
Reshaping and Reconstructing Handwritten Character Typeface Using Dynamic Font Model
INCOS '11: Proceedings of the 2011 Third International Conference on Intelligent Networking and Collaborative Systems

This paper considers the problem of reshaping and reconstructing a typeface of handwritten characters. From a set of handwriting motion data, characters are constructed by a modeling scheme with the dynamic font model and the theory of smoothing ...
Pathwise Optimality in Stochastic Control

We introduce a notion of pathwise optimality for stochastic control problems over an infinite time horizon, and give sufficient conditions for the existence of pathwise optimal controls. We analyze both diffusion processes and processes with discrete ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MOCO '17: Proceedings of the 4th International Conference on Movement Computing

June 2017

206 pages

ISBN:9781450352093

DOI:10.1145/3077981

Conference Chair:
Marco Gillies,
Editor:
Kiona Niehaus

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

University of Surrey

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

MOCO '17

MOCO '17: 4th International Conference on Movement Computing

June 28 - 30, 2017

London, United Kingdom

Acceptance Rates

Overall Acceptance Rate 85 of 185 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
124
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Guo DMin HYan G(2023)SPSOC: Staged Pseudo-Spectral Optimal Control Optimization Model for Robotic Chinese CalligraphyIntelligent Robotics and Applications10.1007/978-981-99-6492-5_36(416-428)Online publication date: 16-Oct-2023
https://doi.org/10.1007/978-981-99-6492-5_36
Berio DLeymarie FAsente PEchevarria J(2022)StrokeStyles: Stroke-based Segmentation and Stylization of FontsACM Transactions on Graphics10.1145/350524641:3(1-21)Online publication date: 28-Apr-2022
https://dl.acm.org/doi/10.1145/3505246
Wu RChao FZhou CHuang YYang LLin CChang XShen QShang C(2022)A Developmental Evolutionary Learning Framework for Robotic Chinese Stroke WritingIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2021.309822914:3(1155-1169)Online publication date: Sep-2022
https://doi.org/10.1109/TCDS.2021.3098229
Chen GBaek SFlorez JQian WLeigh SHutchinson SDellaert F(2022)GTGraffiti: Spray Painting Graffiti Art from Human Painting Motions with a Cable Driven Parallel Robot2022 International Conference on Robotics and Automation (ICRA)10.1109/ICRA46639.2022.9812008(4065-4072)Online publication date: 23-May-2022
https://doi.org/10.1109/ICRA46639.2022.9812008
Chamberlain RBerio DMayer VChana KLeymarie FOrgs G(2021)A dot that went for a walk: People prefer lines drawn with human‐like kinematicsBritish Journal of Psychology10.1111/bjop.12527113:1(105-130)Online publication date: 24-Aug-2021
https://doi.org/10.1111/bjop.12527
Wu RZhou CChao FYang LLin CShang C(2020)GANCCRobotInformation Sciences: an International Journal10.1016/j.ins.2019.12.079516:C(474-490)Online publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.ins.2019.12.079
Berio DAkten MLeymarie FGrierson MPlamondon RGillies M(2017)Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural NetworksProceedings of the 4th International Conference on Movement Computing10.1145/3077981.3078049(1-8)Online publication date: 28-Jun-2017
https://dl.acm.org/doi/10.1145/3077981.3078049

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents