research-article

Fast robotic pencil drawing based on image evolution by means of genetic algorithm

Authors:

Jarmila Pavlovicova,

Andrej Babinec,

Ivan SekajAuthors Info & Claims

Volume 148, Issue C

https://doi.org/10.1016/j.robot.2021.103912

Published: 01 February 2022 Publication History

Abstract

Even if pencil drawing is the basic method of human artistic expression, it remains at the forefront of scientific attention in the field of robotics, which focuses mainly on painting. Although various methods of artistic robotic drawing have been developed in the past, the results are not always satisfactory. The vast majority of existing systems focus only on sketches, and detailed pencil drawings are time-consuming.

In this article, we present a novel general robotic system for creating realistic pencil drawings based on image evolution. We show that by procedural image generation approach using genetic algorithms, we can create realistic drawings with an element of machine creativity. Furthermore, we show that the image approximation using even simple line segments leads to aesthetic and fast drawings. Finally, we describe a hardware solution using an industrial robot, a software implementation, and preliminary experimental results.

References

[1]

Jeon M., Robotic arts: Current practices, potentials, and implications, Multimodal Technol. Interact. 1 (2) (2017) 5,.

[2]

Yu D., Chen H., A review of robotic drawing, in: 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems, CYBER, IEEE, 2018, pp. 334–338.

[3]

Herath D., Kroos C., et al., Robots and Art: Exploring an Unlikely Symbiosis, Springer, 2016,.

[4]

Mazzone M., Elgammal A., Art, creativity, and the potential of artificial intelligence, in: Arts, Vol. 8, No. 1, Multidisciplinary Digital Publishing Institute, 2019, p. 26.

[5]

Hinwood D., Ireland J., Jochum E.A., Herath D., A proposed wizard of OZ architecture for a human-robot collaborative drawing task, in: International Conference on Social Robotics, Springer, 2018, pp. 35–44.

[6]

Scalera L., Seriani S., Gasparetto A., Gallina P., Non-photorealistic rendering techniques for artistic robotic painting, Robotics 8 (1) (2019),. URL https://www.mdpi.com/2218-6581/8/1/10.

[7]

Igno-Rosario O., Hernandez-Aguilar C., Cruz-Orea A., Dominguez-Pacheco A., Interactive system for painting artworks by regions using a robot, Robot. Auton. Syst. (2019),.

Digital Library

[8]

Advanced tone rendition technique for a painting robot, Robot. Auton. Syst. (2019),.

Digital Library

[9]

Jain S., Gupta P., Kumar V., Sharma K., A force-controlled portrait drawing robot, in: Proceedings of the IEEE International Conference on Industrial Technology, 2015,.

[10]

Song D., Lee T., Kim Y.J., Artistic pen drawing on an arbitrary surface using an impedance-controlled robot, in: Proceedings - IEEE International Conference on Robotics and Automation, 2018,.

Digital Library

[11]

Song D., Kim Y.J., Distortion-free robotic surface-drawing using conformal mapping, in: Proceedings - IEEE International Conference on Robotics and Automation, 2019,.

Digital Library

[12]

O’Dowd P.J., A robot that draws and shades with tactile force feedback sensed through a pencil, 2019,.

[13]

Wujek A., Winiarski T., Automated drawing recognition and reproduction with a multisensory robotic manipulation system, in: Advances in Intelligent Systems and Computing, 2016,.

[14]

Gao Q., Chen H., Yu R., Yang J., Duan X., A robot portraits pencil sketching algorithm based on face component and texture segmentation, in: Proceedings of the IEEE International Conference on Industrial Technology, 2019,.

[15]

Tresset P., Fol Leymarie F., Portrait drawing by Paul the robot, Comput. Graph. (Pergamon) (2013),.

Digital Library

[16]

Calinon S., Epiney J., Billard A., A humanoid robot drawing human portraits, in: Proceedings of 2005 5th IEEE-RAS International Conference on Humanoid Robots, 2005,.

[17]

Adamik M., Babinec A., Chovanec L., Tool path generator for artistic drawing with industrial robot, in: 2019 22nd IEEE International Symposium on Measurement and Control in Robotics: Robotics for the Benefit of Humanity, ISMCR 2019, 2019,.

[18]

Singhal A., Kumar A., Thukral S., Raina D., Kumar S., Chitrakar: Robotic system for drawing jordan curve of facial portrait, 2020, ArXiv abs/2011.10781.

[19]

Yi R., Liu Y.J., Lai Y.K., Rosin P.L., Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019,.

[20]

Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y., Generative adversarial nets, in: Advances in Neural Information Processing Systems, 2014,.

[21]

Wang T., Toh W.Q., Zhang H., Sui X., Li S., Liu Y., Jing W., RoboCoDraw: Robotic avatar drawing with GAN-based style transfer and time-efficient path optimization, 2019,. arXiv:1912.05099.

[22]

Calinon S., Epiney J., Billard A., A humanoid robot drawing human portraits, in: 5th IEEE-RAS International Conference on Humanoid Robots, 2005, IEEE, 2005, pp. 161–166.

[23]

Singh A.K., Nandi G., NAO humanoid robot: Analysis of calibration techniques for robot sketch drawing, Robot. Auton. Syst. 79 (2016) 108–121,. URL https://www.sciencedirect.com/science/article/pii/S0921889016000166.

Digital Library

[24]

Lau M.-C., Anderson J., Baltes J., A sketch drawing humanoid robot using image-based visual servoing, Knowl. Eng. Rev. 34 (2019).

[25]

Paul O., O’Dowd P., Robot Poetics: Toward a materially sensitive drawing robot, 2018,.

Digital Library

[26]

Ambrosio C., Unsettling robots and the future of art, Science 365 (6448) (2019) 38–39,. arXiv:https://science.sciencemag.org/content/365/6448/38.full.pdf, URL https://science.sciencemag.org/content/365/6448/38.

[27]

Holland J.H., Adaptation in Natural and Artificial Systems. An Introductory Analysis Applications to Biology, Control and Artificial Intelligence, University of Michigan Press, Ann Arbor, 1975.

[28]

Golberg D.E., Genetic algorithms in search, optimization, and machine learning, Addion Wesley 1989 (102) (1989) 36,.

[29]

Koza J.R., Koza J.R., Genetic Programming: On the Programming of Computers by Means of Natural Selection, Vol. 1, MIT Press, 1992.

[30]

Sims K., Artificial evolution for computer graphics, in: Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1991, 1991,.

Digital Library

[31]

Dawkins R., The Blind Watchmaker - Evidence of Evolution Reveals a Universe Without Design, 1986.

[32]

E. Baker, M. Seltzer, Evolving line drawings, in: Proceedings - Graphics Interface, 1994.

[33]

Machado P., Pereira L., Photogrowth: Non-photorealistic renderings through ant paintings, in: Proceedings of the 14th International Conference on Genetic and Evolutionary Computation, GECCO’12, 2012,.

Digital Library

[34]

Weller C., Generation of vector-based graphics from existing bitmap images by means of the genetic algorithm, 2003.

[35]

Johansson R., Genetic programming: Evolution of mona lisa, 2008, URL https://rogerjohansson.blog/2008/12/07/genetic-programming-evolution-of-mona-lisa/. (Accessed 30 June 2021).

[36]

Barile P., Ciesielski V., Trist K., Non-photorealistic rendering using genetic programming, in: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2008,.

Digital Library

[37]

Trist K., Ciesielski V., Barile P., Can’t see the forest: Using an evolutionary algorithm to produce an animated artwork, in: Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, 2010,.

[38]

Barile P., Ciesielski V., Berry M., Trist K., Animated drawings rendered by genetic programming, in: Proceedings of the 11th Annual Genetic and Evolutionary Computation Conference, GECCO-2009, 2009,.

Digital Library

[39]

Ma Y., The mathematic magic of photoshop blend modes for image processing, in: 2011 International Conference on Multimedia Technology, ICMT 2011, 2011,.

[40]

Bhattacharya M., Evolutionary approaches to expensive optimisation, Int. J. Adv. Res. Artif. Intell. (2013),.

[41]

Fogleman M., Primitive, 2016, https://github.com/fogleman/primitive. (Accessed 30 June 2021).

[42]

Huang Y.-P., Huang C.-H., Real-valued genetic algorithms for fuzzy grey prediction system, Fuzzy Sets and Systems 87 (3) (1997) 265–276.

[43]

T. Pencheva, K. Atanassov, A. Shannon, Modelling of a stochastic universal sampling selection operator in genetic algorithms using generalized nets, in: Proceedings of the Tenth International Workshop on Generalized Nets, Sofia, 2009, pp. 1–7.

[44]

Robotics A., Product specification IRB 120, 2021.

[45]

Robotics A., Technical Reference Manual: RAPID Instructions, Functions and Data Types, ABB Robotics, Västerås, Sweden, 2014.

[46]

Marecek J., The traveling salesman problem: a computational study, Interfaces 38 (4) (2008) 344.

[47]

Harris C.R., Millman K.J., van der Walt S.J., Gommers R., Virtanen P., Cournapeau D., Wieser E., Taylor J., Berg S., Smith N.J., Kern R., Picus M., Hoyer S., van Kerkwijk M.H., Brett M., Haldane A., del Río J.F., Wiebe M., Peterson P., Gérard-Marchant P., Sheppard K., Reddy T., Weckesser W., Abbasi H., Gohlke C., Oliphant T.E., Array programming with NumPy, Nature 585 (7825) (2020) 357–362,.

[48]

Clark A., Pillow (PIL Fork) Documentation, readthedocs, 2015, URL https://buildmedia.readthedocs.org/media/pdf/pillow/latest/pillow.pdf.

[49]

pashango A., Image4Layer, 2017, https://github.com/pashango2/Image4Layer.

[50]

Elliott & Fry - WikiImages A., Charles darwin [Photograph]. Pixabay, 2012, [Online]; URL https://pixabay.com/photos/charles-darwin-scientists-62967/. (Accessed 30 June 2021).

[51]

Cassey Cambridge A., Black and white photo of the flatiron building architecture clock [Photograph]. Pixy, 2014, [Online]; URL https://pixy.org/8216/. (Accessed 30 June 2021).

[52]

USB 3.0 camera series X., xiQ - technical manual version, 2013.

[53]

Sousa M.C., Buchanan J.W., Observational models of graphite pencil materials, Comput. Graph. Forum 19 (1) (2000) 27–49,. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/1467-8659.00386, URL https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-8659.00386.

Cited By

Karimov AKopets ELeonov SScalera LButusov D(2023)A Robot for Artistic Painting in Authentic ColorsJournal of Intelligent and Robotic Systems10.1007/s10846-023-01831-4107:3Online publication date: 4-Mar-2023
https://dl.acm.org/doi/10.1007/s10846-023-01831-4
Guo DYe LYan GMin H(2023)CCD-BSM:composite-curve-dilation brush stroke model for robotic chinese calligraphyApplied Intelligence10.1007/s10489-022-04210-y53:11(14269-14283)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s10489-022-04210-y

Index Terms

Fast robotic pencil drawing based on image evolution by means of genetic algorithm

Index terms have been assigned to the content through auto-classification.

Recommendations

Architectural Drawing Using Pencil Sketches and AutoCAD
Combining sketch and tone for pencil drawing production
NPAR '12: Proceedings of the Symposium on Non-Photorealistic Animation and Rendering

We propose a new system to produce pencil drawing from natural images. The results contain various natural strokes and patterns, and are structurally representative. They are accomplished by novelly combining the tone and stroke structures, which ...
Technical Drawing 101 with AutoCAD 2016

Comments

Information & Contributors

Information

Published In

cover image Robotics and Autonomous Systems

Robotics and Autonomous Systems Volume 148, Issue C

Feb 2022

165 pages

ISSN:0921-8890

Issue’s Table of Contents

The Authors.

Publisher

North-Holland Publishing Co.

Netherlands

Publication History

Published: 01 February 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Karimov AKopets ELeonov SScalera LButusov D(2023)A Robot for Artistic Painting in Authentic ColorsJournal of Intelligent and Robotic Systems10.1007/s10846-023-01831-4107:3Online publication date: 4-Mar-2023
https://dl.acm.org/doi/10.1007/s10846-023-01831-4
Guo DYe LYan GMin H(2023)CCD-BSM:composite-curve-dilation brush stroke model for robotic chinese calligraphyApplied Intelligence10.1007/s10489-022-04210-y53:11(14269-14283)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1007/s10489-022-04210-y

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents