Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

StrokeStyles: Stroke-based Segmentation and Stylization of Fonts

Published: 28 April 2022 Publication History
  • Get Citation Alerts
  • Abstract

    We develop a method to automatically segment a font’s glyphs into a set of overlapping and intersecting strokes with the aim of generating artistic stylizations. The segmentation method relies on a geometric analysis of the glyph’s outline, its interior, and the surrounding areas and is grounded in perceptually informed principles and measures. Our method does not require training data or templates and applies to glyphs in a large variety of input languages, writing systems, and styles. It uses the medial axis, curvilinear shape features that specify convex and concave outline parts, links that connect concavities, and seven junction types. We show that the resulting decomposition in strokes can be used to create variations, stylizations, and animations in different artistic or design-oriented styles while remaining recognizably similar to the input font.

    Supplementary Material

    berio (berio.zip)
    Supplemental movie, appendix, image and software files for, StrokeStyles: Stroke-based Segmentation and Stylization of Fonts

    References

    [1]
    Ery Arias-Castro, Gilad Lerman, and Teng Zhang. 2017. Spectral clustering based on local PCA. J. Mach. Learn. Res. 18, 1 (2017), 253–309.
    [2]
    Anssi Arte. 2015. Forms of Rockin’: Graffiti Letters and Popular Culture. Dokument Press.
    [3]
    Jonas August, Kaleem Siddiqi, and Steven W. Zucker. 1999. Ligature instabilities in the perceptual organization of shape. Comput. Vis. Image Understand. 76, 3 (1999), 231–243.
    [4]
    Samaneh Azadi, Matthew Fisher, Vladimir G. Kim, Zhaowen Wang, Eli Shechtman, and Trevor Darrell. 2018. Multi-content GAN for few-shot font style transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7564–7573.
    [5]
    Alex Bailey. 2001. Class-dependent Features and Multicategory Classification. Ph. D. Dissertation. Southampton University.
    [6]
    Elena Balashova, Amit H. Bermano, Vladimir G. Kim, Stephen DiVerdi, Aaron Hertzmann, and Thomas Funkhouser. 2019. Learning a stroke-based representation for fonts. Comput. Graph. Forum 38, 1 (2019), 429–442.
    [7]
    Brad Barber and H. Huhdanpaa. 1995. QHull. The Geometry Center, University of Minnesota.
    [8]
    Alexander Belyaev and Shin Yoshizawa. 2001. On evolute cusps and skeleton bifurcations. In Proceedings of the International Conference on Shape Modeling and Applications. IEEE, 134–140.
    [9]
    Daniel Berio, Paul Asente, Jose Echevarria, and Frederic Fol Leymarie. 2019. Sketching and layering graffiti primitives. In Proceedings of the 8th ACM/Eurographics Expressive Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering. 51–59.
    [10]
    Daniel Berio, Sylvain Calinon, and Frederic Fol Leymarie. 2017. Dynamic graffiti stylisation with stochastic optimal control. In Proceedings of the 4th International Conference on Movement Computing. Association for Computing Machinery. Article no. 18.
    [11]
    Daniel Berio, Frederic Fol Leymarie, and Réjean Plamondon. 2018. Expressive curve editing with the sigma lognormal model. In Proceedings of the 39th Annual European Association for Computer Graphics Conference: Short Papers. Eurographics Association, 33–36.
    [12]
    Daniel Berio, Frederic Fol Leymarie, and Réjean Plamondon. 2020. Kinematics reconstruction of static calligraphic traces from curvilinear shape features. In Proceedings of the Lognormality Principle and its Applications in e-Security, e-Learning and e-Health, Réjean Plamondon, Angelo Marcelli, and Miguel Ángel Ferrer (Eds.). Series in Machine Perception and Artificial Intelligence, Vol. 88. Chapter 11, 237–268. DOI:
    [13]
    Harry Blum. 1973. Biological shape and visual science (part I). J. Theor. Biol. 38, 2 (1973), 205–287.
    [14]
    Joseph L. Brooks. 2015. Traditional and new principles of perceptual grouping. In The Oxford Handbook of Perceptual Organization, Johan Wagemans (Ed.). Oxford University Press, 57–87.
    [15]
    Neill D. F. Campbell and Jan Kautz. 2014. Learning a manifold of fonts. ACM Trans. Graph. 33, 4 (2014). Article no. 91.
    [16]
    Xudong Chen, Zhouhui Lian, Yingmin Tang, and Jianguo Xiao. 2017. An automatic stroke extraction method using manifold learning. In Proceedings of the European Association for Computer Graphics: Short Papers (EG’17). Eurographics Association, 65–68. DOI:
    [17]
    Charles H. Cox, Philippe Coueignoux, Barry Blesser, and Murray Eden. 1982. Skeletons: A link between theoretical and physical letter descriptions. Pattern Recogn. 15, 1 (1982), 11–22.
    [18]
    Rodrigo Pena Carvalho Dos Anjos Craveiro. 2017. The influence of graffiti writing in contemporary typography. Street Art Urban Creativ. Sci. J. 3, 2 (2017), 65–83.
    [19]
    Joeri De Winter and Johan Wagemans. 2006. Segmentation of object outlines into parts: A large-scale integrative study. Cognition 99, 3 (2006), 275–325.
    [20]
    Oliver Deussen, Thomas Lindemeier, Sören Pirk, and Mark Tautzenberger. 2012. Feedback-guided stroke placement for a painting machine. In Proceedings of the 8th Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. Eurographics Association, 25–33.
    [21]
    Shay Deutsch and Gérard Medioni. 2017. Learning the geometric structure of manifolds with singularities using the tensor voting graph. J. Math. Imag. Vis. 57, 3 (2017), 402–422.
    [22]
    Michel Marie Deza and Elena Deza. 2013. Encyclopedia of Distances. Springer.
    [23]
    Tim Dwyer, Nathan Hurst, and Damian Merrick. 2008. A fast and simple heuristic for metro map path simplification. In Proceedings of the International Symposium on Visual Computing. Springer, 22–30.
    [24]
    Udo A. Ernst, Sunita Mandon, Nadja Schinkel–Bielefeld, Simon D. Neitzel, Andreas K. Kreiter, and Klaus R. Pawelzik. 2012. Optimality of human contour integration. PLOS Comput. Biol. 8, 5 (2012), 1–17.
    [25]
    Andreas Fabri and Sylvain Pion. 2009. CGAL: The computational geometry algorithms library. In Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (GIS’09). 538–539.
    [26]
    Alexandre Faure, Lilian Buzer, and Fabien Feschet. 2009. Tangential cover for thick digital curves. Pattern Recogn. 42, 10 (2009), 2279–2287.
    [27]
    Jean-Dominique Favreau, Florent Lafarge, and Adrien Bousseau. 2016. Fidelity vs. simplicity: A global approach to line drawing vectorization. ACM Trans. Graph. 35, 4 (2016). Article no. 120.
    [28]
    Vicky Froyen, Jacob Feldman, and Manish Singh. 2015. Bayesian hierarchical grouping: Perceptual grouping as mixture estimation. Psychol. Rev. 122, 4 (2015), 575–597.
    [29]
    Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J Mitra. 2011. Animated construction of line drawings. In ACM Trans. Graph. 30 (2011). 1–10.
    [30]
    Mikel Galar, Alberto Fernández, Edurne Barrenechea, Humberto Bustince, and Francisco Herrera. 2011. An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes. Pattern Recogn. 44, 8 (2011), 1761–1776.
    [31]
    Étienne Ghys, Sergei Tabachnikov, and Vladlen Timorin. 2013. Osculating curves: Around the tait-kneser theorem. Math. Intell. 35, 1 (2013), 61–66.
    [32]
    Peter J. Giblin and Benjamin B. Kimia. 2003. On the local form and transitions of symmetry sets, medial axes, and shocks. Int. J. Comput. Vis. 54, 1 (Aug. 2003), 143–157.
    [33]
    Yotam Gingold, David Salesin, and Denis Zorin. 2008. Stroke-by-Stroke Glyph Animation. Technical Report. Creativity and Graphics Lab (CraGL) at George Mason University, Fairfax, Virginia.
    [34]
    Andrew Goldberg, Xiaojin Zhu, Aarti Singh, Zhiting Xu, and Robert Nowak. 2009. Multi-manifold semi-supervised learning. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics, David van Dyk and Max Welling (Eds.), Vol. 5. PMLR, 169–176. https://proceedings.mlr.press/v5/goldberg09a.html.
    [35]
    David Ha and Douglas Eck. 2018. A neural representation of sketch drawings. In Proceedings o f the 6th International Conference on Learning Representations (ICLR’18).
    [36]
    Tom S. F. Haines, Oisin Mac Aodha, and Gabriel J. Brostow. 2016. My text in your handwriting. ACM Trans. Graph. 35, 3 (2016). Article no. 26.
    [37]
    Katherine A. Heller and Zoubin Ghahramani. 2005. Bayesian hierarchical clustering. In Proceedings of the 22nd International Conference on Machine learning (ICML’05). ACM, 297–304.
    [38]
    Jacky Herz, Roger D. Hersch, and Jakob Gonczarowski. 1997. Coherent processing of character skeletal forms. Comput. Graph. 21, 6 (1997), 727–736.
    [39]
    Donald D. Hoffman and Whitman A. Richards. 1984. Parts of recognition. Cognition 18, 1-3 (1984), 65–96.
    [40]
    Donald D. Hoffman and Manish Singh. 1997. Salience of visual parts. Cognition 63, 1 (1997), 29–78.
    [41]
    Douglas R. Hofstadter. 1982. Variations on a theme as the essence of imagination. Sci. Am. 247, 4 (1982), 14–21.
    [42]
    Siu Chi Hsu and Irene H. H. Lee. 1994. Drawing and animation using skeletal strokes. In Proceedings of the21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’94), 109–118.
    [43]
    S. C. Hsu, I. H. H. Lee, and N. E. Wiseman. 1993. Skeletal strokes. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology (UIST’93). 197–206.
    [44]
    Changyuan Hu and Roger D. Hersch. 2001. Parameterizable fonts based on shape components. IEEE Comput. Graph. Appl. 21, 3 (2001), 70–85.
    [45]
    Elena J. Jakubiak, Ronald N. Perry, and Sarah F. Frisken. 2006. An improved representation for stroke-based fonts. In ACM SIGGRAPH 2006 Sketches.
    [46]
    Tingting Jiang, Zhongqian Dong, Chang Ma, and Yizhou Wang. 2013. Toward perception-based shape decomposition. In Proceedings of the Asia Conference onComputer Vision (ACCV’12). Lecture Notes in Computer Science, Vol. LNCS 7725. Springer, 188–201.
    [47]
    Mark Kachanov, Boris Shafiro, and Igor Tsukrov. 2003. Handbook of Elasticity Solutions. Springer Netherlands.
    [48]
    Peter Karow. 1994. Digital Typefaces: Description and Formats. Springer.
    [49]
    Byungsoo Kim, Oliver Wang, A Cengiz Öztireli, and Markus Gross. 2018. Semantic segmentation for line drawing vectorization using neural networks. Comput. Graph. Forum 37, 2 (2018), 329–338. DOI:
    [50]
    Shaunak Kishore. 2018. Make Me a Hanzi Dataset. Retrieved from https://github.com/skishore/makemeahanzi.
    [51]
    Donald E. Knuth. 1979. Mathematical typography. Bull. Am. Math. Soc. 1, 2 (1979), 337–373.
    [52]
    Atsunobu Kotani, Stefanie Tellex, and James Tompkin. 2020. Generating handwriting via decoupled style descriptors. In Proceedings of the European Conference on Computer Vision (ECCV’20). 764–780.
    [53]
    Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. 2015. Human-level concept learning through probabilistic program induction. Science 350, 6266 (2015), 1332–1338.
    [54]
    Bart Lamiroy, Thomas Bouville, Julien Blégean, Hongliu Cao, Salah Ghamizi, Romain Houpin, and Matthias Lloyd. 2015. Re-typograph phase I: A proof-of-concept for typeface parameter extraction from historical documents. In Document Recognition and Retrieval XXII, Eric K. Ringger and Bart Lamiroy (Eds.), Vol. 9402. International Society for Optics and Photonics, SPIE, 80–91. DOI:
    [55]
    Longin Jan Latecki and Rolf Lakämper. 1998. Discrete approach to curve evolution. In Mustererkennung 1998. Springer, 85–92.
    [56]
    R. L. Levien. 2009. From Spiral to Spline: Optimal Techniques in Interactive Curve Design. Ph.D. Dissertation. EECS Department, University of California, Berkeley.
    [57]
    Michael Leyton. 1987. Symmetry-curvature duality. Comput. Vis. Graph. Image Process. 38, 3 (1987), 327–341.
    [58]
    Michael Leyton. 1988. A process-grammar for shape. Artif. Intell. 34, 2 (March 1988), 213–247.
    [59]
    Lei Luo, Chunhua Shen, Xinwang Liu, and Chunyuan Zhang. 2015. A computational model of the short-cut rule for 2D shape decomposition. IEEE Trans. Image Process. 24, 1 (2015), 273–283.
    [60]
    Diego Macrini, Sven Dickinson, David Fleet, and Kaleem Siddiqi. 2011. Bone graphs: Medial shape parsing and abstraction. Comput. Vis. Image Understand. 115, 7 (July 2011), 1044–1061.
    [61]
    Diego Macrini, Kaleem Siddiqi, and Sven Dickinson. 2008. From skeletons to bone graphs: Medial abstraction for object recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08).
    [62]
    Xiaofeng Mi and Doug DeCarlo. 2007. Separating parts from 2D shapes using relatability. In Proceedings of the IEEE 11th International Conference on Computer Vision (ICCV’07).
    [63]
    Martin Nöllenburg. 2014. A survey on automated metro map layout methods. In Proceedings of the 1st Schematic Mapping Workshop. University of Essex, UK.
    [64]
    Gerrit Noordzij. 2005. The Stroke—Theory of Writing. Hyphen Press. Translated from the Dutch original of 1985 by Peter Enneson.
    [65]
    Robert L. Ogniewicz and Markus Ilg. 1992. Voronoi skeletons: Theory and applications. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’92). 63–69.
    [66]
    Nikos Papanelopoulos, Yannis Avrithis, and Stefanos Kollias. 2019. Revisiting the medial axis for planar shape decomposition. Comput. Vis. Image Understand. 179 (2019), 66–78.
    [67]
    Pierre Parent and Steven W. Zucker. 1989. Trace inference, curvature consistency, and curve detection. IEEE Trans. Pattern Anal. Mach. Intell. 11, 8 (1989), 823–839.
    [68]
    Huy Quoc Phan, Hongbo Fu, and Antoni B Chan. 2015. Flexyfont: Learning transferring rules for flexible typeface synthesis. In Computer Graphics Forum, Vol. 34. 245–256.
    [69]
    R. Plamondon and S. N. Srihari. 2000. Online and off-line handwriting recognition: A comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1 (2000), 63–84.
    [70]
    Franco P. Preparata and Michael Ian Shamos. 1985. Intersections. 266–322.
    [71]
    Hock Soon Seah, Zhongke Wu, Feng Tian, Xian Xiao, and Boya Xie. 2005. Artistic brushstroke representation and animation with disk B-spline curve. In Proceedings of the ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. 88–93.
    [72]
    Doron Shaked and Alfred M Bruckstein. 1998. Pruning medial axes. Comput. Vis. Image Understand. 69, 2 (1998), 156–169.
    [73]
    Ariel Shamir. 2003. Constraint-based approach for automatic hinting of digital typefaces. ACM Trans. Graph. 22, 2 (2003), 131–151.
    [74]
    Ariel Shamir and Ari Rappoport. 1996. Extraction of typographic elements from outline representations of fonts. Comput. Graph. Forum 15, 3 (1996), 259–268.
    [75]
    Kaleem Siddiqi and Benjamin B. Kimia. 1995. Parts of visual form: Computational aspects. IEEE Trans. Pattern Anal. Mach. Intell. 17, 3 (1995), 239–251.
    [76]
    Manish Singh and Donald D. Hoffman. 2001. Part-based representations of visual shape and implications for visual cognition. In Advances in Psychology. Vol. 130. 401–459.
    [77]
    Manish Singh, Gregory D. Seyranian, and Donald D. Hoffman. 1999. Parsing silhouettes: The short-cut rule. Percept. Psychophys. 61, 4 (1999), 636–660.
    [78]
    Patrick Spröte, Filipp Schmidt, and Roland W. Fleming. 2016. Visual perception of shape altered by inferred causal history. Sci. Rep. 6, 36245 (2016).
    [79]
    Yuandong Sun, Huihuan Qian, and Yangsheng Xu. 2014. A geometric approach to stroke extraction for the chinese calligraphy robot. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’14). 3207–3212.
    [80]
    Rapee Suveeranont and Takeo Igarashi. 2010. Example-based automatic font generation. In Smart Graphics. Lecture Notes in Computer Science, Vol. 6133. 127–138.
    [81]
    Fan Tang, Weiming Dong, Yiping Meng, Xing Mei, Feiyue Huang, Xiaopeng Zhang, and Oliver Deussen. 2017. Animated construction of chinese brush paintings. IEEE Trans. Vis. Comput. Graph. 24, 12 (2017), 3019–3031.
    [82]
    S. P. Timoshenko and J. N. Goodier. 1951. Theory of Elasticity. McGraw–Hill.
    [83]
    Johan Wagemans. 2018. Perceptual organization. In Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience, Sensation, Perception, and Attention. Vol. 2. Chapter 18, 803–872. 4th Edition.
    [84]
    Johan Wagemans, Andrea J. van Doorn, and Jan J. Koenderink. 2011. Measuring 3D point configurations in pictorial space. i-Perception 2, 1 (2011), 77–111.
    [85]
    Jue Wang, Chenyu Wu, Ying-Qing Xu, Heung-Yeung Shum, and Liang Ji. 2002. Learning-based cursive handwriting synthesis. In Proceedings of the 8th IEEE International Workshop on Frontiers in Handwriting Recognition. 157–162.
    [86]
    Yue Wang. 2013. Interview with Charles Bigelow. TUGboat 34, 2 (2013), 136–167.
    [87]
    Carl-Fredrik Westin, Stephan E Maier, Hatsuho Mamata, Arya Nabavi, Ferenc A Jolesz, and Ron Kikinis. 2002. Processing and visualization for diffusion tensor MRI. Med. Image Anal. 6, 2 (2002), 93–108.
    [88]
    Lance Williams and Karvel K. Thornber. 2001. Orientation, scale, and discontinuity as emergent properties of illusory contour shape. Neural Comput. 13, 8 (August 2001), 1683–1711.
    [89]
    Songhua Xu, Hao Jiang, Francis C. M. Lau, and Yunhe Pan. 2012. Computationally evaluating and reproducing the beauty of chinese calligraphy. IEEE Intell. Syst.3 (2012), 63–72.
    [90]
    Yaoda Xu and Manish Singh. 2002. Early computation of part structure: Evidence from visual search. Percept. Psychophys. 64, 7 (2002), 1039–1054.
    [91]
    Shih Cheng Yen and Leif H. Finkel. 1998. Extraction of perceptually salient contours by striate cortical networks. Vis. Res. 38, 5 (1998), 719–741.
    [92]
    Junsong Zhang, Yu Wang, Weiyi Xiao, and Zhenshan Luo. 2017. Synthesizing ornamental typefaces. Comput. Graph. Forum 36, 1 (2017), 64–75.
    [93]
    Zhiyuan Zhao and Alan Saalfeld. 1997. Linear-time sleeve-fitting polyline simplification algorithms. In Proceedings of the 13th AutoCarto Symposium, Vol. 13. 214–223. https://cartogis.org
    [94]
    Changqing Zou, Junjie Cao, Warunika Ranaweera, Ibraheem Alhashim, Ping Tan, Alla Sheffer, and Hao Zhang. 2016. Legible compact calligrams. ACM Trans. Graph. 35, 4, Article 122 (2016), 12 pages. Article no. 122.

    Cited By

    View all
    • (2024)Internal Model Control Structure Inspired Robotic Calligraphy SystemIEEE Transactions on Industrial Informatics10.1109/TII.2023.329541520:2(2600-2610)Online publication date: Feb-2024
    • (2024)CalliRewrite: Recovering Handwriting Behaviors from Calligraphy Images without Supervision2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610332(8671-8678)Online publication date: 13-May-2024
    • (2024)A skinning technique for modeling artistic disk B-spline shapesComputers and Graphics10.1016/j.cag.2023.06.030115:C(96-106)Online publication date: 1-Feb-2024
    • Show More Cited By

    Index Terms

    1. StrokeStyles: Stroke-based Segmentation and Stylization of Fonts

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Graphics
        ACM Transactions on Graphics  Volume 41, Issue 3
        June 2022
        213 pages
        ISSN:0730-0301
        EISSN:1557-7368
        DOI:10.1145/3517033
        Issue’s Table of Contents

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 28 April 2022
        Accepted: 01 December 2021
        Revised: 01 September 2021
        Received: 01 May 2020
        Published in TOG Volume 41, Issue 3

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. Font structure
        2. stroke-based representations
        3. glyph stylization
        4. junction types
        5. curvilinear shape features
        6. augmented medial axis

        Qualifiers

        • Research-article
        • Refereed

        Funding Sources

        • UK’s EPSRC Centre for Doctoral Training in Intelligent Games and Game Intelligence

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)395
        • Downloads (Last 6 weeks)27
        Reflects downloads up to 10 Aug 2024

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Internal Model Control Structure Inspired Robotic Calligraphy SystemIEEE Transactions on Industrial Informatics10.1109/TII.2023.329541520:2(2600-2610)Online publication date: Feb-2024
        • (2024)CalliRewrite: Recovering Handwriting Behaviors from Calligraphy Images without Supervision2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610332(8671-8678)Online publication date: 13-May-2024
        • (2024)A skinning technique for modeling artistic disk B-spline shapesComputers and Graphics10.1016/j.cag.2023.06.030115:C(96-106)Online publication date: 1-Feb-2024
        • (2023)Word-As-Image for Semantic TypographyACM Transactions on Graphics10.1145/359212342:4(1-11)Online publication date: 26-Jul-2023
        • (2023)DS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00041(374-384)Online publication date: 1-Oct-2023

        View Options

        Get Access

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        Full Text

        HTML Format

        View this article in HTML Format.

        HTML Format

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media