Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

TCB-spline-based Image Vectorization

Published: 14 June 2022 Publication History

Abstract

Vector image representation methods that can faithfully reconstruct objects and color variations in a raster image are desired in many practical applications. This article presents triangular configuration B-spline (referred to as TCB-spline)-based vector graphics for raster image vectorization. Based on this new representation, an automatic raster image vectorization paradigm is proposed. The proposed framework first detects sharp curvilinear features in the image and constructs knot meshes based on the detected feature lines. It iteratively optimizes color and position of control points and updates the knot meshes. By using collinear knots at feature lines, both smooth and discontinuous color variations can be efficiently modeled by the same set of quadratic TCB-splines. A variational knot mesh generation method is designed to adaptively introduce knots and update their connectivity to satisfy the local reconstruction quality. Experiments and comparisons show that our framework outperforms the existing state-of-the-art methods in providing more faithful reconstruction results. In particular, our method is able to model undetected features and subtle or complicated color variations in-between features, which the previous methods cannot handle efficiently. Our vectorization representation also facilitates a variety of editing operations performed directly over vector images.

Supplementary Material

zhu (zhu.zip)
Supplemental movie, appendix, image and software files for, TCB-spline-based Image Vectorization

References

[1]
Curtis A. Armstrong. 2006. Vectorization of Raster Images Using B-Spline Surfaces. Master’s thesis. Brigham Young University-Provo.
[2]
Mikhail Bessmeltsev and Justin Solomon. 2019. Vectorization of line drawings via polyvector fields. ACM Trans. Graph. 38, 1, Article 9 (Jan. 2019), 12 pages.
[3]
John Canny. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6 (1986), 679–698.
[4]
Juan Cao, Zhonggui Chen, Xiaodong Wei, and Yongjie Jessica Zhang. 2019. A finite element framework based on bivariate simplex splines on triangle configurations. Comput. Methods Appl. Mech. Eng. 357 (2019), 112598.
[5]
Juan Cao, Yanyang Xiao, Zhonggui Chen, Wenping Wang, and Chandrajit Bajaj. 2018. Functional data approximation on bounded domains using polygonal finite elements. Comput.-aided Geom. Design 63 (2018), 149–163.
[6]
Kuowei Chen, Yingsheng Luo, Yuchi Lai, Yanlin Chen, Chihyuan Yao, Hungkuo Chu, and Tongyee Lee. 2020. Image vectorization with real-time thin-plate spline. IEEE Trans. Multimedia 22, 1 (2020), 15–29.
[7]
Zhonggui Chen, Jinxin Huang, Juan Cao, and Yongjie Jessica Zhang. 2019. Interpolatory curve modeling with feature points control. Comput.-aided Design 114 (2019), 155–163.
[8]
Zhonggui Chen, Yanyang Xiao, and Juan Cao. 2014. Approximation by piecewise polynomials on Voronoi tessellation. Graph. Models 76, 5 (2014), 522–531.
[9]
Carl De Boor. 1976. Splines as linear combinations of B-splines. A survey. Technical Report No. MRC-TSR-1667. Wisconsin University-Madison Mathematics Research Center.
[10]
Edoardo Alberto Dominici, Nico Schertler, Jonathan Griffin, Shayan Hoshyari, Leonid Sigal, and Alla Sheffer. 2020. PolyFit: Perception-Aligned vectorization of raster clip-art via intermediate polygonal fitting. ACM Trans. Graph. 39, 4, Article 77 (July 2020), 16 pages.
[11]
Dov Dori and Wenyin Liu. 1999. Sparse pixel vectorization: An algorithm and its performance evaluation. IEEE Trans. Pattern Anal. Mach. Intell. 21, 3 (1999), 202–215.
[12]
David H. Douglas and Thomas K. Peucker. 1973. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: Int. J. Geogr. Info. Geovisual. 10, 2 (1973), 112–122.
[13]
Nira Dyn, David Levin, and Samuel Rippa. 1990. Data dependent triangulations for piecewise linear interpolation. IMA J. Numer. Anal. 10, 1 (1990), 137–154.
[14]
Mark Finch, John Snyder, and Hugues Hoppe. 2011. Freeform vector graphics with controlled thin-plate splines. ACM Trans. Graph. 30, 6 (Dec. 2011), 1–0.
[15]
Harley Flanders. 1973. Differentiation under the integral sign. Amer. Math. Month. 80, 6 (1973), 615–627.
[16]
Michael Franssen. 1995. Evaluation of DMS-splines. Master’s thesis. Eindhoven University of Technology.
[17]
Akemi Gálvez and Andrés Iglesias. 2012. Particle swarm optimization for non-uniform rational B-spline surface reconstruction from clouds of 3D data points. Info. Sci. 192 (June 2012), 174–192.
[18]
Yi Guo, Zhuming Zhang, Chu Han, Wenbo Hu, Chengze Li, and Tien-Tsin Wong. 2019. Deep line drawing vectorization via line subdivision and topology reconstruction. Comput. Graph. Forum 38, 7 (2019), 81–90.
[19]
Xavier Hilaire and Karl Tombre. 2006. Robust and accurate vectorization of line drawings. IEEE Trans. Pattern Anal. Mach. Intell. 28, 6 (June 2006), 890–904.
[20]
Kai Hormann and Michael S. Floater. 2006. Mean value coordinates for arbitrary planar polygons. ACM Trans. Graph. 25, 4 (Oct. 2006), 1424–441.
[21]
Shayan Hoshyari, Edoardo Alberto Dominici, Alla Sheffer, Nathan Carr, Zhaowen Wang, Duygu Ceylan, and I.-Chao Shen. 2018. Perception-Driven semi-structured boundary vectorization. ACM Trans. Graph. 37, 4, Article 118 (July 2018), 14 pages.
[22]
Fei Hou, Qian Sun, Zheng Fang, Yong-Jin Liu, Shi-Min Hu, Hong Qin, Aimin Hao, and Ying He. 2020. Poisson vector graphics (PVG). IEEE Trans. Visual. Comput. Graph. 26, 2 (2020), 1361–1371.
[23]
Yixin Hu, Teseo Schneider, Xifeng Gao, Qingnan Zhou, Alec Jacobson, Denis Zorin, and Daniele Panozzo. 2019. TriWild: Robust triangulation with curve constraints. ACM Trans. Graph. 38, 4, Article 52 (July 2019), 15 pages.
[24]
Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. 2011. Bounded biharmonic weights for real-time deformation. ACM Trans. Graph. (Proceedings of ACM SIGGRAPH) 30, 4 (2011), 78:1–78:8.
[25]
Stefan Jeschke, David Cline, and Peter Wonka. 2009. A GPU Laplacian solver for diffusion curves and Poisson image editing. ACM Trans. Graph. 28, 5, Article 116 (Dec. 2009), 8 pages.
[26]
Yue Jia, Yongjie Zhang, Gang Xu, Xiaoying Zhuang, and Timon Rabczuk. 2013. Reproducing kernel triangular B-spline-based FEM for solving PDEs. Comput. Methods Appl. Mech. Eng. 267 (2013), 342–358.
[27]
Ruchin Kansal and Subodh Kumar. 2015. A vectorization framework for constant and linear gradient filled regions. Visual Comput. 31, 5 (May 2015), 717–732.
[28]
Oliver Kreylos and Bernd Hamann. 2001. On simulated annealing and the construction of linear spline approximations for scattered data. IEEE Trans. Visual. Comput. Graph. 7, 1 (Jan 2001), 17–31.
[29]
Kuo-Chin Fan, Den-Fong Chen, and Ming-Gang Wen. 1995. A new vectorization-based approach to the skeletonization of binary images. In Proceedings of 3rd International Conference on Document Analysis and Recognition, Vol. 2. 627–630.
[30]
Yu-Kun Lai, Shi-Min Hu, and Ralph R. Martin. 2009. Automatic and topology-preserving gradient mesh generation for image vectorization. ACM Trans. Graph. 28, 3, Article 85 (July 2009), 8 pages.
[31]
Gregory Lecot and Bruno Lévy. 2006. Ardeco: Automatic region detection and conversion. In Proceedings of the 17th Eurographics Symposium on Rendering (EGSR’06). Nicosia/Cyprus, 349–360.
[32]
Zicheng Liao, Hugues Hoppe, David Forsyth, and Yizhou Yu. 2012. A subdivision-based representation for vector image editing. IEEE Trans. Visual. Comput. Graph. 18, 11 (Nov 2012), 1858–1867.
[33]
Yuanxin Liu. 2007. Computations of Delaunay and higher order triangulations, with applications to splines. Ph.D. Dissertation. University of North Carolina at Chapel Hill.
[34]
Yuanxin Liu and Jack Snoeyink. 2007. Quadratic and cubic B-splines by generalizing higher-order Voronoi diagrams. In Proceedings of the Symposium on Computational Geometry. 150–157.
[35]
Marian Neamtu. 2001. What is the natural generalization of univariate splines to higher dimensions? In Mathematical Methods for Curves and Surfaces. Vanderbilt University, Nashville, TN, 355–392.
[36]
Alexandrina Orzan, Adrien Bousseau, Holger Winnemöller, Pascal Barla, Joëlle Thollot, and David Salesin. 2008. Diffusion curves: A vector representation for smooth-shaded images. ACM Trans. Graph. 27, 3, Article 92 (Aug. 2008), 8 pages.
[37]
Pradyumna Reddy, Michael Gharbi, Michal Lukac, and Niloy J. Mitra. 2021. Im2Vec: Synthesizing Vector Graphics without Vector Supervision. Retrieved from https://arxiv:cs.CV/2102.02798.
[38]
ScanFont. 2017. Font Lab. Retrieved from http://old.fontlab.com/font-converter/scanfont//.
[39]
Dominique Schmitt. 2019. Bivariate B-splines from convex pseudo-circle configurations. In Fundamentals of Computation Theory, Leszek Antoni Gąsieniec, Jesper Jansson, and Christos Levcopoulos (Eds.). Springer International Publishing, Cham, 335–349.
[40]
Olga Sorkine, Daniel Cohen-Or, Yaron Lipman, Marc Alexa, Christian Rössl, and Hans-Peter Seidel. 2004. Laplacian surface editing. In Proceedings of the Eurographics/ACM SIGGRAPH Symposium on Geometry Processing. 175–184.
[41]
Dan Su and Philip Willis. 2004. Image interpolation by pixel-level data-dependent triangulation. Comput. Graph. Forum 23, 2 (2004), 189–201.
[42]
Jian Sun, Lin Liang, Fang Wen, and Heung-Yeung Shum. 2007. Image vectorization using optimized gradient meshes. ACM Trans. Graph. 26, 3, Article 11 (July 2007).
[43]
Timothy Sun, Papoj Thamjaroenporn, and Changxi Zheng. 2014. Fast multipole representation of diffusion curves and points. ACM Trans. Graph. 33, 4, Article 53 (July 2014), 12 pages.
[44]
Xin Sun, Guofu Xie, Yue Dong, Stephen Lin, Weiwei Xu, Wencheng Wang, Xin Tong, and Baining Guo. 2012. Diffusion curve textures for resolution independent texture mapping. ACM Trans. Graph. 31, 4, Article 74 (July 2012), 9 pages.
[45]
Cihan Topal and Cuneyt Akinlar. 2012. Edge drawing: A combined real-time edge and segment detector. J. Visual Commun. Image Represent. 23, 6 (2012), 862–872.
[46]
William Thomas Tutte. 1963. How to draw a graph. Proc. London Math. Soc. s3-13, 1 (1963), 743–767.
[47]
Tian Xia, Binbin Liao, and Yizhou Yu. 2009. Patch-based image vectorization with automatic curvilinear feature alignment. ACM Trans. Graph. 28, 5, Article 115 (Dec. 2009), 115:1–115:10 pages.
[48]
Guofu Xie, Xin Sun, Xin Tong, and Derek Nowrouzezahrai. 2014. Hierarchical diffusion curves for accurate automatic image vectorization. ACM Trans. Graph. 33, 6, Article 230 (Nov. 2014), 11 pages.
[49]
Weicheng Xie, Xiufen Zou, Jiandong Yang, and Jiebin Yang. 2012. Iteration and optimization scheme for the reconstruction of 3D surfaces based on non-uniform rational B-splines. Comput.-aided Design 44, 11 (Nov. 2012), 1127–1140.
[50]
Zhipei Yan, Stephen Schiller, Gregg Wilensky, Nathan Carr, and Scott Schaefer. 2017. K-Curves: Interpolation at local maximum curvature. ACM Trans. Graph. 36, 4, Article 129 (July 2017), 7 pages.
[51]
Fujiichi Yoshimoto, Toshinobu Harada, and Yoshihide Yoshimoto. 2003. Data fitting with a spline using a real-coded genetic algorithm. Comput.-aided Design 35, 8 (2003), 751–760.
[52]
Yuhua Zhang, Juan Cao, Zhonggui Chen, and Xiaoming Zeng. 2017. Surface reconstruction using simplex splines on feature-sensitive configurations. Comput.-aided Geom. Design 50 (2017), 14–28.
[53]
Shuang Zhao, Fredo Durand, and Changxi Zheng. 2018. Inverse diffusion curves using shape optimization. IEEE Trans. Visual. Comput. Graph. 24, 7 (2018), 2153–2166.
[54]
Hailing Zhou, Jianmin Zheng, and Lei Wei. 2014. Representing images using curvilinear feature driven subdivision surfaces. IEEE Trans. Image Process. 23, 8 (Aug 2014), 3268–3280.

Cited By

View all
  • (2024)Optimize & reduceProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i3.27987(2148-2156)Online publication date: 20-Feb-2024
  • (2024)Deep Sketch Vectorization via Implicit Surface ExtractionACM Transactions on Graphics10.1145/365819743:4(1-13)Online publication date: 19-Jul-2024
  • (2024)Intelligent pattern reconstruction based on primitive spatial relationship adaptive transformationFifteenth International Conference on Graphics and Image Processing (ICGIP 2023)10.1117/12.3021534(92)Online publication date: 25-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 41, Issue 3
June 2022
213 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3517033
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2022
Online AM: 12 May 2022
Accepted: 01 January 2022
Revised: 01 January 2022
Received: 01 April 2021
Published in TOG Volume 41, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Vector images
  2. simplex splines
  3. knot placement
  4. mesh optimization

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • NSFC
  • Xiamen Youth Innovation Funds
  • Open Project Program of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University
  • NSF
  • Honda

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)364
  • Downloads (Last 6 weeks)16
Reflects downloads up to 03 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Optimize & reduceProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i3.27987(2148-2156)Online publication date: 20-Feb-2024
  • (2024)Deep Sketch Vectorization via Implicit Surface ExtractionACM Transactions on Graphics10.1145/365819743:4(1-13)Online publication date: 19-Jul-2024
  • (2024)Intelligent pattern reconstruction based on primitive spatial relationship adaptive transformationFifteenth International Conference on Graphics and Image Processing (ICGIP 2023)10.1117/12.3021534(92)Online publication date: 25-Mar-2024
  • (2024)Local color harmony adjustment via color envelope transformationFifteenth International Conference on Graphics and Image Processing (ICGIP 2023)10.1117/12.3021216(57)Online publication date: 25-Mar-2024
  • (2024)Curved Image Triangulation Based on Differentiable RenderingComputer Graphics Forum10.1111/cgf.1523243:7Online publication date: 24-Oct-2024
  • (2024)A Novel Architecture for Image Vectorization with Increasing Granularity2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647480(1560-1566)Online publication date: 27-Oct-2024
  • (2024)Segmentation-Guided Layer-Wise Image Vectorization with Gradient FillsComputer Vision – ECCV 202410.1007/978-3-031-72684-2_10(165-180)Online publication date: 29-Sep-2024
  • (2024)Vectorized Colorization of Icon Line Art Based on Closed Contour ExtractionQuality, Reliability, Security and Robustness in Heterogeneous Systems10.1007/978-3-031-65123-6_4(38-53)Online publication date: 20-Aug-2024
  • (2023)Error‐bounded Image TriangulationComputer Graphics Forum10.1111/cgf.1496742:7Online publication date: 31-Oct-2023
  • (2023)DualVector: Unsupervised Vector Font Synthesis with Dual-Part Representation2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01364(14193-14202)Online publication date: Jun-2023
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media