research-article

Pixelated image abstraction with integrated user constraints

Authors:

Timothy Gerstner,

Adam Finkelstein,

Andrew NealenAuthors Info & Claims

Computers and Graphics, Volume 37, Issue 5

Pages 333 - 347

https://doi.org/10.1016/j.cag.2012.12.007

Published: 01 August 2013 Publication History

Abstract

We present an automatic method that can be used to abstract high resolution images into very low resolution outputs with reduced color palettes in the style of pixel art. Our method simultaneously solves for a mapping of features and a reduced palette needed to construct the output image. The results are an approximation to the results generated by pixel artists. We compare our method against the results of two naive methods common to image manipulation programs, as well as the hand-crafted work of pixel artists. Through a formal user study and interviews with expert pixel artists we show that our results offer an improvement over the naive methods. By integrating a set of manual controls into our algorithm, we give users the ability to add constraints and incorporate their own choices into the iterative process. Graphical abstractDisplay Omitted Highlights We propose an automated image processing method that approximates pixel art. We compare our method to naive methods and the work of pixel artists. Formal user study and interviews with experts demonstrate advantages of our method. Additional user controls allow flexible use of feedback to guide algorithm.

References

[1]

Vermehr K, Sauerteig S, Smital S. eboy. {http://hello.eboy.com}; 2012.

[2]

Marr, D. and Hildreth, E., Theory of edge detection. Proc R Soc London Ser B. v207. 187-217.

[3]

DeCarlo, D., Finkelstein, A., Rusinkiewicz, S. and Santella, A., Suggestive contours for conveying shape. ACM Trans Graph. v22 i3. 848-855.

Digital Library

[4]

Judd, T., Durand, F. and Adelson, E.H., Apparent ridges for line drawing. ACM Trans Graph. v26 i3. 19

Digital Library

[5]

Gooch B, Coombe G, Shirley P. Artistic vision: painterly rendering using computer vision techniques. In: Non-photorealistic animation and rendering (NPAR), 2002. p. 83-90, ISBN 1-58113-494-0. URL {http://doi.acm.org/10.1145/508530.508545}.

Digital Library

[6]

DeCarlo, D. and Santella, A., Stylization and abstraction of photographs. ACM Trans Graph. v21. 769-776,.

Digital Library

[7]

Winnemöller, H., Olsen, S.C. and Gooch, B., Real-time video abstraction. ACM Trans Graph. v25. 1221-1226.

Digital Library

[8]

Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels. Technical Report. IVRG CVLAB; 2010.

Digital Library

[9]

Rose, K., Deterministic annealing for clustering, compression, classification, regression, and related optimization problems. Proc IEEE. v86 i11. 2210-2239.

[10]

Sharma, G. and Trussell, H.J., Digital color imaging. IEEE Trans Image Process. v6. 901-932.

Digital Library

[11]

Gerstner T, DeCarlo D, Alexa M, Finkelstein A, Gingold Y, Nealen A. Pixelated image abstraction. In: Proceedings of the international symposium on non-photorealistic animation and rendering (NPAR), 2012.

Digital Library

[12]

Gervautz M, Purgathofer W. A simple method for color quantization: octree quantization. In: Graphics gems, 1990. p. 287-93, ISBN 0-12-286169-5.

Digital Library

[13]

Heckbert, P., Color image quantization for frame buffer display. SIGGRAPH Comput Graph. v16. 297-307.

Digital Library

[14]

Orchard, M. and Bouman, C., Color quantization of images. IEEE Trans Signal Process. v39. 2677-2690.

Digital Library

[15]

Color quantization by dynamic programming and principal analysis. ACM Trans Graph. v11. 348-372.

Digital Library

[16]

Stollnitz EJ, Ostromoukhov V, Salesin DH. Reproducing color images using custom inks. In: Proceedings of SIGGRAPH, 1998. p. 267-74, ISBN 0-89791-999-8.

Digital Library

[17]

Shi, J. and Malik, J., Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell. v22. 888-905.

Digital Library

[18]

Vedaldi A, Soatto S. Quick shift and kernel methods for mode seeking. In: European conference on computer vision, vol. IV, 2008. p. 705-18.

[19]

Levinshtein, A., Stere, A., Kutulakos, K.N., Fleet, D.J., Dickinson, S.J. and Siddiqi, K., Turbopixels: fast superpixels using geometric flows. IEEE Trans Pattern Anal Mach Intell. v31. 2290-2297.

Digital Library

[20]

Kirkpatrick, S., Gelatt, C.D. and Vecchi, M.P., Optimization by simulated annealing. Science. v220. 671-680.

[21]

Puzicha, J., Held, M., Ketterer, J., Buhmann, J.M. and Fellner, D.W., On spatial quantization of color images. IEEE Trans Image Process. v9. 666-682.

Digital Library

[22]

Inglis TC, Kaplan CS. Pixelating vector line art. In: Proceedings of the symposium on non-photorealistic animation and rendering. NPAR '12. Aire-la-Ville, Switzerland, Switzerland: Eurographics Association; 2012. p. 21-8, ISBN 978-3-905673-90-6.{http://dl.acm.org/citation.cfm?id=2330147.2330153}.

Digital Library

[23]

Kopf, J. and Lischinski, D., Depixelizing pixel art. ACM Trans Graph. v30 i4. 99

Digital Library

[24]

Xu J, Kaplan CS, Mi X. Computer-generated papercutting. In: Proceedings of pacific graphics, 2007. p. 343-50.

Digital Library

[25]

MacQueen JB. Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics and probability, 1967. p. 281-97.

[26]

Forsyth, D.A. and Ponce, J., Computer vision: a modern approach. . 2002. Prentice Hall.

Digital Library

Cited By

Binninger ASorkine-Hornung O(2024)SD-πXL: Generating Low-Resolution Quantized Imagery via Score DistillationSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687570(1-12)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687570
Sacht L(2024)Bottle cap art via clustering and optimal color assignmentsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03512-140:10(6929-6937)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00371-024-03512-1
Lei PXu SZhang S(2024)An art-oriented pixelation method for cartoon imagesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02763-040:1(27-39)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-022-02763-0
Show More Cited By

Recommendations

Pixelated image abstraction
NPAR '12: Proceedings of the Symposium on Non-Photorealistic Animation and Rendering

We present an automatic method that can be used to abstract high resolution images into very low resolution outputs with reduced color palettes in the style of pixel art. Our method simultaneously solves for a mapping of features and a reduced palette ...
A Novel Approach for Colorization of a Grayscale Image using Soft Computing Techniques

Colorization of grayscale image is a process to convert a grayscale image into a color one. Few research works reported in literature on this but there is hardly any generalized method that successfully colorizes all types of grayscale image. This study ...
Abstraction of photographs: a comics style approach
ECC'09: Proceedings of the 3rd international conference on European computing conference

We present an automatic image abstraction framework that abstracts photographs through a comics style approach. Our model extracts image structures and creates a depth map. Then it automatically adapts line drawing and colorization to stylize the image. ...

Comments

Information & Contributors

Information

Published In

cover image Computers and Graphics

Computers and Graphics Volume 37, Issue 5

August, 2013

265 pages

ISSN:0097-8493

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 August 2013

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Binninger ASorkine-Hornung O(2024)SD-πXL: Generating Low-Resolution Quantized Imagery via Score DistillationSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687570(1-12)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3680528.3687570
Sacht L(2024)Bottle cap art via clustering and optimal color assignmentsThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03512-140:10(6929-6937)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s00371-024-03512-1
Lei PXu SZhang S(2024)An art-oriented pixelation method for cartoon imagesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-022-02763-040:1(27-39)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-022-02763-0
Ancilotto APaissan FFarella E(2023)XimSwap: Many-to-Many Face Swapping for TinyMLACM Transactions on Embedded Computing Systems10.1145/360317323:3(1-16)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1145/3603173
Bhatnagar TUpadhyay VSharma ARao PMiodownik MMarquardt NHolloway C(2023)Pixelated Interactions: Exploring Pixel Art for Graphical Primitives on a Pin Array Tactile DisplayProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596044(1194-1208)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596044
Wu ZChai LZhao NDeng BLiu YWen QWang JHe S(2022)Make Your Own SpritesACM Transactions on Graphics10.1145/3550454.355548241:6(1-16)Online publication date: 30-Nov-2022
https://dl.acm.org/doi/10.1145/3550454.3555482
Sacht L(2022)Structure-aware bottle cap artComputers and Graphics10.1016/j.cag.2022.08.004107:C(277-288)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.cag.2022.08.004
Dall’Asen NWang YTang HZanella LRicci E(2022)Graph-Based Generative Face Anonymisation with Pose PreservationImage Analysis and Processing – ICIAP 202210.1007/978-3-031-06430-2_42(503-515)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1007/978-3-031-06430-2_42
Pérez-Delgado MRomán Gallego J(2020)A two-stage method to improve the quality of quantized imagesJournal of Real-Time Image Processing10.1007/s11554-018-0814-817:3(581-605)Online publication date: 1-Jun-2020
https://dl.acm.org/doi/10.1007/s11554-018-0814-8
Dodgson NWyvill BFu H(2018)Abstract depiction of human and animal figuresProceedings of the Joint Symposium on Computational Aesthetics and Sketch-Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering10.1145/3229147.3229152(1-8)Online publication date: 17-Aug-2018
https://dl.acm.org/doi/10.1145/3229147.3229152
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents