article

Stylization and abstraction of photographs

Authors:

Anthony SantellaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 21, Issue 3

Pages 769 - 776

https://doi.org/10.1145/566654.566650

Published: 01 July 2002 Publication History

Abstract

Good information design depends on clarifying the meaningful structure in an image. We describe a computational approach to stylizing and abstracting photographs that explicitly responds to this design goal. Our system transforms images into a line-drawing style using bold edges and large regions of constant color. To do this, it represents images as a hierarchical structure of parts and boundaries computed using state-of-the-art computer vision. Our system identifies the meaningful elements of this structure using a model of human perception and a record of a user's eye movements in looking at the photo; the system renders a new image using transformations that preserve and highlight these visual elements. Our method thus represents a new alternative for non-photorealistic rendering both in its visual style, in its approach to visual form, and in its techniques for interaction.

References

[1]

AGRAWALA, M., AND STOLTE, C. 2001. Rendering effective route maps: improving usability through generalization. In Proc. of ACM SIGGRAPH 2001, 241-249.

[2]

AHUJA, N. 1996. A transform for multiscale image segmentation by integrated edge and region detection. IEEE Trans. on Pattern Analysis and Machine Intelligence 18, 12, 1211-1235.

[3]

BURT, P., AND ADELSON, E. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. on Communications 31, 4, 532-540.

[4]

CAMPBELL, F., AND ROBSON, J. 1968. Application of Fourier analysis to the visibility of gratings. Journal of Physiology 197, 551-566.

[5]

CHRISTOUDIAS, C., GEORGESCU, B., AND MEER, P. 2002. Synergism in low level vision. In Proc. ICPR 2002.

[6]

COMANICIU, D., AND MEER, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. on Pattern Analysis and Machine Intelligence 24, 5.

[7]

CURTIS, C. 1999. Non-photorealistic animation. In ACM SIGGRAPH 1999 Course Notes #17 (Section 9).

[8]

DEUSSEN, O., AND STROTHOTTE, T. 2000. Computer-generated pen-and-ink illustration of trees. In Proc. of ACM SIGGRAPH 2000, 13-18.

[9]

DUCHOWSKI, A., AND VERTEGAAL, R. 2000. Eye-based interaction in graphical systems: Theory and practice. In ACM SIGGRAPH 2000 Course Notes #5.

[10]

DUCHOWSKI, A. 2000. Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. on Image Processing 9, 8 (Aug.), 1437-1440.

[11]

DURAND, F., OSTROMOUKHOV, V., MILLER, M., DURANLEAU, F., AND DORSEY, J. 2001. Decoupling strokes and high-level attributes for interactive traditional drawing. In Proceedings of the 12th Eurographics Workshop on Rendering, 71-82.

[12]

FINKELSTEIN, A., AND SALESIN, D. 1994. Multiresolution curves. In Proc. of ACM SIGGRAPH 94, 261-268.

[13]

FOLEY, J., VAN DAM, A., FEINER, S., AND HUGHES, J. 1997. Computer Graphics: Principles and Practice, 2nd edition. Addison Wesley.

[14]

GOOCH, B., AND GOOCH, A. 2001. Non-Photorealistic Rendering. A K Peters.

[15]

GOOCH, A. A., GOOCH, B., SHIRLEY, P., AND COHEN, E. 1998. A non-photorealistic lighting model for automatic technical illustration. In Proc. of ACM SIGGRAPH 98, 447-452.

[16]

HAEBERLI, P. 1990. Paint by numbers: Abstract image representations. In Proc. of ACM SIGGRAPH 90, 207-214.

[17]

HANDFORD, M. 1987. Where's Waldo? Little, Brown and Company.

[18]

HENDERSON, J. M., AND HOLLINGWORTH, A. 1998. Eye movements during scene viewing: An overview. In Eye Guidance in Reading and Scene Perception, G. Underwood, Ed. Elsevier Science Ltd., 269-293.

[19]

HERMAN, I., AND DUKE, D. 2001. Minimal graphics. IEEE Computer Graphics and Applications 21, 6, 18-21.

[20]

HERTZMANN, A., AND ZORIN, D. 2000. Illustrating smooth surfaces. In Proc. of ACM SIGGRAPH 2000, 517-526.

[21]

HERTZMANN, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proc. of ACM SIGGRAPH 98, 453-460.

[22]

HERTZMANN, A. 2001. Paint by relaxation. In Computer Graphics International, 47-54.

[23]

HOFFMAN, D. D. 1998. Visual intelligence: how we create what we see. Norton.

[24]

JUST, M. A., AND CARPENTER, P. A. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 441-480.

[25]

KELLY, D. 1984. Retinal inhomogenity: I. spatiotemporal contrast sensitivity. Journal of the Optical Society of America A 74, 1, 107-113.

[26]

KOENDERINK, J. J., M. A. BOUMAN, A. B. D. M., AND SLAPPENDEL, S. 1978. Perimetry of contrast detection thresholds of moving spatial sine wave patterns. II. the far peripheral visual field (eccentricity 0-50). Journal of the Optical Society of America A 68, 6, 850-854.

[27]

KOENDERINK, J. J. 1984. The structure of images. Biological Cybernetics 50, 363-370.

[28]

KOENDERINK, J. J. 1984. What does the occluding contour tell us about solid shape? Perception 13, 321-330.

[29]

KOWALSKI, M. A., MARKOSIAN, L., NORTHRUP, J. D., BOURDEV, L., BARZEL, R., HOLDEN, L. S., AND HUGHES, J. 1999. Art-based rendering of fur, grass, and trees. In Proc. of ACM SIGGRAPH 99, 433-438.

[30]

LEYTON, M. 1992. Symmetry, causality, mind. MIT Press.

[31]

LINDEBERG, T. 1994. Scale-Space Theory in Computer Vision. Kluwer Academic Publishers.

[32]

LITWINOWICZ, P. 1997. Processing images and video for an impressionist effect. In Proc. of ACM SIGGRAPH 97, 407-414.

[33]

MACKWORTH, N., AND MORANDI, A. 1967. The gaze selects informative details within pictures. Perception and Psychophysics 2, 547-552.

[34]

MANNOS, J. L., AND SAKRISON, D. J. 1974. The effects of a visual fidelity criterion on the encoding of images. IEEE Trans. on Information Theory 20, 4, 525-536.

[35]

MARKOSIAN, L., KOWALSKI, M. A., TRYCHIN, S. J., BOURDEV, L. D., GOLDSTEIN, D., AND HUGHES, J. F. 1997. Real-time nonphotorealistic rendering. In Proc. of ACM SIGGRAPH 97, 415-420.

[36]

MARR, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W. H. Freeman, San Francisco.

[37]

MEER, P., AND GEORGESCU, B. 2001. Edge detection with embedded confidence. IEEE Trans. on Pattern Analysis and Machine Intelligence 23, 12, 1351-1365.

[38]

PATTANAIK, S. N., FERWERDA, J. A., FAIRCHILD, M. D., AND GREENBERG, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. of ACM SIGGRAPH 98, 287-298.

[39]

REDDY, M. 2001. Perceptually optimized 3D graphics. IEEE Computer Graphics and Applications 21, 5 (September/October), 68-75.

[40]

REGAN, D. 2000. Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance, Color, Texture, Motion and Binocular Disparity. Sinauer.

[41]

ROVAMO, J., AND VIRSU, V. 1979. An estimation and application of the human cortical magnification factor. Experimental Brain Research 37, 495-510.

[42]

SAITO, T., AND TAKAHASHI, T. 1990. Comprehensible rendering of 3-D shapes. In Proc. of ACM SlGGRAPH 90, 197-206.

[43]

SANTELLA, A., AND DECARLO, D. 2002. Abstracted painterly renderings using eye-tracking data. In Proc. of the Second International Symp. on Non-photorealistic Animation and Rendering (NPAR).

[44]

SHIRAISHI, M., AND YAMAGUCHI, Y. 2000. An algorithm for automatic painterly rendering based on local source image approximation. In Proc. of the First International Symp. on Non-photorealistic Animation and Rendering (NPAR), 53-58.

[45]

SIBERT, L. E., AND JACOB, R. J. K. 2000. Evaluation of eye gaze interaction. In Proc. CHI 2000, 281-288.

[46]

TRUCCO, E., AND VERRI, A. 1998. Introductory Techniques for 3-D Computer Vision. Prentice-Hall.

[47]

TUFTE, E. R. 1990. Envisioning Information. Graphics Press.

[48]

VERTEGAAL, R. 1999. The gaze groupware system: Mediating joint attention in mutiparty communication and collaboration. In Proc. CHI '99, 294-301.

[49]

WINKENBACH, G., AND SALESIN, D. H. 1994. Computer-generated pen-and-ink illustration. In Proc. of ACM SIGGRAPH 94, 91-100.

[50]

YARBUS, A. L. 1967. Eye Movements and Vision. Plenum Press.

[51]

ZEKI, S. 1999. Inner Vision: An Exploration of Art and the Brain. Oxford Univ. Press.

Cited By

Tojo KShamir ABickel BUmetani N(2024)Fabricable 3D Wire ArtACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657453(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657453
San TKanematsu YMikami K(2023)Algorithmic Analysis of Color Combinations Principle in Game Concept ArtThe Journal of the Society for Art and Science10.3756/artsci.22.15_122:4(15_1-15_7)Online publication date: 2023
https://doi.org/10.3756/artsci.22.15_1
Cole FGolovinskiy ALimpaecher AStoddart Barros HFinkelstein AFunkhouser TRusinkiewicz S(2023)Where Do People Draw Lines?Seminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596756(409-419)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1145/3596711.3596756
Show More Cited By

Index Terms

Stylization and abstraction of photographs
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Hierarchical representations
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Stylization and abstraction of photographs
SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques

Good information design depends on clarifying the meaningful structure in an image. We describe a computational approach to stylizing and abstracting photographs that explicitly responds to this design goal. Our system transforms images into a line-...
Hardware-accelerated parallel non-photorealistic volume rendering
NPAR '02: Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering

Non-photorealistic rendering can be used to illustrate subtle spatial relationships that might not be visible with more realistic rendering techniques. We present a parallel hardware-accelerated rendering technique, making extensive use of multi-...
Abstraction of photographs: a comics style approach
ECC'09: Proceedings of the 3rd international conference on European computing conference

We present an automatic image abstraction framework that abstracts photographs through a comics style approach. Our model extracts image structures and creates a depth map. Then it automatically adapts line drawing and colorization to stylize the image. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 21, Issue 3

July 2002

548 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/566654

Issue’s Table of Contents

Copyright © 2002 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2002

Published in TOG Volume 21, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

215
Total Citations
View Citations
3,770
Total Downloads

Downloads (Last 12 months)90
Downloads (Last 6 weeks)11

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tojo KShamir ABickel BUmetani N(2024)Fabricable 3D Wire ArtACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657453(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657453
San TKanematsu YMikami K(2023)Algorithmic Analysis of Color Combinations Principle in Game Concept ArtThe Journal of the Society for Art and Science10.3756/artsci.22.15_122:4(15_1-15_7)Online publication date: 2023
https://doi.org/10.3756/artsci.22.15_1
Cole FGolovinskiy ALimpaecher AStoddart Barros HFinkelstein AFunkhouser TRusinkiewicz S(2023)Where Do People Draw Lines?Seminal Graphics Papers: Pushing the Boundaries, Volume 210.1145/3596711.3596756(409-419)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1145/3596711.3596756
Hsiao KYang YChiu YHu MYao CChu H(2023)Img2Logo: Generating Golden Ratio Logos from ImagesComputer Graphics Forum10.1111/cgf.1474242:2(37-49)Online publication date: 23-May-2023
https://doi.org/10.1111/cgf.14742
Chang YSaito S(2023)A Method for Determining the Subjective Dominant Color of an Image Region by Support Vector Regression2023 Nicograph International (NicoInt)10.1109/NICOINT59725.2023.00010(1-6)Online publication date: Jun-2023
https://doi.org/10.1109/NICOINT59725.2023.00010
Yang YWu DTang LZeng LPan Z(2023)Weighted and truncated $$L_1$$ image smoothing based on unsupervised learningThe Visual Computer10.1007/s00371-023-03141-040:8(5871-5882)Online publication date: 20-Nov-2023
https://doi.org/10.1007/s00371-023-03141-0
Tojo KUmetani N(2022)Recolorable Posterization of Volumetric Radiance Fields Using Visibility‐Weighted Palette ExtractionComputer Graphics Forum10.1111/cgf.1459441:4(149-160)Online publication date: 30-Jul-2022
https://doi.org/10.1111/cgf.14594
Chung MKwon T(2022)Fast Text Placement Scheme for ASCII Art SynthesisIEEE Access10.1109/ACCESS.2022.316756710(40677-40686)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3167567
Sacht L(2022)Structure-aware bottle cap artComputers and Graphics10.1016/j.cag.2022.08.004107:C(277-288)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1016/j.cag.2022.08.004
Liu SLiu Z(2022)Line drawing via saliency map and ETFFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-021-1027-z16:5Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s11704-021-1027-z
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents