Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

A natural representation of colors with textures

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present an automated method to transform an image to a set of binary textures that represent the intensities and colors of a full color original image. The foundation of our method is information preservation: creating a set of textures that allows for the reconstruction of the original image’s colors solely from the binarized representation. We present techniques to ensure that the textures created are not visually distracting, preserve the intensity profile of the images, and are natural in that they map sets of colors that are perceptually similar to patterns that are similar. The textures instantiated are fully reversible; the original images’ colors can be recreated from the single-bit per-pixel images. The approach uses deep-neural networks and is entirely self-supervised. The system yields aesthetically pleasing binary images when tested on a variety of image sources, including color and black and white photographs, clip art, and paintings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Bai, Y., Harrington, S.J., Taber, J.: Improved algorithmic mapping of color to texture. In: Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts VI, vol. 4300, pp. 444–451 (2000)

  2. Baluja, S.: Hiding images within images. IEEE Trans. Pattern Anal. Mach. Intell. 42(7), 1685–1697 (2019)

    Article  Google Scholar 

  3. Bernsen, J.: Dynamic thresholding of gray-level images. In: ICPR (1986)

  4. Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)

  5. Calvo-Zaragoza, J., Sánchez, A.: A selectional auto-encoder approach for document image binarization. CoRR arXiv:abs/1706.10241 (2017)

  6. Chaki, N., Shaikh, S.H., Saeed, K.: A comprehensive survey on image binarization techniques. In: Exploring Image Binarization Techniques, pp. 5–15. Springer (2014)

  7. Chang, C.I., Du, Y., Wang, J., Guo, S.M., Thouin, P.: Survey and comparative analysis of entropy and relative entropy thresholding techniques. IEE Proc. Vis. Image Signal Process. 153(6), 837–850 (2006)

    Article  Google Scholar 

  8. Chang, J., Alain, B., Ostromoukhov, V.: Structure-aware error diffusion. In: ACM SIGGRAPH Asia 2009, (SIGGRAPH Asia ’09), New York, NY, USA (2009)

  9. Cricut: Cricut (2020). https://cricut.com/. Accessed 10/7/2020

  10. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR 2009, pp. 248–255. IEEE (2009)

  11. Doyle, W.: Operations useful for similarity-invariant pattern recognition. J. ACM 9(2), 259–267 (1962)

    Article  Google Scholar 

  12. Fisher, R., Perkins, S., Walker, A., Wolfart, E.: Image Processing Learning Resources: Adaptive Tresholding (2017). http://homepages.inf.ed.ac.uk/rbf/HIPR2/adpthrsh.htm. Accessed 2020-10-5

  13. Floyd, R.W., Steinberg, L.: An adaptive algorithm for spatial greyscale. Proc. Soc. Inf. Display 17(2), 75–77 (1976)

    Google Scholar 

  14. Funkhouser, T.: Image Quantization, Halftoning, and Dithering (2008). https://www.cs.princeton.edu/courses/archive/fall00/cs426/lectures/dither/dither.pdf. Accessed 5 Oct 2020

  15. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: CVPR, pp. 2414–2423 (2016)

  16. Glasbey, C.A.: An analysis of histogram-based thresholding algorithms. CVGIP Graph. Models Image Process. 55(6), 532–537 (1993)

    Article  Google Scholar 

  17. Glowforge: Glowforge. https://glowforge.com/ (2020). Accessed 10/7/2020

  18. Gooch, A.A., Olsen, S.C., Tumblin, J., Gooch, B.: Color2gray: salience-preserving color removal. ACM Trans. Graph. 24(3), 634–639 (2005)

    Article  Google Scholar 

  19. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks (2014). arXiv:1406.2661

  20. Harrington, S.J.: Algorithmic mapping of colors to textures. In: Color Hard Copy and Graphic Arts III, vol. 2171, pp. 305 – 312. SPIE (1994)

  21. Helland, T.: Image Dithering: Eleven Algorithms and Source Code (2012). https://tannerhelland.com/2012/12/28/dithering-eleven-algorithms-source-code.html. Accessed 20 Oct 2020

  22. Hinton, G., Srivastava, N., Swersky, K.: Neural networks for machine learning. Coursera Video Lect. 264, 66 (2012)

    Google Scholar 

  23. Hu, D., Wang, L., Jiang, W., Zheng, S., Li, B.: A novel image steganography method via deep convolutional generative adversarial networks. IEEE Access 6, 66 (2018)

    Google Scholar 

  24. Huang, L.K., Wang, M.J.J.: Image thresholding by minimizing the measures of fuzziness. Pattern Recognit. 28(1), 41–51 (1995)

    Article  Google Scholar 

  25. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the CVPR, pp. 1125–1134 (2017)

  26. Isumi, R., Yamamoto, K., Noma, T.: Color2hatch: conversion of color to hatching for low-cost printing. Vis. Comput. 37(12), 3103–3113 (2021)

    Article  Google Scholar 

  27. Jiang, J.: Image compression with neural networks—a survey. Signal Proc. Image Commun. 14(9), 737–760 (1999)

    Article  Google Scholar 

  28. Kovasznay, G., Leslie, S., Joseph, H.M.: Image processing. Proc. IRE 43(5), 560–570 (1955). https://doi.org/10.1109/JRPROC.1955.278100

    Article  Google Scholar 

  29. Kalogerakis, E., Nowrouzezahrai, D., Breslav, S., Hertzmann, A.: Learning hatching for pen-and-ink illustration of surfaces. ACM Trans. Graph. 31(1), 66 (2012)

    Article  Google Scholar 

  30. Kapur, J.N., Sahoo, P.K., Wong, A.K.: A new method for gray-level picture thresholding using the entropy of the histogram. CVGIP 29(3), 273–285 (1985)

    Google Scholar 

  31. Kittler, J., Illingworth, J.: Minimum error thresholding. Pattern Recognit. 19(1), 41–47 (1986)

    Article  Google Scholar 

  32. Kramer, M.: Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 37(2), 233–243 (1991)

    Article  Google Scholar 

  33. Lee, S.U., Chung, S.Y., Park, R.H.: A comparative performance study of several global thresholding techniques for segmentation. Comput. Vis. Graph. Image Process. 52(2), 171–190 (1990)

    Article  Google Scholar 

  34. Li, C.H., Tam, P.K.S.: An iterative algorithm for minimum cross entropy thresholding. Pattern Recognit. Lett. 19(8), 771–776 (1998)

    Article  Google Scholar 

  35. Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. In: NIPS 29, pp. 4898–4906 (2016)

  36. Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015)

  37. Minnen, D., Toderici, G., Singh, S., Hwang, S.J., Covell, M.: Image-dependent local entropy models for learned image compression. arXiv (2018)

  38. Niblack, W.: An Introduction to Digital Image Processing. Englewood Cliffs (1986)

  39. Ostromoukhov, V.: A simple and efficient error-diffusion algorithm. In: Proceedings of the 28th Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’01), pp. 567–572 (2001)

  40. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  41. Pang, W.M., Qu, Y., Wong, T.T., Cohen-Or, D., Heng, P.A.: Structure-aware halftoning. In: ACM SIGGRAPH 2008 Papers (SIGGRAPH’08) (2008)

  42. Phansalkar, N., More, S., Sabale, A., Joshi, M.: Adaptive local thresholding for detection of nuclei in diversity stained cytology images. In: 2011 International Conference on Communications and Signal Processing, pp. 218–220 (2011)

  43. Praun, E., Hoppe, H., Webb, M., Finkelstein, A.: Real-time hatching. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, p. 581 (2001)

  44. Prewitt, J.M., Mendelsohn, M.L.: The analysis of cell images. Ann. N. Y. Acad. Sci. 128(3), 1035–1053 (1966)

    Article  Google Scholar 

  45. Qu, Y., Pang, W.M., Wong, T.T., Heng, P.A.: Richness-preserving manga screening. ACM Trans. Graph. 27(5), 1–8 (2008)

    Article  Google Scholar 

  46. Rasband, W.S.: Imagej (1997). U. S. National Institutes of Health, Bethesda, MD, USA. https://imagej.nih.gov/. Accessed 10/7/2020

  47. Ridler, T., Calvard, S., et al.: Picture thresholding using an iterative selection method. IEEE Trans. Syst. Man Cybern. 8(8), 630–632 (1978)

    Article  Google Scholar 

  48. Roetling, P.G.: Pseudo-gray techniques for the black and white display of color images. In: Image Quality, vol. 310, pp. 133–136. International Society for Optics and Photonics (1981)

  49. Rougier, N.: [re]weighted Voronoi stippling. The ReScience journal, GitHub, vol. 3(no. 1) (2017). https://github.com/ReScience-Archives/Rougier-2017

  50. Sauvola, J., Pietikäinen, M.: Adaptive document image binarization. Pattern Recognit. 33(2), 225–236 (2000)

  51. Schneider, C.A., Rasband, W.S., Eliceiri, K.W.: Nih image to imagej: 25 years of image analysis. Nat. Methods 9(7), 671–675 (2012)

    Article  Google Scholar 

  52. Sezgin, M., Sankur, B.: Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imaging 13(1), 146–166 (2004)

    Article  Google Scholar 

  53. Shanbhag, A.G.: Utilization of information measure as a means of image thresholding. CVGIP Graph. Models Image Process. 56(5), 414–419 (1994)

  54. Soille, P.: Morphological Image Analysis: Principles and Applications. Springer (2013)

  55. Stathis, P., Kavallieratou, E., Papamarkos, N.: An evaluation survey of binarization algorithms on historical documents. In: 2008 19th ICPR, pp. 1–4. IEEE (2008)

  56. Theis, L., Shi, W., Cunningham, A., Huszár, F.: Lossy image compression with compressive autoencoders. In: ICLR (2017)

  57. Tsai, W.H.: Moment-preserving thresolding: a new approach. Comput. Vis. Graph. Image Process. 29(3), 377–393 (1985)

    Article  Google Scholar 

  58. Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., Cottrell, G.: Understanding convolution for semantic segmentation. In: IEEE-WACV. IEEE (2018)

  59. Wikipedia: E ink. Wikipedia (2020). https://en.wikipedia.org/wiki/E_Ink. Accessed 10/7/2020

  60. Xia, M., Liu, X., Wong, T.T.: Invertible grayscale. ACM Trans. Graph. 37(6), 1–10 (2018)

    Article  Google Scholar 

  61. Yen, J.C., Chang, F.J., Chang, S.: A new criterion for automatic multilevel thresholding. IEEE Trans. Image Process. 4(3), 370–378 (1995)

    Article  MathSciNet  Google Scholar 

  62. Zack, G.W., Rogers, W.E., Latt, S.A.: Automatic measurement of sister chromatid exchange frequency. J. Histochem. Cytochem. 25(7), 741–753 (1977)

    Article  Google Scholar 

  63. Zander, J., Isenberg, T., Schlechtweg, S., Strothotte, T.: High quality hatching. Comp. Graph. Forum 23(3), 421–430 (2004)

    Article  Google Scholar 

  64. Zhou, B., Fang, X.: Improving mid-tone quality of variable-coefficient error diffusion using threshold modulation. In: ACM SIGGRAPH (SIGGRAPH’03), pp. 437–444 (2003)

  65. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICCV, pp. 2223–2232 (2017)

Download references

Author information

Authors and Affiliations

  1. Google Research, Mountain View, USA

    Shumeet Baluja

Authors
  1. Shumeet Baluja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shumeet Baluja.

Ethics declarations

Conflict of interest

The author is employed at Google, Inc.. He has no conflicts of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baluja, S. A natural representation of colors with textures. Vis Comput 38, 3267–3278 (2022). https://doi.org/10.1007/s00371-022-02568-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02568-1

Keywords

Search

Navigation