research-article

Scalable Color Quantization for Task-centric Image Compression

Authors:

Jong Hwan KoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications and Applications, Volume 19, Issue 2s

Article No.: 82, Pages 1 - 18

https://doi.org/10.1145/3551389

Published: 17 February 2023 Publication History

Abstract

Conventional image compression techniques targeted for the perceptual quality are not generally optimized for classification tasks using deep neural networks (DNNs). To compress images for DNN inference tasks, recent studies have proposed task-centric image compression methods with quantization techniques optimized for DNN inference. Among them, color quantization was proposed to reduce the amount of data per pixel by limiting the number of distinct colors (color space) in an image. However, quantizing images into various color space sizes requires training and inference of multiple DNNs, each of which is dedicated to each color space. To overcome this limitation, we propose a scalable color quantization method, where images with variable color space sizes can be extracted from a master image generated by a single DNN model. This scalability is enabled by weighted color grouping that constructs a color palette using critical color components for the classification task. We also propose an adaptive training method that can jointly optimize images with various color-space sizes. The results show that the proposed method supports dynamic changes of the color space size between 1–6 bit color space per pixel, while even increasing the inference accuracy at a low bit precision up to 20.2% and 46.6% compared to other task- and human-centric color quantizations, respectively.

References

[1]

Wilhelm Burger and Mark J. Burge. 2016. Color quantization. In Digital Image Processing. Springer, 329–339.

[2]

Lahiru D. Chamain, Fabien Racapé, Jean Bégaint, Akshay Pushparaja, and Simon Feltman. 2021. End-to-end optimized image compression for machines, a study. In Data Compression Conference (DCC). IEEE, 163–172.

[3]

Jinyoung Choi and Bohyung Han. 2020. Task-aware quantization network for JPEG image compression. In European Conference on Computer Vision. Springer, 309–324.

Digital Library

[4]

Adam Coates, Andrew Ng, and Honglak Lee. 2011. An analysis of single-layer networks in unsupervised feature learning. In 14th International Conference on Artificial Intelligence and Statistics. 215–223.

[5]

Yining Deng, Charles Kenney, Michael S. Moore, and B. S. Manjunath. 1999. Peer group filtering and perceptual color image quantization. In IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 21–24.

[6]

Yining Deng and Bangalore S. Manjunath. 2001. Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Pattern Anal. Mach. Intell. 23, 8 (2001), 800–810.

Digital Library

[7]

Changsheng Gao, Dong Liu, Li Li, and Feng Wu. 2021. Towards task-generic image compression: A study of semantics-oriented metrics. IEEE Trans. Multim. (2021).

[8]

Michael Gervautz and Werner Purgathofer. 1988. A simple method for color quantization: Octree quantization. In New Trends in Computer Graphics. Springer, 219–231.

[9]

Naftaly Goldberg. 1991. Colour image quantization for high resolution graphics display. Image. Vis. Comput. 9, 5 (1991), 303–312.

[10]

Teofilo F. Gonzalez. 1985. Clustering to minimize the maximum intercluster distance. Theoret. Comput. Sci. 38 (1985), 293–306.

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770–778.

[12]

Paul Heckbert. 1982. Color image quantization for frame buffer display. ACM Siggraph Comput. Graph. 16, 3 (1982), 297–307.

Digital Library

[13]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).

[14]

Yunzhong Hou, Liang Zheng, and Stephen Gould. 2020. Learning to structure an image with few colors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10116–10125.

[15]

G. Houle and E. Dubois. 1986. Quantization of color images for display on graphics terminals. In IEEE Global Telecommunications Conference. 1138–1142.

[16]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).

[17]

Jack Kiefer, Jacob Wolfowitz, et al. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23, 3 (1952), 462–466.

[18]

Sang Hoon Kim, Jae Hyun Park, and Jong Hwan Ko. 2021. Target-dependent scalable image compression using a reconfigurable recurrent neural network. IEEE Access 9 (2021), 119418–119429.

[19]

Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. https://www.cs.toronto.edu/kriz/learning-features-2009-TR.pdf.

[20]

Ya Le and Xuan Yang. 2015. Tiny ImageNet visual recognition challenge. CS 231N 7 (2015).

[21]

Zihao Liu, Tao Liu, Wujie Wen, Lei Jiang, Jie Xu, Yanzhi Wang, and Gang Quan. 2018. DeepN-JPEG: A deep neural network favorable JPEG-based image compression framework. In 55th Annual Design Automation Conference. 1–6.

Digital Library

[22]

Ilya Loshchilov and Frank Hutter. 2016. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016).

[23]

ISO/IEC JTC 1/SC29/WG1 N100094. 2022. Coding of Still Pictures - ds.jpeg.org. Retrieved from https://ds.jpeg.org/documents/jpegai/wg1n90021-REQ-JPEG_AI_Use_Cases_and_Requirements.pdf.

[24]

Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition. 427–436.

[25]

Michael T. Orchard, Charles A. Bouman, et al. 1991. Color quantization of images. IEEE Trans. Sig. Process. 39, 12 (1991), 2677–2690.

Digital Library

[26]

Neel Patwa, Nilesh Ahuja, Srinivasa Somayazulu, Omesh Tickoo, Srenivas Varadarajan, and Shashidhar Koolagudi. 2020. Semantic-preserving image compression. In IEEE International Conference on Image Processing (ICIP). IEEE, 1281–1285.

[27]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 234–241.

[28]

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-Cam: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision. 618–626.

[29]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[30]

Leslie N. Smith and Nicholay Topin. 2019. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Vol. 11006. International Society for Optics and Photonics, 1100612.

[31]

Qiang Wang, Liquan Shen, and Yuan Shi. 2020. Recognition-driven compressed image generation using semantic-prior information. IEEE Sig. Process. Lett. 27 (2020), 1150–1154.

[32]

Maurice Weber, Cedric Renggli, Helmut Grabner, and Ce Zhang. 2019. Lossy image compression with recurrent neural networks: From human perceived visual quality to classification accuracy. arXiv preprint arXiv:1910.03472 (2019).

[33]

Maurice Weber, Cedric Renggli, Helmut Grabner, and Ce Zhang. 2021. Observer dependent lossy image compression. In Pattern Recognition: 42nd DAGM German Conference, DAGM GCPR’20, Tübingen, Germany, September 28–October 1, 2020, Proceedings 42. Springer, 130–144.

Digital Library

[34]

Xiaolin Wu. 1992. Color quantization by dynamic programming and principal analysis. ACM Trans. Graph. 11, 4 (1992), 348–372.

Digital Library

[35]

Zhigang Xiang. 1997. Color image quantization by minimizing the maximum intercluster distance. ACM Trans. Graph. 16, 3 (1997), 260–276.

Digital Library

[36]

Haichao Yu, Haoxiang Li, Honghui Shi, Thomas S. Huang, and Gang Hua. 2019. Any-precision deep neural networks. arXiv preprint arXiv:1911.07346 (2019).

[37]

Jiahui Yu and Thomas S. Huang. 2019. Universally slimmable networks and improved training techniques. In IEEE/CVF International Conference on Computer Vision. 1803–1811.

[38]

Yichi Zhang, Ritchie Zhao, Weizhe Hua, Nayun Xu, G. Edward Suh, and Zhiru Zhang. 2020. Precision gating: Improving neural network efficiency with dynamic dual-precision activations. arXiv preprint arXiv:2002.07136 (2020).

Cited By

Liu SLin WChen YZhang YDai WSee JXiong H(2024)A Unified Framework for Jointly Compressing Visual and Semantic DataACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365480020:7(1-24)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3654800
Wang LShi YWang JChen SYin BLing N(2024)Graph Based Cross-Channel Transform for Color Image CompressionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363171020:4(1-25)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3631710
Wu KLi ZYang YLiu Q(2024)Deep video compression based on long-range temporal context learningComputer Vision and Image Understanding10.1016/j.cviu.2024.104127(104127)Online publication date: Aug-2024
https://doi.org/10.1016/j.cviu.2024.104127
Show More Cited By

Index Terms

Scalable Color Quantization for Task-centric Image Compression
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Appearance and texture representations
  2. Computer graphics
    1. Image compression

Recommendations

Comparative analysis of the quantization of color spaces on the basis of the CIELAB color-difference formula

This article discusses the CIELAB color spave within the limits of optimal colors including the complete volume of object colors. A graphical representation of this color space is composed of planes of constant lightness L* with an net of lines parallel ...
DWT-based scene-adaptive color quantization
Special issue on multi-dimensional image processing

Color quantization is the process of reducing the number of colors in an image. That is, color quantization maps a large number of colors into a much smaller number of representative colors while keeping color distortion to an acceptable level. The ...
Color image quantization with peak-picking and color space
Abstract
Color image quantization is a significant procedure of reducing the huge range of color values of a digital color image into a limited range. In this paper, an automated clustering of pixels and color quantization algorithm is proposed. The ideal ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 19, Issue 2s

April 2023

545 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3572861

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 February 2023

Online AM: 01 August 2022

Accepted: 20 July 2022

Revised: 19 May 2022

Received: 21 May 2021

Published in TOMM Volume 19, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Institute of Information and Communication Technology Planning Evaluation (IITP)
Information Technology Research Center (ITRC)
ICT Creative Consilience program
Artificial Intelligence Innovation Hub program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
416
Total Downloads

Downloads (Last 12 months)174
Downloads (Last 6 weeks)13

Reflects downloads up to 03 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu SLin WChen YZhang YDai WSee JXiong H(2024)A Unified Framework for Jointly Compressing Visual and Semantic DataACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365480020:7(1-24)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3654800
Wang LShi YWang JChen SYin BLing N(2024)Graph Based Cross-Channel Transform for Color Image CompressionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363171020:4(1-25)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3631710
Wu KLi ZYang YLiu Q(2024)Deep video compression based on long-range temporal context learningComputer Vision and Image Understanding10.1016/j.cviu.2024.104127(104127)Online publication date: Aug-2024
https://doi.org/10.1016/j.cviu.2024.104127
Shi WTao FWen Y(2024)Joint super-resolution-based fast face image coding for human and machine visionThe Visual Computer10.1007/s00371-024-03428-wOnline publication date: 20-May-2024
https://doi.org/10.1007/s00371-024-03428-w
Celebi M(2023)Forty years of color quantization: a modern, algorithmic surveyArtificial Intelligence Review10.1007/s10462-023-10406-656:12(13953-14034)Online publication date: 27-Apr-2023
https://dl.acm.org/doi/10.1007/s10462-023-10406-6

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents