research-article

Perceptual Image Compression with Block-Level Just Noticeable Difference Prediction

Authors:

C.-C. Jay KuoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 16, Issue 4

Article No.: 126, Pages 1 - 15

https://doi.org/10.1145/3408320

Published: 28 January 2021 Publication History

Abstract

A block-level perceptual image compression framework is proposed in this work, including a block-level just noticeable difference (JND) prediction model and a preprocessing scheme. Specifically speaking, block-level JND values are first deduced by utilizing the OTSU method based on the variation of block-level structural similarity values between two adjacent picture-level JND values in the MCL-JCI dataset. After the JND value for each image block is generated, a convolutional neural network–based prediction model is designed to forecast block-level JND values for a given target image. Then, a preprocessing scheme is devised to modify the discrete cosine transform coefficients during JPEG compression on the basis of the distribution of block-level JND values of the target test image. Finally, the test image is compressed by the max JND value across all of its image blocks in the light of the initial quality factor setting. The experimental results demonstrate that the proposed block-level perceptual image compression method is able to achieve 16.75% bit saving as compared to the state-of-the-art method with similar subjective quality. The project page can be found at https://mic.tongji.edu.cn/43/3f/c9778a148287/page.htm.

References

[1]

Rich Franzen. 1999. Kodak Image Dataset. Retrieved September 24, 2020 from http://r0k.us/graphics/kodak/.

[2]

Website Hosting Rating. 2020. 40+ Instagram Statistics 8 Facts for 2020. Retrieved September 24, 2020 from https://www.websitehostingrating.com/instagram-statistics/.

[3]

S.-H. Bae, J. Kim, and M. Kim. 2016. HEVC-based perceptually adaptive video coding using a DCT-based local distortion detection probability model. IEEE Transactions on Image Processing 25, 7 (July 2016), 3343--3357.

[4]

M. Bouchakour, G. Jeannic, and F. Autrusseau. 2008. JND mask adaptation for wavelet domain watermarking. In Proceedings of the IEEE International Conference on Multimedia and Expo. 201--204.

[5]

Y.-J. Chiu and T. Berger. 1999. A software-only videocodec using pixelwise conditional differential replenishment and perceptual enhancements. IEEE Transactions on Circuits and Systems for Video Technology 9, 3 (April 1999), 438--450.

[6]

C. H. Chou and K. C. Liu. 2010. A perceptually tuned watermarking scheme for color images. IEEE Transactions on Image Processing 19, 11 (Nov. 2010), 2966--2982.

[7]

J. D. Cock, Z. Li, M. Manohara, and A. Aaron. 2016. Complexity-based consistent-quality encoding in the cloud. In Proceedings of the IEEE International Conference on Image Processing. 179--183.

[8]

ITU. 2002. Methodology for the Subjective Assessment of the Quality of Television Pictures. Rec. ITU-R BT.500-11. ITU, Geneva, Switzerland.

[9]

F. Ernawan and S. H. Nugraini. 2014. The optimal quantization matrices for jpeg image compression from psychovisual threshold. Journal of Theoretical and Applied Information Technology 70, 3 (Dec. 2014), 566--572.

[10]

S. Hu, H. Wang, and C.-C. J. Kuo. 2016. A GMM-based stair quality model for human perceived JPEG images. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. 1070--1074.

Digital Library

[11]

L. Jin, J. Y. Lin, S. Hu, H. Wang, P. Wang, L. Katsavounidis, A. Aaron, and C.-C. J. Kuo. 2016. Statistical study on perceived JPEG image quality via MCL-JCI dataset construction and analysis. Electronic Imaging 9 (Feb. 2016), 1--9.

[12]

S. Ki, S.-H. Bae, M. Kim, and H. Ko. 2018. Learning-based just-noticeable-quantization-distortion modeling for perceptual video coding. IEEE Transactions on Image Processing 27, 7 (July 2018), 3178--3193.

[13]

A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the Neural Information Processing Systems Conference. 1097--1105.

[14]

L. Li, W. Lin, and H. Zhu. 2014. Learning structural regularity for evaluating blocking artifacts in JPEG images. IEEE Signal Processing Letters 21, 8 (Aug. 2014), 918--922.

[15]

Y. Li, H. Liu, and Z. Chen. 2016. Perceptually-lossless image coding based on foveated-JND and H.265/HEVC. Journal of Visual Communication and Image Representation 40 (Aug. 2016), 600--610.

Digital Library

[16]

J. Y. Lin, L. Jin, S. Hu, Z. Li, A. Aaron, and C.-C. J. Kuo. 2015. Experimental design and analysis of JND test on coded image/video. Applications of Digital Image Processing XXXVIII 9599 (Sept 2015), 1--11.

[17]

W. Lin, L. Dong, and P. Xue. 2005. Visual distortion gauge based on discrimination of noticeable contrast changes. IEEE Transactions on Circuits and Systems for Video Technology 15, 7 (July 2005), 900--909.

Digital Library

[18]

A. Liu, W. Lin, M. Paul, C. Deng, and F. Zhang. 2010. Just noticeable difference for images with decomposition model for separating edge and textured regions. IEEE Transactions on Circuits and Systems for Video Technology 20, 11 (Nov. 2010), 1648--1652.

Digital Library

[19]

Z. Lu, W. Lin, X. Yang, E. Ong, and S. Yao. 2005. Modeling visual attention’s modulatory aftereffects on visual sensitivity and quality evaluation. IEEE Transactions on Image Processing 14, 11 (Nov. 2005), 1928--1942.

[20]

Z. Pan, J. Lei, Y. Zhang, and F. L. Wang. 2018. Adaptive fractional-pixel motion estimation skipped algorithm for efficient HEVC motion estimation. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 1 (Jan. 2018), 1--19.

Digital Library

[21]

G. J. Sullivan, J. Ohm, W.-J. Han, and T. Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (Sept. 2012), 1649--1668.

Digital Library

[22]

M. Takeuchi, S. Saika, Y. Sakamoto, T. Nagashima, Z. Cheng, K. Kanai, J. Katto, K. Wei, Z. Ju, and W. Xu. 2018. Perceptual quality driven adaptive video coding using JND estimation. In Proceedings of the Picture Coding Symposium. 179--183.

[23]

T. Tian and H. Wang. 2018. Large-scale video compression: Recent advances and challenges. Frontiers of Computer Science 12, 5 (Oct. 2018), 825--839.

Digital Library

[24]

G. K. Wallace. 1992. The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics 38, 1 (Feb. 1992), xviii–xxxiv.

Digital Library

[25]

H. Wang, W. Gan, S. Hu, J. Y. Lin, L. Jin, L. Song, P. Wang, I. Katsavounidis, A. Aaron, and C.-C. J. Kuo. 2016. MCL-JCV: A JND-based H.264/AVC video quality assessment dataset. In Proceedings of the IEEE International Conference on Image Processing. 1509--1513.

[26]

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (April 2004), 1--14.

Digital Library

[27]

A. B. Watson. 1993. DCT quantization matrices visually optimized for individual images. Human Vision, Visual Processing, and Digital Display IV 1913 (Sept. 1993), 202--217.

[28]

A. B. Watson, J. Hu, and J. F. McGowan. 2001. Digital video quality metric based on human vision. Journal of Electronic Imaging 10, 1 (Jan. 2001), 20--29.

[29]

Z. Wei and K. N. Ngan. 2009. Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. IEEE Transactions on Circuits and Systems for Video Technology 19, 3 (March 2009), 337--346.

[30]

T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra. 2003. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (Aug. 2003), 560--576.

Digital Library

[31]

R. B. Wolfgang, C. I. Podilchuk, and E. J. Delp. 2013. Perceptual watermarks for digital images and video. Proceedings of the IEEE 87, 7 (July 2013), 1108--1126.

[32]

J. Wu, L. Li, W. Dong, G. Shi, W. Lin, and C.-C. J. Kuo. 2017. Enhanced just noticeable difference model for images with pattern complexity. IEEE Transactions on Image Processing 26, 6 (March 2017), 2682--2693.

Digital Library

[33]

J. Wu, G. Shi, W. Lin, A. Liu, and F. Qi. 2013. Just noticeable difference estimation for images with free-energy principle. IEEE Transactions on Multimedia 15, 7 (Nov. 2013), 1705--1710.

Digital Library

[34]

X. Yang, W. Lin, Z. Lu, E. Ong, and S. Yao. 2005. Motion-compensated residue preprocessing in video coding based on just-noticeable-distortion profile. IEEE Transactions on Circuits and Systems for Video Technology 15, 6 (June 2005), 742--752.

[35]

X. Zhang, W. Lin, and P. Xue. 2005. Improved estimation for just-noticeable visual distortion. Signal Processing 85, 4 (April 2005), 795--808.

Digital Library

[36]

X. Zhang, S. Wang, K. Gu, W. Lin, S. Ma, and W. Gao. 2017. Just-noticeable difference-based perceptual optimization for JPEG compression. IEEE Signal Processing Letters 24, 1 (Jan. 2017), 96--100.

Cited By

Liu HLiu SXiao JXu DFan X(2024)Video-Wise Just-Noticeable Distortion Prediction Model for Video Compression with a Spatial–Temporal NetworkElectronics10.3390/electronics1324497713:24(4977)Online publication date: 18-Dec-2024
https://doi.org/10.3390/electronics13244977
Wang MZhang RHuang LLi YWooldridge MDy JNatarajan S(2024)Visual redundancy removal for composite imagesProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i9.28884(10189-10197)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i9.28884
Wu HWang ZLi YLiu XLee T(2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 12-Mar-2024
https://dl.acm.org/doi/10.1145/3652518
Show More Cited By

Index Terms

Perceptual Image Compression with Block-Level Just Noticeable Difference Prediction
1. Computing methodologies
  1. Computer graphics

Recommendations

Locally Adaptive Perceptual Compression for Color Images

The main idea in perceptual image compression is to remove the perceptual redundancy for representing images at the lowest possible bit rate without introducing perceivable distortion. A certain amount of perceptual redundancy is inherent in the color ...
No-reference noticeable blockiness estimation in images

Blocking artifact is a prevailing degradation caused by the block-based Discrete Cosine Transform (BDCT) coding technique under low bit-rate conditions. In this paper, we propose an efficient no-reference noticeable blockiness estimation algorithm for ...
A new full-reference image quality metric based on just noticeable difference

An Image Quality Metric (IQM) is used to measure how the image on which modifications are carried out is distorted. Knowing the distortion scale of the image is of significance for the intended purposes (compression, forensic, medical, etc.). The amount ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 16, Issue 4

November 2020

372 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3444749

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 January 2021

Accepted: 01 May 2020

Received: 01 March 2020

Published in TOMM Volume 16, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
Ministry of Science and Technology of China
Shanghai Engineering Research Center of Industrial Vision Perception 8 Intelligent Computing

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)7

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu HLiu SXiao JXu DFan X(2024)Video-Wise Just-Noticeable Distortion Prediction Model for Video Compression with a Spatial–Temporal NetworkElectronics10.3390/electronics1324497713:24(4977)Online publication date: 18-Dec-2024
https://doi.org/10.3390/electronics13244977
Wang MZhang RHuang LLi YWooldridge MDy JNatarajan S(2024)Visual redundancy removal for composite imagesProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i9.28884(10189-10197)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i9.28884
Wu HWang ZLi YLiu XLee T(2024)Suitable and Style-Consistent Multi-Texture Recommendation for Cartoon IllustrationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251820:7(1-26)Online publication date: 12-Mar-2024
https://dl.acm.org/doi/10.1145/3652518
Wang LShi YWang JChen SYin BLing N(2024)Graph Based Cross-Channel Transform for Color Image CompressionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363171020:4(1-25)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3631710
Nami SPakdaman FHashemi MShirmohammadi SGabbouj M(2024)Lightweight Multitask Learning for Robust JND Prediction Using Latent Space and Reconstructed FramesIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.338998834:9(8657-8671)Online publication date: Sep-2024
https://doi.org/10.1109/TCSVT.2024.3389988
Xiao X(2024)FPSNet: Focus-Perceptual-Semantic Full Flow Visual Redundancy Predicting for Camera ImagePattern Recognition and Computer Vision10.1007/978-981-97-8692-3_2(15-26)Online publication date: 1-Nov-2024
https://doi.org/10.1007/978-981-97-8692-3_2
Yang CLin CMao TAnggara AWu C(2023)Leisure Motivation and Happiness, Mediation of Leisure Attitude and Perceived Value: An Evidence from Large and Heavy Motorbike Riders in TaiwanAnnals of Applied Sport Science10.61186/aassjournal.114011:2(0-0)Online publication date: 1-Aug-2023
https://doi.org/10.61186/aassjournal.1140
Xie WWang STian SHuang LLiu YWang MWilliams BChen YNeville J(2023)Just noticeable visual redundancy forecastingProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i3.25399(2965-2973)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i3.25399
Xue YJin JSun WLin W(2023)HVS-inspired adversarial image generation with high perceptual qualityJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00470-212:1Online publication date: 13-Jun-2023
https://dl.acm.org/doi/10.1186/s13677-023-00470-2
Xie WWang SZhang RWang MEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Visual Redundancy Removal of Composite Images via Multimodal LearningProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612118(6765-6773)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612118
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents