Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

3R-INN: How to Be Climate Friendly While Consuming/Delivering Videos?

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15133))

Included in the following conference series:

  • 126 Accesses

Abstract

The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas (GHG) emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single invertible network that does three tasks at once: given a high-resolution (HR) grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the hr grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, 3R-INN enables significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), while outperforming state-of-the-art film grain removal and synthesis, energy-aware and downscaling methods on different test-sets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The additional dataset is proposed at www.interdigital.com/data_sets/filmgrainstyle740k-dataset.

References

  1. Energy consumption household. https://www.energybot.com/blog/average-energy-consumption.html

  2. Netflix subscribers. https://www.usnews.com/news/business/articles/2024-01-23/netflixs-subscriber-growth-surges-as-streaming-service-unwraps-best-ever-holiday-season-results

  3. Vtm-19.0. https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tags/VTM-19.0

  4. Agustsson, E., Timofte, R.: NTIRE 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 126–135 (2017)

    Google Scholar 

  5. Ameur, Z., Demarty, C.H., Le Meur, O., Ménard, D., François, E.: Style-based film grain analysis and synthesis. In: Proceedings of the 14th Conference on ACM Multimedia Systems, pp. 229–238 (2023)

    Google Scholar 

  6. Ameur, Z., Hamidouche, W., François, E., Radosavljević, M., Menard, D., Demarty, C.H.: Deep-based film grain removal and synthesis. IEEE Trans. Image Process. (2023)

    Google Scholar 

  7. Bonnineau, C., Hamidouche, W., Travers, J.F., Déforges, O.: Versatile video coding and super-resolution for efficient delivery of 8k video with 4k backward-compatibility. In: ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2048–2052 (2020). https://doi.org/10.1109/ICASSP40776.2020.9054716

  8. Boyce, J., Suehring, K., Li, X., Seregin, V.: JVET-J1010: JVET common test conditions and software reference configurations. In: 10th Meeting of the Joint Video Experts Team, pp. JVET–J1010 (2018)

    Google Scholar 

  9. Chen, Z., Liu, T., Huang, J.J., Zhao, W., Bi, X., Wang, M.: Invertible mosaic image hiding network for very large capacity image steganography. arXiv preprint arXiv:2309.08987 (2023)

  10. Dai, J., Au, O.C., Pang, C., Yang, W., Zou, F.: Film grain noise removal and synthesis in video coding. In: 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 890–893. IEEE (2010)

    Google Scholar 

  11. Demarty, C.H., Blondé, L., Le Meur, O.: Display power modeling for energy consumption control. In: 2023 IEEE International Conference on Image Processing (ICIP). IEEE (2023)

    Google Scholar 

  12. Dinh, L., Sohl-Dickstein, J., Bengio, S.: Density estimation using real NVP. arXiv preprint arXiv:1605.08803 (2016)

  13. Du, W., Chen, H., Zhang, Y., Yang, H.: Hierarchical disentangled representation for invertible image denoising and beyond. arXiv preprint arXiv:2301.13358 (2023)

  14. Franzen, R.: Kodak lossless true color image suite. Source: http://r0k.us/graphics/kodak4(2), 9 (1999)

  15. Gomila, C.: SEI message for film grain encoding. JVT document, May 2003

    Google Scholar 

  16. Herglotz, C., Brand, F., Regensky, A., Rievel, F., Kaup, A.: Processing energy modeling for neural network based image compression. In: 2023 IEEE International Conference on Image Processing (ICIP), pp. 2390–2394. IEEE (2023)

    Google Scholar 

  17. Herglotz, C., Kränzler, M., Schober, R., Kaup, A.: Sweet streams are made of this: the system engineer’s view on energy efficiency in video communications [feature]. IEEE Circuits Syst. Mag. 23(1), 57–77 (2023)

    Article  Google Scholar 

  18. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

    Google Scholar 

  19. Hwang, I., Jeong, J., Choi, J., Choe, Y.: Enhanced film grain noise removal for high fidelity video coding. In: 2013 International Conference on Information Science and Cloud Computing Companion, pp. 668–674. IEEE (2013)

    Google Scholar 

  20. Kang, S.J.: Image-quality-based power control technique for organic light emitting diode displays. J. Display Technol. 11(1), 104–109 (2015)

    Article  Google Scholar 

  21. Kang, S.J., Kim, Y.H.: Image integrity-based gray-level error control for low power liquid crystal displays. IEEE Trans. Consum. Electron. 55(4), 2401–2406 (2009). https://doi.org/10.1109/TCE.2009.5373816

  22. Kim, H., Choi, M., Lim, B., Mu Lee, K.: Task-aware image downscaling. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 419–434. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_25

    Chapter  Google Scholar 

  23. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  24. Kingma, D.P., Dhariwal, P.: Glow: generative flow with invertible 1 \(\times \) 1 convolutions. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  25. Le Meur, O., Demarty, C.H.: Invertible energy-aware images. IEEE Signal Process. Lett. (2023)

    Google Scholar 

  26. Le Meur, O., Demarty, C.H., Blondé, L.: Deep-learning-based energy aware images. In: 2023 IEEE International Conference on Image Processing (ICIP), pp. 590–594. IEEE (2023)

    Google Scholar 

  27. Liu, Y., et al.: Invertible denoising network: a light solution for real noise removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13365–13374 (2021)

    Google Scholar 

  28. Lu, S.P., Wang, R., Zhong, T., Rosin, P.L.: Large-capacity image steganography based on invertible neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10816–10825 (2021)

    Google Scholar 

  29. Malmodin, J.: The power consumption of mobile and fixed network data services-the case of streaming video and downloading large files. In: Electronics Goes Green, vol. 2020 (2020)

    Google Scholar 

  30. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 2, pp. 416–423. IEEE (2001)

    Google Scholar 

  31. Newson, A., Delon, J., Galerne, B.: A stochastic film grain model for resolution-independent rendering. In: Computer Graphics Forum, vol. 36, pp. 684–699. Wiley Online Library (2017)

    Google Scholar 

  32. Norkin, A., Birkbeck, N.: Film grain synthesis for AV1 video codec. In: 2018 Data Compression Conference, pp. 3–12. IEEE (2018)

    Google Scholar 

  33. Nugroho, K.A., Ruan, S.J.: R-ACE network for OLED image power saving. In: 2022 IEEE 4th Global Conference on Life Sciences and Technologies (LifeTech), pp. 284–285. IEEE (2022)

    Google Scholar 

  34. Radosavljevic, M., François, E., Reinhard, E., Hamidouche, W., Amestoy, T.: Implementation of film-grain technology within VVC. In: Applications of Digital Image Processing XLIV, vol. 11842, pp. 85–95. SPIE (2021)

    Google Scholar 

  35. Reddi, S.J., Kale, S., Kumar, S.: On the convergence of Adam and beyond. arXiv preprint arXiv:1904.09237 (2019)

  36. Reinhard, E., Demarty, C.H., Blondé, L.: Pixel value adjustment to reduce the energy requirements of display devices. SMPTE Motion Imaging J. 132(7), 10–19 (2023)

    Article  Google Scholar 

  37. Robinson, D.: Greening of streaming: the less accord: low energy sustainable streaming. In: Proceedings of the 2nd Mile-High Video Conference (MHV 2023), p. 115 (2023)

    Google Scholar 

  38. Shin, Y.G., Park, S., Yoo, M.J., Ko, S.J.: Unsupervised deep power saving and contrast enhancement for OLED displays. arXiv preprint arXiv:1905.05916 (2019)

  39. Stoyan, D., Kendall, W.S., Chiu, S.N., Mecke, J.: Stochastic Geometry and Its Applications. Wiley, New York (2013)

    Google Scholar 

  40. Sun, W., Chen, Z.: Learned image downscaling for upscaling using content adaptive resampler. IEEE Trans. Image Process. 29, 4027–4040 (2020)

    Article  Google Scholar 

  41. Trust, T.C.: Carbon impact of video streaming (2021). https://www.carbontrust.com/en-eu/node/1537

  42. Xiao, M., et al.: Invertible image rescaling. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020, Proceedings, Part I 16, pp. 126–144. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_8

  43. Yin, J.L., Chen, B.H., Peng, Y.T., Tsai, C.C.: Deep battery saver: end-to-end learning for power constrained contrast enhancement. IEEE Trans. Multimedia 23, 1049–1059 (2020)

    Article  Google Scholar 

  44. Zhao, R., Liu, T., Xiao, J., Lun, D.P., Lam, K.M.: Invertible image decolorization. IEEE Trans. Image Process. 30, 6081–6095 (2021)

    Article  Google Scholar 

  45. Zhu, F., Chen, G., Hao, J., Heng, P.A.: Blind image denoising via dependent Dirichlet process tree. IEEE Trans. Pattern Anal. Mach. Intell. 39(8), 1518–1531 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. InterDigital, Rennes, France

    Zoubida Ameur, Claire-Hélène Demarty & Olivier Le Meur

  2. IETR, Rennes, France

    Daniel Ménard

Authors
  1. Zoubida Ameur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claire-Hélène Demarty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Ménard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivier Le Meur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoubida Ameur .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Hessen, Germany

    Stefan Roth

  4. Princeton University, Palo Alto, CA, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 7330 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ameur, Z., Demarty, CH., Ménard, D., Meur, O.L. (2025). 3R-INN: How to Be Climate Friendly While Consuming/Delivering Videos?. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15133. Springer, Cham. https://doi.org/10.1007/978-3-031-73226-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73226-3_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73225-6

  • Online ISBN: 978-3-031-73226-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation