Abstract
In traditional collaborative rendering architecture, the front-end computes direct lighting, which imposes certain performance requirements on the front-end devices. To further reduce the front-end load in complex 3D scenes, we propose a Super-resolution Screen Space Irradiance Sampling technique (SRSSIS), which is applied to our designed architecture, a lightweight collaborative rendering system built on Web3D. In our system, the back-end samples low-resolution screen-space irradiance, while the front-end implements our SRSSIS technique to reconstruct high-resolution and high-quality images. We also introduce frame interpolation in the architecture to further reduce the backend load and the transmission frequency. Moreover, we propose a self-adaptive sampling strategy to improve the robustness of super-resolution. Our experiments show that, under ideal conditions, our reconstruction performance is comparable to DLSS and FSR real-time super-resolution technology. The bandwidth consumption of our system ranges from 8% to 66% of pixel streaming at different super-resolution rates, while the back-end’s computational cost is approximately 33% to 46% of pixel streaming at different super-resolution rates.
H. Long and Y. Yang—Both authors contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
AMD: AMD fidelityFX super resolution (2023). https://www.amd.com/en/technologies/fidelityfx-super-resolution
Bauszat, P., Eisemann, M., Magnor, M.: Guided image filtering for interactive high-quality global illumination. In: Computer Graphics Forum, vol. 30, pp. 1361–1368. Wiley Online Library (2011)
Bugeja, K., Debattista, K., Spina, S.: An asynchronous method for cloud-based rendering. Vis. Comput. 35, 1827–1840 (2019)
Caballero, J., et al.: Real-time video super-resolution with spatio-temporal networks and motion compensation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4778–4787 (2017)
Crassin, C., et al.: CloudLight: a system for amortizing indirect lighting in real-time rendering. J. Comput. Graph. Tech. 4(4), 1–27 (2015)
Dammertz, H., Sewtz, D., Hanika, J., Lensch, H.P.: Edge-avoiding A-Trous wavelet transform for fast global illumination filtering. In: Proceedings of the Conference on High Performance Graphics, pp. 67–75 (2010)
Guo, J., et al.: ExtraNet: real-time extrapolated rendering for low-latency temporal supersampling. ACM Trans. Graph. (TOG) 40(6), 1–16 (2021)
Hladky, J., Seidel, H.P., Steinberger, M.: Tessellated shading streaming. In: Computer Graphics Forum, vol. 38, pp. 171–182. Wiley Online Library (2019)
Hladky, J., Stengel, M., Vining, N., Kerbl, B., Seidel, H.P.: QuadStream: a quad-based scene streaming architecture for novel viewpoint reconstruction. ACM Trans. Graph. (TOG) 41, C32 (2022)
Kaplanyan, A.: Cryengine 3: Reaching the speed of light. Talk, SIGGRAPH (2010)
Laghari, A.A., He, H., Memon, K.A., Laghari, R.A., Halepoto, I.A., Khan, A.: Quality of experience (QoE) in cloud gaming models: a review. Multiagent Grid Syst. 15(3), 289–304 (2019)
Liu, C., Ooi, W.T., Jia, J., Zhao, L.: Cloud baking: collaborative scene illumination for dynamic Web3D scenes. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 14(3s), 1–20 (2018)
Liu, C., et al.: Web-cloud collaborative mobile online 3D rendering system. Secur. Commun. Netw. 2022 (2022)
Magro, M., Bugeja, K., Spina, S., Debattista, K.: Cloud-based dynamic GI for shared VR experiences. IEEE Comput. Graph. Appl. 40(5), 10–25 (2020)
Mansouri, J.E.E.: Rendering ‘rainbow six \(|\) siege’. Website (2016). https://www.gdcvault.com/play/1022990/Rendering-Rainbow-Six-Siege GDC
Mueller, J.H., et al.: Shading atlas streaming. ACM Trans. Graph. (TOG) 37(6), 1–16 (2018)
Myszkowski, K., Tawara, T., Akamine, H., Seidel, H.P.: Perception-guided global illumination solution for animation rendering. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 221–230 (2001)
Nehab, D., Sander, P.V., Lawrence, J., Tatarchuk, N., Isidoro, J.R.: Accelerating real-time shading with reverse reprojection caching. In: Graphics Hardware, vol. 41, pp. 61–62 (2007)
NVIDIA: GeForce NOW. Website (2023). https://www.nvidia.com/en-us/geforce-now/
NVIDIA: NVIDIA DLSS. Website (2023). https://www.nvidia.com/en-us/geforce/technologies/dlss/
Peñaherrera-Pulla, O.S., Baena, C., Fortes, S., Baena, E., Barco, R.: Measuring key quality indicators in cloud gaming: framework and assessment over wireless networks. Sensors 21(4), 1387 (2021)
Perkis, A., et al.: QUALINET white paper on definitions of immersive media experience (IMEx). arXiv:2007.07032 (2020)
Schied, C., et al.: Spatiotemporal variance-guided filtering: real-time reconstruction for path-traced global illumination. In: Proceedings of High Performance Graphics, pp. 1–12 (2017)
Schied, C., Peters, C., Dachsbacher, C.: Gradient estimation for real-time adaptive temporal filtering. Proc. ACM Comput. Graph. Interact. Tech. 1(2), 1–16 (2018)
Shao, W., Liu, C., Jia, J.: Lightmap-based GI collaborative rendering system for Web3D application. J. Syst. Simul. 32(4), 649 (2020)
Shea, R., Liu, J., Ngai, E.C.H., Cui, Y.: Cloud gaming: architecture and performance. IEEE Netw. 27(4), 16–21 (2013)
SONY: PlayStation Now. Website (2023). https://www.playstation.com/en-us/ps-now/
Stengel, M., Majercik, Z., Boudaoud, B., McGuire, M.: A distributed, decoupled system for losslessly streaming dynamic light probes to thin clients. In: Proceedings of the 12th ACM Multimedia Systems Conference, pp. 159–172 (2021)
Walter, B., Drettakis, G., Parker, S.: Interactive rendering using the render cache. In: Lischinski, D., Larson, G.W. (eds.) EGSR 1999. E, pp. 19–30. Springer, Vienna (1999). https://doi.org/10.1007/978-3-7091-6809-7_3
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wihlidal, G.: 4K checkerboard in ‘battlefield 1’ and ‘mass effect andromeda’. Website (2017). https://www.gdcvault.com/play/1022990/Rendering-Rainbow-Six-Siege GDC
Xiao, L., Nouri, S., Chapman, M., Fix, A., Lanman, D., Kaplanyan, A.: Neural supersampling for real-time rendering. ACM Trans. Graph. (TOG) 39(4), 142:1 (2020)
Zhan, Z., et al.: Achieving on-mobile real-time super-resolution with neural architecture and pruning search. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4821–4831 (2021)
Zhang, X., Zeng, H., Zhang, L.: Edge-oriented convolution block for real-time super resolution on mobile devices. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4034–4043 (2021)
Acknowledgements
This research is partially supported by the Basic Grant of Natural Science Foundation of China (No. 62072339), the Key Project of Regional Joint Grant of Science Natural Foundation of China (No. U19A2063) and a grant from the National Natural Science Foundation of China (No. 62262043).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Long, H., Yang, Y., Liu, C., Jia, J. (2024). SRSSIS: Super-Resolution Screen Space Irradiance Sampling for Lightweight Collaborative Web3D Rendering Architecture. In: Hu, SM., Cai, Y., Rosin, P. (eds) Computer-Aided Design and Computer Graphics. CADGraphics 2023. Lecture Notes in Computer Science, vol 14250. Springer, Singapore. https://doi.org/10.1007/978-981-99-9666-7_20
Download citation
DOI: https://doi.org/10.1007/978-981-99-9666-7_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9665-0
Online ISBN: 978-981-99-9666-7
eBook Packages: Computer ScienceComputer Science (R0)
