Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

3D visual saliency and convolutional neural network for blind mesh quality assessment

  • IAPR-MedPRAI
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

A number of full reference and reduced reference methods have been proposed in order to estimate the perceived visual quality of 3D meshes. However, in most practical situations, there is a limited access to the information related to the reference and the distortion type. For these reasons, the development of a no-reference mesh visual quality (MVQ) approach is a critical issue, and more emphasis needs to be devoted to blind methods. In this work, we propose a no-reference convolutional neural network (CNN) framework to estimate the perceived visual quality of 3D meshes. The method is called SCNN-BMQA (3D visual saliency and CNN for blind mesh quality assessment). The main contribution is the usage of a CNN and 3D visual saliency to estimate the perceived visual quality of distorted meshes. To do so, the CNN architecture is fed by small patches selected carefully according to their level of saliency. First, the visual saliency of the 3D mesh is computed. Afterward, we render 2D projections from the 3D mesh and its corresponding 3D saliency map. Then the obtained views are split into 2D small patches that pass through a saliency filter in order to select the most relevant patches. Finally, a CNN is used for the feature learning and the quality score estimation. Extensive experiments are conducted on four prominent MVQ assessment databases, including several tests to study the effect of the CNN parameters, the effect of visual saliency and comparison with existing methods. Results show that the trained CNN achieves good rates in terms of correlation with human judgment and outperforms the most effective state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Botsch M, Kobbelt L, Pauly M, Alliez P, Lévy B (2010) Polygon mesh processing. CRC Press, Boca Raton

    Book  Google Scholar 

  2. Alliez P, Gotsman C (2005) Recent advances in compression of 3d meshes. In: Dodgson N, Floater M, Sabin M (eds) Advances in multiresolution for geometric modelling. Springer, Berlin, pp 3–26

    Chapter  Google Scholar 

  3. Lee H, Dikici Ç, Lavoué G, Dupont F (2011) Joint reversible watermarking and progressive compression of 3d meshes. Vis Comput 27(6):781–792

    Article  Google Scholar 

  4. Wang K, Lavoué G, Denis F, Baskurt A (2008) A comprehensive survey on three-dimensional mesh watermarking. IEEE Trans Multimed 10(8):1513–1527

    Article  Google Scholar 

  5. Wang Y-P, Shi-Min H (2009) A new watermarking method for 3d models based on integral invariants. IEEE Trans Vis Comput Graph 15(2):285–294

    Article  Google Scholar 

  6. Garland M, Heckbert PS (1997) Surface simplification using quadric error metrics. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 209–216

  7. Luebke DP (2001) A developer’s survey of polygonal simplification algorithms. IEEE Comput Graph Appl 21(3):24–35

    Article  Google Scholar 

  8. Corsini M, Larabi M-C, Lavoué G, Petřík O, Váša L, Wang K (2013) Perceptual metrics for static and dynamic triangle meshes. In: Computer graphics forum, vol 32, issue 1. Blackwell, Oxford, UK, pp 101–125

  9. Cignoni P, Rocchini C, Scopigno R (1998) Metro: measuring error on simplified surfaces. In: Computer graphics forum, vol 17, issue 2. Blackwell, Oxford, UK, pp 167–174

  10. Aspert N, Santa-Cruz D, Ebrahimi T (2002) Mesh: measuring errors between surfaces using the Hausdorff distance. In: Proceedings. 2002 IEEE international conference on multimedia and expo, 2002. ICME’02, vol 1. IEEE, pp 705–708

  11. Lavoué G, Corsini M (2010) A comparison of perceptually-based metrics for objective evaluation of geometry processing. IEEE Trans Multimed 12(7):636–649

    Article  Google Scholar 

  12. Bulbul A, Capin T, Lavoué G, Preda M (2011) Assessing visual quality of 3-d polygonal models. IEEE Signal Process Mag 28(6):80–90

    Article  Google Scholar 

  13. Lin W, Jay Kuo C-C (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312

    Article  Google Scholar 

  14. Lin W, Ebrahimi T, Loizou PC, Moller S, Reibman AR (2012) Introduction to the special issue on new subjective and objective methodologies for audio and visual signal processing. IEEE J Sel Top Signal Process 6(6):614–615

    Article  Google Scholar 

  15. Karni Z, Gotsman C (2000) Spectral compression of mesh geometry. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 279–286

  16. Sorkine O, Cohen-Or D, Toledo S (2003) High-pass quantization for mesh encoding. In: Symposium on geometry processing, vol 42

  17. Pan Y, Cheng I, Basu A (2005) Quality metric for approximating subjective evaluation of 3-d objects. IEEE Trans Multimed 7(2):269–279

    Article  Google Scholar 

  18. Bian Z, Shi-Min H, Martin RR (2009) Evaluation for small visual difference between conforming meshes on strain field. J Comput Sci Technol 24(1):65–75

    Article  Google Scholar 

  19. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  20. Lavoué G, Gelasca ED, Dupont F, Baskurt A, Ebrahimi T (2006) Perceptually driven 3d distance metrics with application to watermarking. In: SPIE optics + photonics. International Society for Optics and Photonics, pp 63120L–63120L

  21. Lavoué G (2011) A multiscale metric for 3d mesh visual quality assessment. In: Computer graphics forum, vol 30, issue 5. Blackwell, Oxford, UK, pp 1427–1437

  22. Torkhani F, Wang K, Chassery J-M (2012) A curvature tensor distance for mesh visual quality assessment. In: Computer vision and graphics, pp 253–263

  23. Corsini M, Gelasca ED, Ebrahimi T, Barni M (2007) Watermarked 3-d mesh quality assessment. IEEE Trans Multimed 9(2):247–256

    Article  Google Scholar 

  24. Váša L, Rus J (2012) Dihedral angle mesh error: a fast perception correlated distortion measure for fixed connectivity triangle meshes. In: Computer graphics forum, vol 31. Wiley Online Library, pp 1715–1724

  25. Wang K, Torkhani F, Montanvert A (2012) A fast roughness-based approach to the assessment of 3d mesh visual quality. Comput Graph 36(7):808–818

    Article  Google Scholar 

  26. Abouelaziz I, El Hassouni M, Cherifi H (2016) No-reference 3d mesh quality assessment based on dihedral angles model and support vector regression. In: International conference on image and signal processing. Springer, pp 369–377

  27. Abouelaziz I, El Hassouni M, Cherifi H (2016) A curvature based method for blind mesh visual quality assessment using a general regression neural network. In: 2016 12th international conference on signal-image technology & internet-based systems (SITIS). IEEE, pp 793–797

  28. Nouri A, Charrier C, Lézoray O (2017) 3d blind mesh quality assessment index. Electron Imaging 2017(20):9–26

    Article  Google Scholar 

  29. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516

    Article  Google Scholar 

  30. Saad MA, Bovik AC, Charrier C (2010) A dct statistics-based blind image quality index. IEEE Signal Process Lett 17(6):583–586

    Article  Google Scholar 

  31. Li C, Bovik AC, Wu X (2011) Blind image quality assessment using a general regression neural network. IEEE Transa Neural Netw 22(5):793–799

    Article  Google Scholar 

  32. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  33. Zhang W, Chenfei Q, Ma L, Guan J, Huang R (2016) Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network. Pattern Recognit 59:176–187

    Article  Google Scholar 

  34. Nouri A, Charrier C, Lézoray O (2016) Full-reference saliency-based 3d mesh quality assessment index. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 1007–1011

  35. Engelke U, Pepion R, Le Callet P, Zepernick H-J (2010) Linking distortion perception and visual saliency in h. 264/AVC coded video containing packet loss. In: Visual communications and image processing 2010. International Society for optics and photonics, vol 7744, p 774406

  36. Lee CH, Varshney A, Jacobs DW (2005) Mesh saliency. ACM Trans Graph: TOG 24:659–666

    Article  Google Scholar 

  37. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  38. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  Google Scholar 

  39. Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814

  40. Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1733–1740

  41. Lavoué G, Larabi MC, Váša L (2016) On the efficiency of image metrics for evaluating the visual quality of 3d models. IEEE Trans Vis Comput Graph 22(8):1987–1999

    Article  Google Scholar 

  42. Silva S, Santos BS, Ferreira C, Madeira J (2009) A perceptual data repository for polygonal meshes. In: Second international conference in visualisation, 2009. VIZ’09. IEEE, pp 207–212

  43. Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lect Image Video Multimed Process 2(1):1–156

    Article  Google Scholar 

  44. Engeldrum PG (2001) Psychometric scaling: avoiding the pitfalls and hazards. In: PICS, pp 101–107

Download references

Author information

Authors and Affiliations

  1. LRIT, URAC No 29, Faculty of Sciences, Mohammed V University in Rabat, B.P. 1014 RP, Rabat, Morocco

    Ilyass Abouelaziz & Mohammed El Hassouni

  2. University of Orleans PRISME Laboratory, Orléans, France

    Aladine Chetouani

  3. FLSHR, Mohammed V University in Rabat, Rabat, Morocco

    Mohammed El Hassouni

  4. Department of Computer and Information Sciences, Temple University, Philadelphia, USA

    Longin Jan Latecki

  5. LE2I, UMR 6306 CNRS, University of Burgundy, Dijon, France

    Hocine Cherifi

Authors
  1. Ilyass Abouelaziz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aladine Chetouani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed El Hassouni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Longin Jan Latecki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hocine Cherifi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilyass Abouelaziz.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abouelaziz, I., Chetouani, A., El Hassouni, M. et al. 3D visual saliency and convolutional neural network for blind mesh quality assessment. Neural Comput & Applic 32, 16589–16603 (2020). https://doi.org/10.1007/s00521-019-04521-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04521-1

Keywords

Search

Navigation