Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Advances and Challenges in Computational Image Aesthetics

  • Chapter
  • First Online:
Human Perception of Visual Information

Abstract

Computational image aesthetics aims at designing algorithmic approaches to perform aesthetic decisions, in a similar fashion as humans. In the past fifteen years, computational aesthetics has undergone unprecedented development, thanks to the availability of large annotated datasets and deep learning approaches, impacting many applications in multimedia from image enhancement to recommendation and retrieval. In this chapter, we first overview the several interpretations that aesthetics has received over the centuries and propose a set of suitable dimensions for a taxonomy of computational aesthetic approaches. Then, we present the advances of computational aesthetics in the past decade by providing a critical analysis of the most popular datasets, early methods based on hand-crafted features, and modern approaches using deep neural networks. In the last part of the chapter, we discuss some open challenges in computational aesthetic quality assessment: dealing with the intrinsic subjectivity of the scores, and providing explainable aesthetic predictions. In particular, throughout the chapter, we stress the fundamental importance of data collection in computational aesthetics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.socialreport.com/insights/article/360000094166-The-Latest-Facebook-Statistics-2018.

  2. 2.

    https://blog.youtube/press/.

  3. 3.

    Notice that this relation has become looser in modern and contemporary art, where producing beautiful depictions is often not the primary purpose of the artwork.

  4. 4.

    This sentence is attributed to the nineteenth-century Irish novelist Margaret Hungerford. However, the expression has a much older origin, e.g., see Shakespeare’s Love’s Labour Lost (1588): “Beauty is bought by judgment of the eye“.

References

  • Amirshahi, S. A., Denzler, J., & Redies, C. (2013). Jenaesthetics—a public dataset of paintings for aesthetic research. In Poster workshop at the european conference on computer vision.

    Google Scholar 

  • Aydın, T. O., Smolic, A., & Gross, M. (2014). Automated aesthetic analysis of photographic images. IEEE Transactions on Visualization and Computer Graphics, 21(1), 31–42.

    Article  Google Scholar 

  • Barrett, T. (2020). Criticizing photographs: An introduction to understanding images. Routledge.

    Book  Google Scholar 

  • Bhattacharya, S., Nojavanasghari, B., Chen, T., Liu, D., Chang, S. F., & Shah, M. (2013). Towards a comprehensive computational model for aesthetic assessment of videos. In Proceedings of the 21st ACM international conference on Multimedia (pp. 361–364).

    Google Scholar 

  • Bianco, S., Cadene, R., Celona, L., & Napoletano, P. (2018a). Benchmark analysis of representative deep neural network architectures. IEEE Access, 6, 64270–64277.

    Article  Google Scholar 

  • Bianco, S., Celona, L., & Schettini, R. (2018b). Aesthetics assessment of images containing faces. In 2018 25th IEEE international conference on image processing (ICIP) (pp. 2820–2824). IEEE.

    Google Scholar 

  • Birkhoff, G. D. (1933). Aesthetic measure. Cambridge, MA: Harvard University Press.

    Book  MATH  Google Scholar 

  • Cela-Conde, C. J., Ayala, F. J., Munar, E., Maestú, F., Nadal, M., Capó, M. A., del Río, D., López-Ibor, J. J., Ortiz, T., Mirasso, C., et al. (2009). Sex-related similarities and differences in the neural correlates of beauty. Proceedings of the National Academy of Sciences, 106(10), 3847–3852.

    Article  Google Scholar 

  • Chang, H., Yu, F., Wang, J., Ashley, D., & Finkelstein, A. (2016). Automatic triage for a photo series. ACM Transactions on Graphics (TOG), 35(4), 1–10.

    Article  Google Scholar 

  • Chang, K. Y., Lu, K. H., & Chen, C. S. (2017). Aesthetic critiques generation for photos. In Proceedings of the IEEE international conference on computer vision (pp. 3514–3523).

    Google Scholar 

  • Chatterjee, A. (2003). Prospects for a cognitive neuroscience of visual aesthetics. Bulletin of Psychology and the Arts, 4, 55–60. https://doi.org/10.1037/e514602010-003

    Google Scholar 

  • Chopra, S., Hadsell, R., & LeCun, Y. (2005). Learning a similarity metric discriminatively, with application to face verification. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05) (vol. 1, pp. 539–546). https://doi.org/10.1109/CVPR.2005.202

  • Constantin, M. G., Kang, C., Dinu, G., Dufaux, F., Valenzise, G., & Ionescu, B. (2019). Using aesthetics and action recognition-based networks for the prediction of media memorability. In MediaEval 2019 workshop. France: Sophia Antipolis. https://hal.archives-ouvertes.fr/hal-02368920.

  • Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004). Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV (vol. 1, pp. 1–2). Prague.

    Google Scholar 

  • Cui, C., Fang, H., Deng, X., Nie, X., Dai, H., & Yin, Y. (2017). Distribution-oriented aesthetics assessment for image search. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval (pp. 1013–1016).

    Google Scholar 

  • Cui, C., Yang, W., Shi, C., Wang, M., Nie, X., & Yin, Y. (2020). Personalized image quality assessment with social-sensed aesthetic preference. Information Sciences, 512, 780–794.

    Article  Google Scholar 

  • Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2006). Studying aesthetics in photographic images using a computational approach. In European conference on computer vision (pp. 288–301). Springer.

    Google Scholar 

  • Datta, R., & Wang, J. Z. (2010). ACQUINE: aesthetic quality inference engine-real-time automatic rating of photo aesthetics. In Proceedings of the international conference on multimedia information retrieval (pp. 421–424).

    Google Scholar 

  • Deng, Y., Loy, C.C., & Tang, X. (2017). Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine, 34(4), 80–106.

    Article  Google Scholar 

  • Deng, Y., Loy, C. C., & Tang, X. (2018). Aesthetic-driven image enhancement by adversarial learning. In 2018 ACM multimedia conference on multimedia conference (pp. 870–878). ACM.

    Google Scholar 

  • Dhar, S., Ordonez, V., & Berg, T. L. (2011). High level describable attributes for predicting aesthetics and interestingness. In CVPR 2011 (pp. 1657–1664). IEEE.

    Google Scholar 

  • Ghosal, K., Rana, A., & Smolic., A. (2019). Aesthetic image captioning from weakly-labelled photographs. In ICCV 2019 workshop on cross-modal learning in real world.

    Google Scholar 

  • Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. Preprint. arXiv:1412.6572.

    Google Scholar 

  • Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press.

    MATH  Google Scholar 

  • Götz-Hahn, F., Hosu, V., Lin, H., & Saupe, D. (2019). No-reference video quality assessment using multi-level spatially pooled features. Preprint. arXiv:1912.07966.

    Google Scholar 

  • Guo, G., Wang, H., Shen, C., Yan, Y., & Liao, H. Y. M. (2018). Automatic image cropping for visual aesthetic enhancement using deep neural networks and cascaded regression. IEEE Transactions on Multimedia, 20(8), 2073–2085.

    Article  Google Scholar 

  • Gygli, M., Grabner, H., Riemenschneider, H., Nater, F., & Van Gool, L. (2013). The interestingness of images. In Proceedings of the IEEE international conference on computer vision (pp. 1633–1640).

    Google Scholar 

  • Hayn-Leichsenring, G. U., Lehmann, T., & Redies, C. (2017). Subjective ratings of beauty and aesthetics: correlations with statistical image properties in western oil paintings. i-Perception, 8(3), 2041669517715474.

    Article  Google Scholar 

  • He, J., Wang, L., Zhou, W., Zhang, H., Cui, X., & Guo, Y. (2019). Viewpoint assessment and recommendation for photographing architectures. IEEE Transactions on Visualization and Computer Graphics, 25(8), 2636–2649. https://doi.org/10.1109/TVCG.2018.2853751.

    Article  Google Scholar 

  • Hoenig, F. (2005). Defining computational aesthetics. In Proceedings of the first eurographics conference on computational aesthetics in graphics, visualization and imaging (pp. 13–18).

    Google Scholar 

  • Hoffer, E., & Ailon, N. (2015). Deep metric learning using triplet network. In International workshop on similarity-based pattern recognition (pp. 84–92). Springer.

    Google Scholar 

  • Hulusic, V., Valenzise, G., Provenzi, E., Debattista, K., & Dufaux, F. (2016). Perceived dynamic range of HDR images. In IEEE int. conference on quality of multimedia experience (pp. 1–6).

    Google Scholar 

  • Isola, P., Xiao, J., Torralba, A., & Oliva, A. (2011). What makes an image memorable? In CVPR 2011 (pp. 145–152). IEEE.

    Google Scholar 

  • ITU-R. (2012). Methodology for the subjective assessment of the quality of television pictures. ITU-R Recommendation BT.500-13.

    Google Scholar 

  • Jaakkola, T. S., Haussler, D., et al. (1999). Exploiting generative models in discriminative classifiers. Advances in Neural Information Processing Systems, 487–493.

    Google Scholar 

  • Jin, B., Segovia, M. V. O., & Süsstrunk, S. (2016a). Image aesthetic predictors based on weighted CNNs. In IEEE international conference on image processing (pp. 2291–2295). Phoenix, AZ, USA: IEEE.

    Google Scholar 

  • Jin, X., Chi, J., Peng, S., Tian, Y., Ye, C., & Li, X. (2016b). Deep image aesthetics classification using inception modules and fine-tuning connected layer. In 2016 8th international conference on wireless communications & signal processing (WCSP), (pp. 1–6). IEEE.

    Google Scholar 

  • Jin, X., Wu, L., Li, X., Chen, S., Peng, S., Chi, J., Ge, S., Song, C., & Zhao, G. (2018). Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence. In Thirty-second AAAI conference on artificial intelligence.

    Google Scholar 

  • John, L. K., Mochon, D., Emrich, O., & Schwartz, J. (2017). What’s the value of a like. Harvard Business Review, 95(2), 108–115.

    Google Scholar 

  • Kairanbay, M., See, J., Wong, L. K., & Hii, Y. L. (2017). Filling the gaps: Reducing the complexity of networks for multi-attribute image aesthetic prediction. In Proceedings of the IEEE international conference on image processing (pp. 3051–3055). IEEE.

    Google Scholar 

  • Kairanbay, M., See, J., & Wong, L. K. (2019). Beauty is in the eye of the beholder: Demographically oriented analysis of aesthetics in photographs. ACM Transactions on Multimedia Computing, Communications, and Applications, 15(2s), 1–21. https://doi.org/10.1145/3328993.

    Article  Google Scholar 

  • Kang, C., Valenzise, G., & Dufaux, F. (2019). Predicting Subjectivity in Image Aesthetics Assessment. In: 21st international workshop on multimedia signal processing (MMSP’2019), Kuala Lumpur, Malaysia. https://hal.archives-ouvertes.fr/hal-02191142.

  • Kang, C., Valenzise, G., & Dufaux, F. (2020). EVA: An Explainable Visual Aesthetics Dataset. In: Joint workshop on aesthetic and technical quality assessment of multimedia and media analytics for societal trends (ATQAM/MAST’20). Seattle, USA: ACM Multimedia. https://hal.archives-ouvertes.fr/hal-02934292.

  • Kao, Y., Wang, C., & Huang, K. (2015). Visual aesthetic quality assessment with a regression model. In IEEE international conference on image processing (ICIP) (pp. 1583–1587). IEEE.

    Google Scholar 

  • Kao, Y., Huang, K., & Maybank, S. (2016). Hierarchical aesthetic quality assessment using deep convolutional neural networks. Signal Processing: Image Communication, 47, 500–510.

    Google Scholar 

  • Kao, Y., He, R., & Huang, K. (2017a). Automatic image cropping with aesthetic map and gradient energy map. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 1982–1986). IEEE.

    Google Scholar 

  • Kao, Y., He, R., & Huang, K. (2017b). Deep aesthetic quality assessment with semantic information. IEEE Transactions on Image Processing, 26(3), 1482–1495.

    Article  MathSciNet  MATH  Google Scholar 

  • Ke, Y., Tang, X., & Jing, F. (2006). The design of high-level features for photo quality assessment. In IEEE int. conference on computer vision and pattern recognition (CVPR) (vol. 1, pp. 419–426). IEEE.

    Google Scholar 

  • Kim, W. H., Choi, J. H., & Lee, J. S. (2020). Objectivity and subjectivity in aesthetic quality assessment of digital photographs. IEEE Transactions on Affective Computing, 11(3), 493–506. https://doi.org/10.1109/TAFFC.2018.2809752.

    Article  Google Scholar 

  • Kirk, U., Skov, M., Christensen, M. S., & Nygaard, N. (2009). Brain correlates of aesthetic expertise: a parametric fMRI study. Brain and Cognition, 69(2), 306–315.

    Article  Google Scholar 

  • Kong, S., Shen, X., Lin, Z., Mech, R., & Fowlkes, C. (2016). Photo aesthetics ranking network with attributes and content adaptation. In European conference on computer vision (pp. 662–679). Springer.

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 1097–1105.

    Google Scholar 

  • Kucer, M., Loui, A. C., & Messinger, D. W. (2018). Leveraging expert feature knowledge for predicting image aesthetics. IEEE Transactions on Image Processing, 27(10), 5100–5112.

    Article  MathSciNet  MATH  Google Scholar 

  • Kuzovkin, D. (2019). Assessment of photos in albums based on aesthetics and context. Theses, Université Rennes 1. https://hal.inria.fr/tel-02345620.

  • Kuzovkin, D., Pouli, T., Cozot, R., Meur, O. L., Kervec, J., & Bouatouch, K. (2017). Context-aware clustering and assessment of photo collections. In Proceedings of the symposium on computational aesthetics (pp. 1–10)

    Google Scholar 

  • Lakhal, S., Darmon, A., Bouchaud, J. P., & Benzaquen, M. (2020). Beauty and structural complexity. Physical Review Research, 2(2), 022058.

    Article  Google Scholar 

  • Lebreton, P., Raake, A., & Barkowsky, M. (2016). Evaluation of aesthetic appeal with regard of user’s knowledge. Electronic Imaging, 2016(16), 1–6.

    Article  Google Scholar 

  • Li, C., Loui, A. C., & Chen, T. (2010). Towards aesthetics: A photo quality assessment and photo selection system. In Proceedings of the 18th ACM international conference on Multimedia (pp. 827–830).

    Google Scholar 

  • Li, J., Barkowsky, M., & Le Callet, P. (2013). Boosting paired comparison methodology in measuring visual discomfort of 3dtv: performances of three different designs. In Stereoscopic displays and applications XXIV (vol. 8648, p. 86481V). International Society for Optics and Photonics.

    Google Scholar 

  • Ling, S., Wang, J., Huang, W., Guo, Y., Zhang, L., Jing, Y., & Le Callet, P. (2020). A subjective study of multi-dimensional aesthetic assessment for mobile game image. In Proceedings of the 1st workshop on quality of experience (QoE) in visual multimedia applications (pp. 47–53).

    Google Scholar 

  • Liu, W., & Wang, Z. (2017). A database for perceptual evaluation of image aesthetics. In 2017 IEEE international conference on image processing (ICIP) (pp. 1317–1321). https://doi.org/10.1109/ICIP.2017.8296495.

  • Liu, D., Puri, R., Kamath, N., & Bhattacharya, S. (2020). Composition-aware image aesthetics assessment. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (WACV).

    Google Scholar 

  • Lo, K. Y., Liu, K. H., & Chen, C. S. (2012). Intelligent photographing interface with on-device aesthetic quality assessment. In Asian conference on computer vision (pp. 533–544). Springer.

    Google Scholar 

  • Lowe, D. G.: Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision (vol. 2, pp. 1150–1157). IEEE (1999)

    Google Scholar 

  • Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2014). RAPID: Rating pictorial aesthetics using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 457–466). ACM.

    Google Scholar 

  • Lu, P., Peng, X., Li, R., & Wang, X. (2015a). Towards aesthetics of image: a bayesian framework for color harmony modeling. Signal Processing: Image Communication, 39, 487–498.

    Google Scholar 

  • Lu, X., Lin, Z., Jin, H., Yang, J., & Wang, J. Z. (2015b). Rating image aesthetics using deep learning. IEEE Transactions on Multimedia, 17(11), 2021–2034. https://doi.org/10.1109/TMM.2015.2477040.

    Article  Google Scholar 

  • Lu, X., Lin, Z., Shen, X., Mech, R., & Wang, J. Z. (2015c). Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In Proceedings of the IEEE international conference on computer vision (pp. 990–998)

    Google Scholar 

  • Lu, P., Peng, X., Zhu, X., & Li, R. (2016). An EL-LDA based general color harmony model for photo aesthetics assessment. Signal Processing, 120, 731–745.

    Article  Google Scholar 

  • Luo, Y., & Tang, X. (2008). Photo and video quality evaluation: Focusing on the subject. In European conference on computer vision (pp. 386–399). Springer.

    Google Scholar 

  • Luo, W., Wang, X., & Tang, X. (2011). Content-based photo quality assessment. In 2011 international conference on computer vision (pp. 2206–2213). IEEE.

    Google Scholar 

  • Ma, S., Liu, J., & Wen Chen, C. (2017). A-lamp: Adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4535–4544).

    Google Scholar 

  • Machado, P., & Cardoso, A. (1998). Computing aesthetics. In Brazilian symposium on artificial intelligence (pp. 219–228). Springer.

    Google Scholar 

  • Mai, L., Le, H., Niu, Y., & Liu, F. (2011). Rule of thirds detection from photograph. In 2011 IEEE international symposium on Multimedia (pp. 91–96).

    Google Scholar 

  • Mai, L., Le, H., Niu, Y., Lai, Y. C., & Liu, F. (2012). Detecting rule of simplicity from photos. In Proceedings of the 20th ACM international conference on Multimedia (pp. 1149–1152)

    Google Scholar 

  • Mai, L., Jin, H., & Liu, F. (2016). Composition-preserving deep photo aesthetics assessment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 497–506).

    Google Scholar 

  • Maître, H. (2018). Qu’est-ce qu’une belle photo? Essai sur l’esthétique en photographie numérique. https://hal.archives-ouvertes.fr/hal-01864135/.

  • Marchesotti, L., Perronnin, F., & Meylan, F. (2013). Learning beautiful (and ugly) attributes. In BMVC (vol. 7, pp. 1–11).

    Google Scholar 

  • Marchesotti, L., Perronnin, F., Larlus, D., & Csurka, G. (2011). Assessing the aesthetic quality of photographs using generic image descriptors. In IEEE international conference on computer vision (pp. 1784–1791). IEEE.

    Google Scholar 

  • Moon, P., & Spencer, D. (1944). Geometric formulation of classical color harmony. Journal of the Optical Society of America (1917-1983), 34(1), 46.

    Article  Google Scholar 

  • Murray, N., & Gordo, A. (2017). A deep architecture for unified aesthetic prediction. Preprint. arXiv:1708.04890.

    Google Scholar 

  • Murray, N., Marchesotti, L., & Perronnin, F. (2012). AVA: a large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition, (pp. 2408–2415). IEEE.

    Google Scholar 

  • Park, T. S., & Zhang, B. T. (2015). Consensus analysis and modeling of visual aesthetic perception. IEEE Transactions on Affective Computing, 6(3), 272–285.

    Article  Google Scholar 

  • Park, K., Hong, S., Baek, M., & Han, B. (2017). Personalized image aesthetic quality assessment by joint regression and ranking. In 2017 IEEE winter conference on applications of computer vision (WACV) (pp. 1206–1214). IEEE.

    Google Scholar 

  • Pele, O., & Werman, M. (2010). The quadratic-chi histogram distance family. In European conference on computer vision (pp. 749–762). Springer.

    Google Scholar 

  • Perez-Ortiz, M., Mikhailiuk, A., Zerman, E., Hulusic, V., Valenzise, G., & Mantiuk, R. (2019). From pairwise comparisons and rating to a unified quality scale. IEEE Transactions on Image Processing, 29, 1139–1151. https://doi.org/10.1109/TIP.2019.2936103. https://hal.archives-ouvertes.fr/hal-02400863.

  • Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8(4), 364–382.

    Article  Google Scholar 

  • Redi, M., Rasiwasia, N., Aggarwal, G., & Jaimes, A. (2015). The beauty of capturing faces: Rating the quality of digital portraits. In 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (vol. 1, pp. 1–8). IEEE.

    Google Scholar 

  • Redi, M., Crockett, D., Manovich, L., & Osindero, S. (2016). What makes photo cultures different? In Proceedings of the 24th ACM international conference on Multimedia (pp. 287–291).

    Google Scholar 

  • Ren, J., Shen, X., Lin, Z., Mech, R., & Foran, D. J. (2017). Personalized image aesthetics. In Proceedings of the IEEE international conference on computer vision (pp. 638–647).

    Google Scholar 

  • Ribeiro, F., Florêncio, D., Zhang, C., & Seltzer, M. (2011). Crowdmos: An approach for crowdsourcing mean opinion score studies. In IEEE international conference on acoustics, speech and signal processing (pp. 2416–2419). IEEE.

    Google Scholar 

  • Rigau, J., Feixas, M., & Sbert, M. (2008). Informational aesthetics measures. IEEE Computer Graphics and Applications, 28(2), 24–34.

    Article  Google Scholar 

  • Rosenblum, N. (2008). A world history of photography. New York, USA: Abbeville Press.

    Google Scholar 

  • San Pedro, J., Yeh, T., & Oliver, N. (2012). Leveraging user comments for aesthetic aware image search reranking. In Proceedings of the 21st international conference on World Wide Web (pp. 439–448).

    Google Scholar 

  • Schifanella, R., Redi, M., & Aiello, L. M. (2015). An image is worth more than a thousand favorites: Surfacing the hidden beauty of flickr pictures. In ICWSM’15: Proceedings of the 9th AAAI international conference on weblogs and social media. AAAI.

    Google Scholar 

  • Schwarz, K., Wieschollek, P., & Lensch, H. P. A. (2018). Will people like your image? learning the aesthetic space. In 2018 IEEE winter conference on applications of computer vision (WACV) (pp. 2048–2057). https://doi.org/10.1109/WACV.2018.00226.

  • Selvaraju, R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618–626).

    Google Scholar 

  • Sheng, K., Dong, W., Huang, H., Ma, C., & Hu, B. G. (2018a). Gourmet photography dataset for aesthetic assessment of food images. In SIGGRAPH Asia 2018 technical briefs (pp. 1–4).

    Google Scholar 

  • Sheng, K., Dong, W., Ma, C., Mei, X., Huang, F., & Hu, B. G. (2018b). Attention-based multi-patch aggregation for image aesthetic assessment. In Proceedings of the 26th ACM international conference on Multimedia (pp. 879–886).

    Google Scholar 

  • Siahaan, E., Hanjalic, A., & Redi, J. (2016). A reliable methodology to collect ground truth data of image aesthetic appeal. IEEE Transactions on Multimedia, 18(7), 1338–1350. https://doi.org/10.1109/TMM.2016.2559942.

    Article  Google Scholar 

  • Simond, F., Arvanitopoulos, N., & Süsstrunk, S. (2015). Image aesthetics depends on context. In IEEE international conference on image processing (ICIP) (pp. 3788–3792). IEEE.

    Google Scholar 

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint. arXiv:1409.1556.

    Google Scholar 

  • Song, Y., Redi, M., Vallmitjana, J., & Jaimes, A. (2016). To click or not to click: Automatic selection of beautiful thumbnails from videos. In Proceedings of the 25th ACM international on conference on information and knowledge management (pp. 659–668).

    Google Scholar 

  • Spape, M., Davis, K., Kangassalo, L., Ravaja, N., Sovijarvi-Spape, Z., & Ruotsalo, T. (2021). Brain-computer interface for generating personally attractive images. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2021.3059043.

  • Stollenga, M. F., Masci, J., Gomez, F., & Schmidhuber, J. (2014). Deep networks with internal selective attention through feedback connections. In Proceedings of the 27th international conference on neural information processing systems (vol. 2, pp. 3545–3553).

    Google Scholar 

  • Suchecki, M., & Trzciski, T. (2017). Understanding aesthetics in photography using deep convolutional neural networks. In 2017 Signal processing: Algorithms, architectures, arrangements, and applications (SPA) (pp. 149–153). https://doi.org/10.23919/SPA.2017.8166855.

  • Sun, W., Chao, T., Kuo, Y., & Hsu, W. H. (2017). Photo filter recommendation by category-aware aesthetic learning. IEEE Transactions on Multimedia, 19(8), 1870–1880. https://doi.org/10.1109/TMM.2017.2688929.

    Article  Google Scholar 

  • Talebi, H., & Milanfar, P. (2018). NIMA: Neural image assessment. IEEE Transactions on Image Processing, 27(8), 3998–4011.

    Article  MathSciNet  MATH  Google Scholar 

  • Tang, X., Luo, W., & Wang, X. (2013). Content-based photo quality assessment. IEEE Transactions on Multimedia, 15(8), 1930–1943.

    Article  Google Scholar 

  • Tifentale, A., & Manovich, L. (2018). Competitive photography and the presentation of the self. In Exploring the selfie (pp. 167–187). Springer.

    Google Scholar 

  • Wang, W., Zhao, M., Wang, L., Huang, J., Cai, C., & Xu, X. (2016). A multi-scene deep learning model for image aesthetic evaluation. Signal Processing: Image Communication, 47, 511–518.

    Google Scholar 

  • Wang, Z., Liu, D., Chang, S., Dolcos, F., Beck, D., & Huang, T. (2017). Image aesthetics assessment using Deep Chatterjee’s machine. In International joint conference on neural networks (IJCNN) (pp. 941–948). https://doi.org/10.1109/IJCNN.2017.7965953.

  • Wang, W., Yang, S., Zhang, W., & Zhang, J. (2019). Neural aesthetic image reviewer. IET Computer Vision, 13(8), 749–758.

    Article  Google Scholar 

  • Wu, O., Hu, W., & Gao, J. (2011). Learning to predict the perceived visual quality of photos. In International conference on computer vision (pp. 225–232). https://doi.org/10.1109/ICCV.2011.6126246.

  • Xu, M., Chen, F., Li, L., Shen, C., Lv, P., Zhou, B., & Ji, R. (2018). Bio-inspired deep attribute learning towards facial aesthetic prediction. IEEE Transactions on Affective Computing, 227–238.

    Google Scholar 

  • Xu, Y., Zhang, N., Wei, P., Sang, G., Li, L., & Yuan, F. (2020). Deep neural framework with visual attention and global context for predicting image aesthetics. IEEE Access, 1–1. https://doi.org/10.1109/ACCESS.2020.3015060.

  • Ye, P., & Doermann, D. (2014). Active sampling for subjective image quality assessment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4249–4256).

    Google Scholar 

  • Yeh, H. H., Yang, C. Y., Lee, M. S., & Chen, C. S. (2013). Video aesthetic quality assessment by temporal integration of photo-and motion-based features. IEEE Transactions on Multimedia, 15(8), 1944–1957.

    Article  Google Scholar 

  • Yu, J., Cui, C., Geng, L., Ma, Y., & Yin, Y. (2019). Towards unified aesthetics and emotion prediction in images. In 2019 IEEE international conference on image processing (ICIP) (pp. 2526–2530). https://doi.org/10.1109/ICIP.2019.8803388.

  • Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818–833). Springer.

    Google Scholar 

  • Zerman, E., Hulusic, V., Valenzise, G., Mantiuk, R., & Dufaux, F. (2018). The relation between MOS and pairwise comparisons and the importance of cross-content comparisons. In Human vision and electronic imaging conference, IS&T international symposium on electronic imaging, Burlingame, USA. https://hal.archives-ouvertes.fr/hal-01654133.

  • Zerman, E., Rana, A., & Smolic, A. (2019). Colornet-estimating colorfulness in natural images. In IEEE international conference on image processing (pp. 3791–3795). IEEE.

    Google Scholar 

  • Zhang, F., Wang, M., & Hu, S. (2013). Aesthetic image enhancement by dependence-aware object recomposition. IEEE Transactions on Multimedia, 15(7), 1480–1490. https://doi.org/10.1109/TMM.2013.2268051.

    Article  Google Scholar 

  • Zhang, L., Gao, Y., Zhang, C., Zhang, H., Tian, Q., & Zimmermann, R. (2014). Perception-guided multimodal feature fusion for photo aesthetics assessment. In Proceedings of the 22nd ACM international conference on Multimedia (pp. 237–246).

    Google Scholar 

  • Zhang, C., Zhu, C., Xu, X., Liu, Y., Xiao, J., & Tillo, T. (2018). Visual aesthetic understanding: Sample-specific aesthetic classification and deep activation map visualization. Signal Processing: Image Communication, 67, 12–21.

    Google Scholar 

  • Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016). Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2921–2929).

    Google Scholar 

  • Zuckert, R. (2007). Kant on Beauty and Biology: An Interpretation of the ‘Critique of Judgment’. Cambridge University Press.

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des Signaux et Systèmes, Gif-sur-Yvette, France

    Giuseppe Valenzise, Chen Kang & Frédéric Dufaux

Authors
  1. Giuseppe Valenzise
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frédéric Dufaux
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giuseppe Valenzise .

Editor information

Editors and Affiliations

  1. Politehnica University of Bucharest, Bucharest, Romania

    Bogdan Ionescu

  2. University of Chicago, Chicago, IL, USA

    Wilma A. Bainbridge

  3. NAVER Labs Europe, Meylan, France

    Naila Murray

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Valenzise, G., Kang, C., Dufaux, F. (2022). Advances and Challenges in Computational Image Aesthetics. In: Ionescu, B., Bainbridge, W.A., Murray, N. (eds) Human Perception of Visual Information. Springer, Cham. https://doi.org/10.1007/978-3-030-81465-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-81465-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-81464-9

  • Online ISBN: 978-3-030-81465-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation