research-article

An EL-LDA based general color harmony model for photo aesthetics assessment

Authors:

Ruifan LiAuthors Info & Claims

Signal Processing, Volume 120, Issue C

Pages 731 - 745

https://doi.org/10.1016/j.sigpro.2014.12.008

Published: 01 March 2016 Publication History

Abstract

The goal of photo aesthetics assessment is to build a computational model which can estimate the aesthetics quality of digital images with respect to human's perception. As one of the most important features that determine the degree of image's aesthetics quality, color harmony has gained increasing attentions. To overcome the problems of most classical color harmony models, which are heavily relied on heuristic rules and ignore the semantic information of images, we propose a statistical learning framework in this paper to train a color harmony model from a large number of natural images. In this framework, the semantic label information, which indicates the content of each image, along with the visual features is used to facilitate the latent Dirichlet allocation (LDA) training. Then, the degree of color harmony can be estimated by using supervised/unsupervised models, which is applied to indicate the photo's aesthetics score. By using the proposed color harmony model, we attempt to uncover the underlying principles that generate pleasing color combinations based on natural images. The experimental results show that the proposed approach outperforms the conventional heuristic color harmony models for image aesthetics assessment. HighlightsA novel framework is proposed to learn color harmony for photo aesthetics assessment.An extended labeled-LDA model is presented to learn the complex color combinations.A thorough analysis of different color spaces for color harmony learning is given.The proposed method outperforms the existing color harmony models.

References

[1]

N. Murray, L. Marchesotti, F. Perronnin, Learning to rank images using semantic and aesthetic labels, in: 23rd British Machine and Vision Conference (BMVC), 2012.

[2]

B. Ni, M. Xu, B. Cheng, M. Wang, S. Yan, Q. Tian, Learning to photograph, IEEE Trans. Multimed., 15 (2013) 1138-1151.

Digital Library

[3]

L. Zhang, Y. Gao, Y. Xia, Q. Dai, X. Li, A fine-grained image categorization system by cellet-encoded spatial pyramid modeling, IEEE Trans. Ind. Electron. 62(1) (2015) 564-571.

[4]

L. Zhang, M. Song, X. Liu, J. Bu, C. Chen, Fast multi-view segment graph kernel for object classification, Signal Process., 93 (2013) 1597-1607.

Digital Library

[5]

M. Wang, Y. Gao, K. Lu, Y. Rui, View-based discriminative probabilistic modeling for 3d object retrieval and recognition, IEEE Trans. Image Process., 22 (2013) 1395-1407.

Digital Library

[6]

L. Zhang, Y. Gao, C. Hong, Y. Feng, J. Zhu, D. Cai, Feature correlation hypergraph, IEEE Trans. Cybern., 44 (2014) 1408-1419.

[7]

M. Wang, R. Hong, X.-T. Yuan, S. Yan, T.-S. Chua, Movie2comics, IEEE Trans. Multimed., 14 (2012) 858-870.

Digital Library

[8]

N. Damera-Venkata, T. Kite, W. Geisler, B. Evans, A. Bovik, Image quality assessment based on a degradation model, IEEE Trans. Image Process., 9 (2000) 636-650.

Digital Library

[9]

X. Li, Blind image quality assessment, in: Proceedings of the International Conference on Image Processing, vol. 1, 2002, pp. 449-452.

[10]

R. Datta, D. Joshi, J. Li, J. Z. Wang, Studying aesthetics in photographic images using a computational approach, in: 9th European Conference on Computer Vision, Lecture Notes in Computer Science, vol. 3953, 2006, pp. 288-301.

Digital Library

[11]

Y. Ke, X. Tang, F. Jing, The design of high-level features for photo quality assessment, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 2006, pp. 419-426.

Digital Library

[12]

A. Saha, Q.J. Wu, Perceptual image quality assessment using phase deviation sensitive energy features, Signal Process., 93 (2013) 3182-3191.

Digital Library

[13]

L. Zhang, Y. Gao, R. Zimmermann, Q. Tian, X. Li, Fusion of multichannel local and global structural cues for photo aesthetics evaluation, IEEE Trans. Image Process., 23 (2014) 1419-1429.

Digital Library

[14]

C. Li, T. Chen, Aesthetic visual quality assessment of paintings, J. Sel. Top. Signal Process., 3 (2009) 236-252.

[15]

Y. Luo, X. Tang, Photo and video quality evaluation: focusing on the subject, in: Proceedings of the 10th European Conference on Computer Vision: Part III, 2008, pp. 386-399.

Digital Library

[16]

W. Luo, X. Wang, X. Tang, Content-based photo quality assessment, in: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 2206-2213.

Digital Library

[17]

X. Tang, W. Luo, X. Wang, Content-based photo quality assessment, IEEE Trans. Multimed., 15 (2013) 1930-1943.

Digital Library

[18]

J. Zhang, T.M. Le, S. Ong, T.Q. Nguyen, No-reference image quality assessment using structural activity, Signal Process., 91 (2011) 2575-2588.

Digital Library

[19]

L. Zhang, Y. Han, Y. Yang, M. Song, S. Yan, Q. Tian, Discovering discriminative graphlets for aerial image categories recognition, IEEE Trans. Image Process., 22 (2013) 5071-5084.

Digital Library

[20]

M. Wang, X.-S. Hua, J. Tang, R. Hong, Beyond distance measurement, IEEE Trans. Multimed., 11 (2009) 465-476.

Digital Library

[21]

M. Wang, B. Ni, X.-S. Hua, T.-S. Chua, Assistive tagging, ACM Comput. Surv., 44 (2012).

Digital Library

[22]

L. Zhang, M. Song, X. Liu, L. Sun, C. Chen, J. Bu, Recognizing architecture styles by hierarchical sparse coding of blocklets, Inf. Sci., 254 (2014) 141-154.

Digital Library

[23]

Y. Wang, Q. Dai, R. Feng, Y.-G. Jiang, Beauty is here: evaluating aesthetics in videos using multimodal features and free training data, in: Proceedings of ACM Multimedia, 2013.

Digital Library

[24]

P. Lu, Z. Kuang, X. Peng, R. Li, Discovering harmony: a hierarchical colour harmony model for aesthetic assessment, in: 12th Asian Conference on Computer Vision, 2014.

[25]

S. Westland, K. Laycock, V. Cheung, P. Henry, F. Mahyar, Colour harmony, J. Int. Colour Assoc., 1 (2007) 1-15.

[26]

J. Itten, The Art of Color: The Subjective Experience and Objective Rationale of Color, John Wiley & Sons, Hoboken, NJ, USA, 1997.

[27]

Y. Matsuda, Color Design, Asakura Shoten, Tokyo, 1995.

[28]

A.H. Munsell, A Grammar of Color: A Basic Treatise on the Color System, Van Nostrand Reinhold Co., New York, USA, 1969.

[29]

P. Moon, D.E. Spencer, Geometric formulation of classical color harmony, J. Opt. Soc. Am., 34 (1944) 46-50.

[30]

A. Hård, L. Sivik, A theory of colors in combination descriptive model related to the ncs color-order system, Color Res. Appl., 26 (2001) 4-28.

[31]

A. Pope, Notes on the problem of color harmony and the geometry of color space, J. Opt. Soc. Am., 34 (1944) 759-765.

[32]

M. Nishiyama, T. Okabe, I. Sato, Y. Sato, Aesthetic quality classification of photographs based on color harmony, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 33-40.

Digital Library

[33]

D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, Y.-Q. Xu, Color harmonization, ACM Trans. Graph., 25 (2006) 624-630.

Digital Library

[34]

Z. Tang, Z. Miao, Y. Wan, Z. Wang, Color harmonization for images, J. Electron. Imaging, 20 (2011).

[35]

P. O'Donovan, A. Agarwala, A. Hertzmann, Color compatibility from large datasets, ACM Trans. Graph., 30 (2011).

Digital Library

[36]

L. Holtzschue, Understanding Color: An Introduction for Designers, John Wiley & Sons Inc., Hoboken, NJ, USA, 2011.

Digital Library

[37]

D.M. Blei, A.Y. Ng, M.I. Jordan, Latent Dirichlet allocation, J. Mach. Learn. Res., 3 (2003) 993-1022.

Digital Library

[38]

N. Murray, L. Marchesotti, F. Perronnin, Ava: a large-scale database for aesthetic visual analysis, in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 2408-2415.

Digital Library

[39]

D. Ramage, D. Hall, R. Nallapati, C. D. Manning, Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora, in: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol. 1, 2009, pp. 248-256.

Digital Library

[40]

Z. O'Connor, Colour harmony revisited, Color Res. Appl., 35 (2010) 267-273.

[41]

Z. Kuang, P. Lu, X. Wang, Learning self-adaptive color harmony model for aesthetic quality classification, in: 6th International Conference on Graphic and Image Processing, 2014.

[42]

P. Li, M. Wang, J. Cheng, C. Xu, H. Lu, Spectral hashing with semantically consistent graph for image indexing, IEEE Trans. Multimed., 15 (2013) 141-152.

Digital Library

Cited By

Yang BWei TLi FLiang XDeng ZFang Y(2024)Color Theme Evaluation through User Preference ModelingACM Transactions on Applied Perception10.1145/366532921:3(1-35)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3665329
Lu PZhang HPeng XJin X(2021)Learning the Relation Between Interested Objects and Aesthetic Region for Image CroppingIEEE Transactions on Multimedia10.1109/TMM.2020.302988223(3618-3630)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1109/TMM.2020.3029882
Lu PPeng XYuan CLi RWang X(2016)Image color harmony modeling through neighbored co-occurrence colorsNeurocomputing10.1016/j.neucom.2016.03.035201:C(82-91)Online publication date: 12-Aug-2016
https://dl.acm.org/doi/10.1016/j.neucom.2016.03.035

Recommendations

A method for cloth-color match for skin based on computer-aided color harmony
MS'06: Proceedings of the 17th IASTED international conference on Modelling and simulation

The main purpose of this study is to discuss the relationships of color combination between skin and cloth colors based on the aesthetics measures. The definition of color harmony is summarized first and aesthetics measurement is then used to accomplish ...
Towards aesthetics of image

Color harmony is one of the most important features that determine the aesthetics quality of images. The existing color harmony models can be roughly divided into two groups: empirical based models defined by artists and designers and learning based ...
Color Harmony Based on Fitting Functions
IFITA '09: Proceedings of the 2009 International Forum on Information Technology and Applications - Volume 01

Color harmony for digital image is an interesting research topic. In this paper, we present a novel algorithm for color harmony based on fitting functions. In order to specify the standard and the style of color harmony, a reference image and its ...

Comments

Information & Contributors

Information

Published In

cover image Signal Processing

Signal Processing Volume 120, Issue C

March 2016

824 pages

ISSN:0165-1684

Issue’s Table of Contents

Copyright © Elsevier B.V.

Publisher

Elsevier North-Holland, Inc.

United States

Publication History

Published: 01 March 2016

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang BWei TLi FLiang XDeng ZFang Y(2024)Color Theme Evaluation through User Preference ModelingACM Transactions on Applied Perception10.1145/366532921:3(1-35)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3665329
Lu PZhang HPeng XJin X(2021)Learning the Relation Between Interested Objects and Aesthetic Region for Image CroppingIEEE Transactions on Multimedia10.1109/TMM.2020.302988223(3618-3630)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1109/TMM.2020.3029882
Lu PPeng XYuan CLi RWang X(2016)Image color harmony modeling through neighbored co-occurrence colorsNeurocomputing10.1016/j.neucom.2016.03.035201:C(82-91)Online publication date: 12-Aug-2016
https://dl.acm.org/doi/10.1016/j.neucom.2016.03.035

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents