Cited By
View all- Wu CLian DGe YZhou MChen E(2024)Attacking Social Media via Behavior PoisoningACM Transactions on Knowledge Discovery from Data10.1145/365467318:7(1-27)Online publication date: 19-Jun-2024
CreaGAN: An Automatic Creative Generation Framework for Display Advertising
Authors: 
Shiyao Wang, Qi Liu, Yicheng Zhong, Zhilong Zhou, Tiezheng Ge, Defu Lian, Yuning JiangAuthors Info & Claims
Shiyao Wang, Qi Liu, Yicheng Zhong, Zhilong Zhou, Tiezheng Ge, Defu Lian, Yuning JiangAuthors Info & ClaimsPages 7261 - 7269
Abstract
Creatives are an effective form of delivering product information on the E-commerce platform. Designing an exquisite creative is a time-consuming but crucial task for sellers. In order to accelerate the process, we propose an automatic creative generation framework, named CreaGAN, to make the design procedure easier, faster, and more accurate. Given a well-designed creative for one product, our method can generalize to other product materials by utilizing existing design elements (e.g., background material). The framework consists of two major parts: aesthetics-aware placement and creative inpainting model. The placement model aims to generate plausible locations for new products by considering aesthetic principles. And the inpainting model focus on filling the mismatched regions through contextual information. We conduct experiments on both the public dataset and the real-world creative dataset. Quantitative and qualitative results demonstrate that our method outperforms current state-of-the-art methods and obtains more reasonable and aesthetic visualization results.
Supplementary Material
In this work, we propose CreaGAN, an automatic creative generation framework that makes the design procedure easier, faster, and more accurate. Given a well-designed creative for one product, our method can generalize to other product materials by utilizing existing design elements (e.g., background material). The framework consists of two major parts: aesthetics-aware placement and creative inpainting model. The placement model aims to generate plausible locations for new products by considering aesthetic principles. And the inpainting model focus on filling the mismatched regions through contextual information. We conduct experiments on both the public dataset and the real-world creative dataset. Quantitative and qualitative results demonstrate that our method outperforms current state-of-the-art methods and obtains more reasonable and aesthetic visualization results.
- Download
- 159.42 MB
References
[1]
Diego Martin Arroyo, Janis Postels, and Federico Tombari. 2021. Variational transformer networks for layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13642--13652.
[2]
Coloma Ballester, Marcelo Bertalmio, Vicent Caselles, Guillermo Sapiro, and Joan Verdera. 2001. Filling-in by joint interpolation of vector fields and gray levels. TIP, Vol. 10, 8 (2001), 1200--1211.
[3]
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. 2009. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., Vol. 28, 3 (2009), 24.
[4]
Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. 2000. Image inpainting. In SIGGRAPH. 417--424.
[5]
Ying Cao, Antoni B Chan, and Rynson WH Lau. 2012. Automatic stylistic manga layout. ACM Transactions on Graphics (TOG), Vol. 31, 6 (2012), 1--10.
[6]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[7]
Selim Esedoglu and Jianhong Shen. 2002. Digital inpainting based on the Mumford--Shah--Euler image model. European Journal of Applied Mathematics, Vol. 13, 4 (2002), 353--370.
[8]
Jian Fan. 2012. Photo layout with a fast evaluation method and genetic algorithm. In 2012 IEEE International Conference on Multimedia and Expo Workshops. IEEE, 308--313.
[9]
Joe Geigel and Alexander Loui. 2003. Using genetic algorithms for album page layouts. IEEE multimedia, Vol. 10, 4 (2003), 16--27.
[10]
Steven J Harrington, J Fernando Naveda, Rhys Price Jones, Paul Roetling, and Nishant Thakkar. 2004. Aesthetic measures for automated document layout. In Proceedings of the 2004 ACM symposium on Document engineering. 109--111.
[11]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. NIPS, Vol. 30 (2017).
[12]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In CVPR. 1125--1134.
[13]
Charles Jacobs, Wilmot Li, Evan Schrier, David Bargeron, and David Salesin. 2003. Adaptive grid-based document layout. ACM transactions on graphics (TOG), Vol. 22, 3 (2003), 838--847.
[14]
Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. 2021. Constrained graphic layout generation via latent optimization. In Proceedings of the 29th ACM International Conference on Multimedia. 88--96.
[15]
Liang Liao, Jing Xiao, Zheng Wang, Chia-Wen Lin, and Shin'ichi Satoh. 2021. Image Inpainting Guided by Coherence Priors of Semantics and Textures. In CVPR. 6539--6548.
[16]
Chen-Hsuan Lin, Ersin Yumer, Oliver Wang, Eli Shechtman, and Simon Lucey. 2018. St-gan: Spatial transformer generative adversarial networks for image compositing. In CVPR. 9455--9464.
[17]
Guilin Liu, Fitsum A Reda, Kevin J Shih, Ting-Chun Wang, Andrew Tao, and Bryan Catanzaro. 2018. Image inpainting for irregular holes using partial convolutions. In ECCV. 85--100.
[18]
Kui-long Liu, Wei Li, Chang-yuan Yang, and Guang Yang. 2019. Intelligent design of multimedia content in Alibaba. Frontiers of Information Technology & Electronic Engineering, Vol. 20, 12 (2019), 1657--1664.
[19]
Kim-Fung Man, Kit-Sang Tang, and Sam Kwong. 1996. Genetic algorithms: concepts and applications [in engineering design]. IEEE transactions on Industrial Electronics, Vol. 43, 5 (1996), 519--534.
[20]
Kamyar Nazeri, Eric Ng, Tony Joseph, Faisal Z Qureshi, and Mehran Ebrahimi. 2019. Edgeconnect: Generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212 (2019).
[21]
Li Niu, Wenyan Cong, Liu Liu, Yan Hong, Bo Zhang, Jing Liang, and Liqing Zhang. 2021. Making Images Real Again: A Comprehensive Survey on Deep Image Composition. arXiv preprint arXiv:2106.14490 (2021).
[22]
Peter O'Donovan, Aseem Agarwala, and Aaron Hertzmann. 2014. Learning layouts for single-pagegraphic designs. IEEE transactions on visualization and computer graphics, Vol. 20, 8 (2014), 1200--1213.
[23]
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. 2016. Context encoders: Feature learning by inpainting. In CVPR. 2536--2544.
[24]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[25]
Yuhang Song, Chao Yang, Zhe Lin, Xiaofeng Liu, Qin Huang, Hao Li, and C-C Jay Kuo. 2018. Contextual-based image inpainting: Infer, match, and translate. In ECCV. 3--19.
[26]
Fuwen Tan, Crispin Bernier, Benjamin Cohen, Vicente Ordonez, and Connelly Barnes. 2018. Where and who automatic semantic-aware person composition. In WACV. IEEE, 1519--1528.
[27]
Sreekanth Vempati, Korah T Malayil, V Sruthi, and R Sandeep. 2020. Enabling hyper-personalisation: Automated ad creative generation and ranking for fashion e-commerce. In Fashion Recommender Systems. Springer, 25--48.
[28]
Bo Wang, Quan Chen, Min Zhou, Zhiqiang Zhang, Xiaogang Jin, and Kun Gai. 2020. Progressive feature polishing network for salient object detection. In AAAI, Vol. 34. 12128--12135.
[29]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. TIP, Vol. 13, 4 (2004), 600--612.
[30]
Jie Yang, Zhiquan Qi, and Yong Shi. 2020. Learning to incorporate structure knowledge for image inpainting. In AAAI, Vol. 34. 12605--12612.
[31]
Zili Yi, Qiang Tang, Shekoofeh Azizi, Daesik Jang, and Zhan Xu. 2020. Contextual residual aggregation for ultra high-resolution image inpainting. In CVPR. 7508--7517.
[32]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2018. Generative image inpainting with contextual attention. In CVPR. 5505--5514.
[33]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2019. Free-form image inpainting with gated convolution. In ICCV. 4471--4480.
[34]
Lingzhi Zhang, Tarmily Wen, Jie Min, Jiancong Wang, David Han, and Jianbo Shi. 2020. Learning object placement by inpainting for compositional data augmentation. In ECCV. Springer, 566--581.
[35]
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million Image Database for Scene Recognition. TPAMI (2017).
[36]
Jun-Yan Zhu, Philipp Krahenbuhl, Eli Shechtman, and Alexei A Efros. 2015. Learning a discriminative model for the perception of realism in composite images. In ICCV. 3943--3951.
Index Terms
- CreaGAN: An Automatic Creative Generation Framework for Display Advertising
Recommendations
A New Creative Generation Pipeline for Click-Through Rate with Stable Diffusion Model
WWW '24: Companion Proceedings of the ACM Web Conference 2024In online advertising scenario, sellers often create multiple creatives to provide comprehensive demonstrations, making it essential to present the most appealing design to maximize the Click-Through Rate (CTR). However, sellers generally struggle to ...
MP-Trees: A Packing-Based Macro Placement Algorithm for Modern Mixed-Size Designs
In this paper, we present a new multipacking-tree (MP-tree) representation for macro placements to handle modern mixed-size designs with large macros and high chip utilization rates. Based on binary trees, the MP-tree is very efficient, effective, and ...
Supporting computer-aided product form design research with a cognitive model of the creative process
AbstractCompanies need to innovate quickly to adapt to the current rapidly changing market environment. Therefore, methods to support the computer-aided creative design process have become a hot research topic, and a variety of methods to support research ...
Comments
Information & Contributors
Information
Published In
October 2022
7537 pages
ISBN:9781450392037
DOI:10.1145/3503161
- General Chairs:
- João Magalhães,
- Alberto del Bimbo,
- Shin'ichi Satoh,
- Nicu Sebe,
- Program Chairs:
- Xavier Alameda-Pineda,
- Qin Jin,
- Vincent Oria,
- Laura Toni
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 995 of 4,171 submissions, 24%
Upcoming Conference
MM '24
- Sponsor:
- sigmm
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 216Total Downloads
- Downloads (Last 12 months)56
- Downloads (Last 6 weeks)1
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
Cited By
View all- Wu CLian DGe YZhou MChen E(2024)Attacking Social Media via Behavior PoisoningACM Transactions on Knowledge Discovery from Data10.1145/365467318:7(1-27)Online publication date: 19-Jun-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in