research-article

Layout Generation for Various Scenarios in Mobile Shopping Applications

Authors:

Yichun DuAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 130, Pages 1 - 18

https://doi.org/10.1145/3544548.3581446

Published: 19 April 2023 Publication History

Abstract

Layout is essential for the product listing pages (PLPs) in mobile shopping applications. To clearly convey the information that consumers require and to achieve specific functions, PLPs layouts often have many variations driven by scenarios. In this work, we study the PLPs layout design for different scenarios and propose a design space to guide the large-scale creation of PLPs. We propose LayoutVQ-VAE, a novel model specialized in generating layouts with internal and external constraints. LayoutVQ-VAE differs from previous methods as it learns a discrete latent representation of layout and can model the relationship between layout representation and scenarios without applying heuristics. Experiments on publicly available benchmarks for different layout types validate that our method performs comparably or favorably against the state-of-the-art methods. Case studies show that the proposed approach including the design space and model is effective in producing large-scale high-quality PLPs layouts for mobile shopping platforms.

Supplementary Material

Supplemental Materials (3544548.3581446-supplemental-materials.zip)

Download
1.32 MB

MP4 File (3544548.3581446-video-figure.mp4)

Video Figure

Download
103.53 MB

MP4 File (3544548.3581446-talk-video.mp4)

Pre-recorded Video Presentation

Download
101.53 MB

References

[1]

[n. d.]. Sketch. https://www.sketch.com/

[2]

Diego Martin Arroyo, Janis Postels, and Federico Tombari. 2021. Variational Transformer Networks for Layout Generation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Nashville, TN, USA, 13637–13647. https://doi.org/10.1109/CVPR46437.2021.01343

[3]

Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, and Magy Seif Seif El-Nasr. 2021. VINS: Visual Search for Mobile User Interface Design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–14. https://doi.org/10.1145/3411764.3445762

Digital Library

[4]

Niraj Ramesh Dayama, Kashyap Todi, Taru Saarelainen, and Antti Oulasvirta. 2020. GRIDS: Interactive Layout Design with Integer Programming. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376553

Digital Library

[5]

Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017. Rico: A Mobile App Dataset for Building Data-Driven Design Applications. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 845–854. https://doi.org/10.1145/3126594.3126651

Digital Library

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).

[7]

Christoph Goller and Andreas Kuchler. 1996. Learning task-dependent distributed representations by backpropagation through structure. In Proceedings of International Conference on Neural Networks (ICNN’96), Vol. 1. IEEE, 347–352.

[8]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.

Digital Library

[9]

Shunan Guo, Zhuochen Jin, Fuling Sun, Jingwen Li, Zhaorui Li, Yang Shi, and Nan Cao. 2021. Vinci: An Intelligent Graphic Design System for Generating Advertising Posters. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1–17. https://doi.org/10.1145/3411764.3445117

Digital Library

[10]

Kamal Gupta, Justin Lazarow, Alessandro Achille, Larry Davis, Vijay Mahadevan, and Abhinav Shrivastava. 2021. LayoutTransformer: Layout Generation and Completion with Self-attention. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Montreal, QC, Canada, 984–994. https://doi.org/10.1109/ICCV48922.2021.00104

[11]

Ruohong Hao, Bingjia Shao, and Rong Ma. 2019. Impacts of Video Display on Purchase Intention for Digital and Home Appliance Products—Empirical Study from China. Future Internet 11, 11 (Nov. 2019), 224. https://doi.org/10.3390/fi11110224 Number: 11 Publisher: Multidisciplinary Digital Publishing Institute.

[12]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

Digital Library

[13]

Weiyin Hong, James YL Thong, and Kar Yan Tam. 2004. Designing product listing pages on e-commerce websites: an examination of presentation mode and information format. International Journal of Human-Computer Studies 61, 4 (2004), 481–503. Number: 4 Publisher: Elsevier.

Digital Library

[14]

WEIYIN HONG, JAMES Y.L. THONG, and KAR YAN TAM. 2004. The Effects of Information Format and Shopping Task on Consumers’ Online Shopping Behavior: A Cognitive Fit Perspective. Journal of Management Information Systems 21, 3 (2004), 149–184. https://doi.org/10.1080/07421222.2004.11045812 arXiv:https://doi.org/10.1080/07421222.2004.11045812

Digital Library

[15]

Zhenhui Jiang and Izak Benbasat. 2007. Research note—investigating the influence of the functional mechanisms of online product presentations. Information Systems Research 18, 4 (2007), 454–470. https://doi.org/10.1287/isre.1070.0124

Digital Library

[16]

Zhaoyun Jiang, Shizhao Sun, Jihua Zhu, Jian-Guang Lou, and Dongmei Zhang. 2022. Coarse-to-Fine Generative Modeling for Graphic Layouts. Proceedings of the AAAI Conference on Artificial Intelligence 36, 1 (June 2022), 1096–1103. https://doi.org/10.1609/aaai.v36i1.19994

[17]

Akash Abdu Jyothi, Thibaut Durand, Jiawei He, Leonid Sigal, and Greg Mori. 2019. LayoutVAE: Stochastic Scene Layout Generation From a Label Set. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

[18]

Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, and Noam Shazeer. [n. d.]. Fast Decoding in Sequence Models Using Discrete Latent Variables. ([n. d.]), 10.

[19]

Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. 2021. Constrained Graphic Layout Generation via Latent Optimization. In Proceedings of the 29th ACM International Conference on Multimedia. ACM, Virtual Event China, 88–96. https://doi.org/10.1145/3474085.3475497

Digital Library

[20]

Minjeong Kim and Sharron Lennon. 2008. The effects of visual and verbal information on attitudes and purchase intentions in internet shopping. Psychology & Marketing 25, 2 (Feb. 2008), 146–178. https://doi.org/10.1002/mar.20204 Publisher: John Wiley & Sons, Ltd.

[21]

Seeun Kim, Tae Hyun Baek, and Sukki Yoon. 2020. The effect of 360-degree rotatable product images on purchase intention. Journal of Retailing and Consumer Services 55 (July 2020), 102062. https://doi.org/10.1016/j.jretconser.2020.102062

[22]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013).

[23]

Xiang Kong, Lu Jiang, Huiwen Chang, Han Zhang, Yuan Hao, Haifeng Gong, and Irfan Essa. 2022. BLT: Bidirectional Layout Transformer for Controllable Layout Generation. http://arxiv.org/abs/2112.05112 arXiv:2112.05112 [cs].

[24]

Hsin-Ying Lee, Lu Jiang, Irfan Essa, Phuong B. Le, Haifeng Gong, Ming-Hsuan Yang, and Weilong Yang. 2020. Neural Design Network: Graphic Layout Generation with Constraints. In Computer Vision – ECCV 2020(Lecture Notes in Computer Science), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 491–506. https://doi.org/10.1007/978-3-030-58580-8_29

Digital Library

[25]

Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, and Tingfa Xu. 2019. LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators. https://doi.org/10.48550/arXiv.1901.06767 arXiv:1901.06767 [cs].

[26]

Jianan Li, Jimei Yang, Jianming Zhang, Chang Liu, Christina Wang, and Tingfa Xu. 2021. Attribute-Conditioned Layout GAN for Automatic Graphic Design. IEEE Transactions on Visualization and Computer Graphics 27, 10 (Oct. 2021), 4039–4048. https://doi.org/10.1109/TVCG.2020.2999335

Digital Library

[27]

Mengxiang Li, Kwok-Kee Wei, Giri Kumar Tayi, and Chuan-Hoo Tan. 2016. The moderating role of information load on online product presentation. Information & Management 53, 4 (June 2016), 467–480. https://doi.org/10.1016/j.im.2015.11.002

Digital Library

[28]

Gerald L Lohse and Peter Spiller. 1998. Electronic shopping. Commun. ACM 41, 7 (1998), 81–87.

Digital Library

[29]

Naresh K. Malhotra. 1982. Information Load and Consumer Decision Making. Journal of Consumer Research 8, 4 (03 1982), 419–430. https://doi.org/10.1086/208882 arXiv:https://academic.oup.com/jcr/article-pdf/8/4/419/5068452/8-4-419.pdf

[30]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d’ Alché-Buc, E. Fox, and R. Garnett (Eds.). Vol. 32. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf

Digital Library

[31]

Akshay Gadi Patil, Omri Ben-Eliezer, Or Perel, and Hadar Averbuch-Elor. 2020. READ: Recursive Autoencoders for Document Layout Generation. 544–545. https://openaccess.thecvf.com/content_CVPRW_2020/html/w34/Patil_READ_ Recursive_Autoencoders_for_Document_Layout_Generation_CVPRW_2020_paper.html

[32]

Steven F. Roth, John Kolojejchick, Joe Mattis, and Jade Goldstein. 1994. Interactive graphic design using automatic presentation knowledge. In Proceedings of the SIGCHI conference on Human factors in computing systems. 112–117.

Digital Library

[33]

Reijo Savolainen. 2009. Information use and information processing: Comparison of conceptualizations. Journal of Documentation 65, 2 (Jan. 2009), 187–207. https://doi.org/10.1108/00220410910937570 Publisher: Emerald Group Publishing Limited.

[34]

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The Graph Neural Network Model. IEEE Transactions on Neural Networks 20, 1 (2009), 61–80. https://doi.org/10.1109/TNN.2008.2005605

Digital Library

[35]

Maria Sicilia and Salvador Ruiz. 2010. The effects of the amount of information on cognitive responses in online purchasing tasks. Electronic Commerce Research and Applications 9, 2 (March 2010), 183–191. https://doi.org/10.1016/j.elerap.2009.03.004

Digital Library

[36]

Mary Stribley. 10. Rules of Composition All Designers Live By. Retrieved May 23(10), 2016.

[37]

Aaron Van Den Oord and Oriol Vinyals. 2017. Neural discrete representation learning. Advances in neural information processing systems 30 (2017).

[38]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

[39]

Kota Yamaguchi. 2021. CanvasVAE: Learning To Generate Vector Graphic Documents. 5481–5489. https://openaccess.thecvf.com/content/ICCV2021/html/Yamaguchi_CanvasVAE_Learning_ To_Generate_Vector_Graphic_Documents_ICCV_2021_paper.html

[40]

Wanxian Zeng and Alex Richardson. 2016. Adding Dimension to Content: Immersive Virtual Reality for e-Commerce. ACIS 2016 Proceedings (Jan. 2016). https://aisel.aisnet.org/acis2016/24

[41]

Xinru Zheng, Xiaotian Qiao, Ying Cao, and Rynson W. H. Lau. 2019. Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics 38, 4 (Aug. 2019), 1–15. https://doi.org/10.1145/3306346.3322971

Digital Library

[42]

Xu Zhong, Jianbin Tang, and Antonio Jimeno Yepes. 2019. PubLayNet: Largest Dataset Ever for Document Layout Analysis. In 2019 International Conference on Document Analysis and Recognition (ICDAR). 1015–1022. https://doi.org/10.1109/ICDAR.2019.00166

Cited By

Chen LJing QTsang YZhou T(2024)Iris: a multi-constraint graphic layout generation systemIris:一个满足多条件约束的图形布局生成系统Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.230031225:7(968-987)Online publication date: 27-Jul-2024
https://doi.org/10.1631/FITEE.2300312
Chen LJing QTsang YWang QLiu RXia DZhou YSun L(2024)AutoSpark: Supporting Automobile Appearance Design Ideation with Kansei Engineering and Generative AIProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676337(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676337
Liu ZLiu FZhang M(2024)Intelligent Graphic Layout Generation: Current Status and Future Perspectives2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580676(2632-2637)Online publication date: 8-May-2024
https://doi.org/10.1109/CSCWD61410.2024.10580676
Show More Cited By

Index Terms

Layout Generation for Various Scenarios in Mobile Shopping Applications
1. Human-centered computing
  1. Interaction design
    1. Interaction design process and methods
      1. User interface design

Recommendations

GTLayout: Learning General Trees for Structured Grid Layout Generation
Computational Visual Media
Abstract
Structured grid layouts are preferable in many scenarios of 2D visual content creation since their structures facilitate further layout editing. Multiple geometry-based methods can effectively create structured grid layouts but require user-...
Two-stage Content-Aware Layout Generation for Poster Designs
MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Automatic layout generation models can generate numerous design layouts in a few seconds, which significantly reduces the amount of repetitive work for designers. However, most of these models consider the layout generation task as arranging layout ...
A study on the automatic generation of banner layouts
Highlights
- Filling design elements to the layout to generate a banner
- Aesthetic measures ...
Abstract
This paper addresses the automatic banner design problem as a layout generation task, where a banner is generated by filling design elements to a given layout. We initially introduce the traditional method of generating design layouts ...
Graphical abstract

Display Omitted

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

14911 pages

ISBN:9781450394215

DOI:10.1145/3544548

Editors:
Albrecht Schmidt
LMU Munich, Germany60028717
,
Kaisa Väänänen
Tampere University, Finland60011170
,
Tesh Goyal
Google Research, USA60006191
,
Per Ola Kristensson
University of Cambridge, UK60031101
,
Anicia Peters
University of Namibia, Namibia60072704
,
Stefanie Mueller
Massachusetts Institute of Technology, USA60022195
,
Julie R. Williamson
University of Glasgow, UK60001490
,
Max L. Wilson
University of Nottingham, UK60015138

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Natural Science Foundation of China
Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
996
Total Downloads

Downloads (Last 12 months)517
Downloads (Last 6 weeks)32

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen LJing QTsang YZhou T(2024)Iris: a multi-constraint graphic layout generation systemIris:一个满足多条件约束的图形布局生成系统Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.230031225:7(968-987)Online publication date: 27-Jul-2024
https://doi.org/10.1631/FITEE.2300312
Chen LJing QTsang YWang QLiu RXia DZhou YSun L(2024)AutoSpark: Supporting Automobile Appearance Design Ideation with Kansei Engineering and Generative AIProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676337(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676337
Liu ZLiu FZhang M(2024)Intelligent Graphic Layout Generation: Current Status and Future Perspectives2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580676(2632-2637)Online publication date: 8-May-2024
https://doi.org/10.1109/CSCWD61410.2024.10580676
Chen LJing QZhou YLi ZShi LSun L(2024)Element-conditioned GAN for graphic layout generationNeurocomputing10.1016/j.neucom.2024.127730591:COnline publication date: 28-Jul-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127730

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten