Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

StyleWe: Towards Style Fusion in Generative Fashion Design with Efficient Federated AI

Published: 08 November 2024 Publication History

Abstract

Collaboration can amalgamate diverse ideas, styles, and visual elements, fostering creativity and innovation among different designers. In collaborative design, sketches play a pivotal role as a means of expressing design creativity. However, designers often tend to not openly share these meticulously crafted sketches. This phenomenon of data island in the design area hinders its digital transformation under the third wave of AI. In this paper, we introduce a Federated Generative Artificial Intelligence Clothing system, namely StyleWe, employing federated learning to aid in sketch design. StyleWe is committed to establishing an ecosystem wherein designers can exchange sketch styles among themselves. Through StyleWe, designers can generate sketches that incorporate various designers' styles from their peers, drawing inspiration from collaboration without the need for data disclosure or upload. Extensive performance evaluations and user studies indicate that our StyleWe system can produce multi-styled sketches of comparable quality to human-designed ones while significantly enhancing efficiency compared to hand-drawn sketches.

Supplemental Material

MP4 File
Supplemental video

References

[1]
Oguz A Acar. 2019. Motivations and solution appropriateness in crowdsourcing challenges for innovation. Research Policy 48, 8 (2019), 103716.
[2]
Allan Afuah and Christopher L Tucci. 2012. Crowdsourcing as a solution to distant search. Academy of Management review 37, 3 (2012), 355--375.
[3]
Hani Al-bloush and Badariah Solemon. 2018. Intellectual property challenges in the crowdsourced software engineering: An analysis of crowdsourcing platforms. In Recent Trends in Information and Communication Technology: Proceedings of the 2nd International Conference of Reliable Information and Communication Technology (IRICT 2017). Springer, 875--884.
[4]
Salvatore Andolina, Hendrik Schneider, Joel Chan, Khalil Klouche, Giulio Jacucci, and Steven Dow. 2017. Crowdboard: augmenting in-person idea generation with real-time crowds. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 106--118.
[5]
Ankan Kumar Bhunia, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Jorma Laaksonen, and Michael Felsberg. 2022. Doodleformer: Creative sketch drawing with transformers. In Computer Vision--ECCV 2022: 17th European Conference. 338--355.
[6]
Birute Birgelyte. 2019. Open Innovation Management on Crowd-based Platforms: An Analysis of Managerial Approaches to Knowledge Sharing, Crowd Control and Intellectual Property Protection. (2019).
[7]
Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18392--18402.
[8]
Hancheng Cao, Yujie Lu, Yuting Deng, Daniel Mcfarland, and Michael S Bernstein. 2023. Breaking Out of the Ivory Tower: A Large-scale Analysis of Patent Citations to HCI Research. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1--24.
[9]
Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S Yu, and Lichao Sun. 2023. A comprehensive survey of AI-generated content (aigc): A history of generative ai from gan to chatgpt. arXiv preprint arXiv:2303.04226 (2023).
[10]
Shenglan Cui, Fang Liu, Yinman Guo, and Wei Wang. 2022. Plagiarism or reference? Exploring the detection criteria and solutions of visual design plagiarism. (2022).
[11]
Jeremy De Beer, Ian P McCarthy, Adam Soliman, and Emily Treen. 2017. Click here to agree: Managing intellectual property when crowdsourcing solutions. Business Horizons 60, 2 (2017), 207--217.
[12]
Hanhui Deng, Jianan Jiang, Zhiwang Yu, Jinhui Ouyang, and Di Wu. 2024. CrossGAI: A Cross-Device Generative AI Framework for Collaborative Fashion Design. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 1 (2024), 1--27.
[13]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, DirkWeissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2010. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv 2020. arXiv preprint arXiv:2010.11929 (2010).
[14]
Chenyou Fan and Ping Liu. 2020. Federated generative adversarial learning. In Pattern Recognition and Computer Vision: Third Chinese Conference, PRCV 2020, Nanjing, China, October 16--18, 2020, Proceedings, Part III 3. Springer, 3--15.
[15]
Casey Fiesler and Amy Bruckman. 2014. Copyright terms in online creative communities. In CHI'14 Extended Abstracts on Human Factors in Computing Systems. 2551--2556.
[16]
J Nils Foege, Ghita Dragsdahl Lauritzen, Frank Tietze, and Torsten Oliver Salge. 2019. Reconceptualizing the paradox of openness: How solvers navigate sharing-protecting tensions in crowdsourcing. Research Policy 48, 6 (2019), 1323--1339.
[17]
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2414--2423.
[18]
Antonio Ghezzi, Donata Gabelloni, Antonella Martini, and Angelo Natalicchio. 2018. Crowdsourcing: a review and suggestions for future research. International Journal of management reviews 20, 2 (2018), 343--363.
[19]
Xintong Han, Xiaojun Hu, Weilin Huang, and Matthew R Scott. 2019. Clothflow: A flow-based model for clothed person generation. In Proceedings of the IEEE/CVF international conference on computer vision. 10471--10480.
[20]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017).
[21]
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).
[22]
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017).
[23]
Youngseung Jeon, Seungwan Jin, Patrick C Shih, and Kyungsik Han. 2021. FashionQ: an AI-driven creativity support tool for facilitating ideation in fashion design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--18.
[24]
Bing Jiang, SenyinWang, Xinhui Zhao, and Jiuqiang Fu. 2021. Research on Innovative Elements of Product Appearance Design in International Intellectual Property Rights - Taking Furniture Design as an Example. In 2021 14th International Symposium on Computational Intelligence and Design (ISCID). 242--245.
[25]
Jianan Jiang, Di Wu, Hanhui Deng, Yidan Long, Wenyi Tang, Xiang Li, Can Liu, Zhanpeng Jin, Wenlei Zhang, and Tangquan Qi. 2024. HAIGEN: Towards Human-AI Collaboration for Facilitating Creativity and Style Generation in Fashion Design. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 3 (2024), 1--27.
[26]
Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design studies 26, 6 (2005), 613--624.
[27]
Rea Karachiwalla and Felix Pinkow. 2021. Understanding crowdsourcing projects: A review on the key design elements of a crowdsourcing initiative. Creativity and innovation management 30, 3 (2021), 563--584.
[28]
Yu Hwan Kim, Se Hyun Nam, Seung Baek Hong, and Kang Ryoung Park. 2022. GRA-GAN: Generative adversarial network for image style transfer of Gender, Race, and age. Expert Systems with Applications 198 (2022), 116792.
[29]
Jakub Konecny, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016).
[30]
Farinaz Koushanfar. 2022. Intellectual property (ip) protection for deep learning and federated learning models. In Proceedings of the 2022 ACM Workshop on Information Hiding and Multimedia Security. 5--5.
[31]
Ashwani Kumar. 2019. Design of secure image fusion technique using cloud for privacy-preserving and copyright protection. International Journal of Cloud Applications and Computing (IJCAC) 9, 3 (2019), 22--36.
[32]
Mohammed Lansari, Reda Bellafqira, Katarzyna Kapusta, Vincent Thouvenot, Olivier Bettan, and Gouenou Coatrieux. 2023. When Federated Learning meets Watermarking: A Comprehensive Overview of Techniques for Intellectual Property Protection. arXiv preprint arXiv:2308.03573 (2023).
[33]
Muyang Li, Ji Lin, Yaoyao Ding, Zhijian Liu, Jun-Yan Zhu, and Song Han. 2020. Gan compression: Efficient architectures for interactive conditional gans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5284--5294.
[34]
Shaojie Li, Jie Wu, Xuefeng Xiao, Fei Chao, Xudong Mao, and Rongrong Ji. 2021. Revisiting discriminator in GAN compression: A generator-discriminator cooperative compression scheme. Advances in Neural Information Processing Systems 34 (2021), 28560--28572.
[35]
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems 2 (2020), 429--450.
[36]
Bingchen Liu, Yizhe Zhu, Kunpeng Song, and Ahmed Elgammal. 2021. Self-supervised sketch-to-image synthesis. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 2073--2081.
[37]
Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Federico Perazzi, and Sun-Yuan Kung. 2021. Content-aware gan compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12156--12166.
[38]
Brady D Lund and Ting Wang. 2023. Chatting about ChatGPT: how may AI and GPT impact academia and libraries? Library Hi Tech News 40, 3 (2023), 26--29.
[39]
Anja M Maier, Claudia M Eckert, and P John Clarkson. 2021. Factors influencing communication in collaborative design. Journal of Engineering Design 32, 12 (2021), 671--702.
[40]
Marian Garcia Martinez. 2017. Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust. Technological Forecasting and Social Change 117 (2017), 296--304.
[41]
Erica Mazzola, Nuran Acur, Mariangela Piazza, and Giovanni Perrone. 2018. To own or not to own A study on the determinants and consequences of alternative intellectual property rights arrangements in crowdsourcing for innovation contests. Journal of Product Innovation Management 35, 6 (2018), 908--929.
[42]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273--1282.
[43]
Zhaoxiong Meng, Tetsuya Morizumi, Sumiko Miyata, and Hirotsugu Kinoshita. 2018. Design Scheme of Copyright Management System Based on Digital Watermarking and Blockchain. In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), Vol. 02. 359--364.
[44]
Regina Moirano, Marisa Analía Sánchez, and Libor tepánek. 2020. Creative interdisciplinary collaboration: A systematic literature review. Thinking Skills and Creativity 35 (2020), 100626.
[45]
Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2016. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016).
[46]
Mona Mrad, Maya F Farah, and Stephanie Haddad. 2019. From Karl Lagerfeld to Erdem: a series of collaborations between designer luxury brands and fast-fashion brands. Journal of Brand Management 26 (2019), 567--582.
[47]
Tobias O. Nyumba, Kerrie Wilson, Christina J Derrick, and Nibedita Mukherjee. 2018. The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods in Ecology and evolution 9, 1 (2018), 20--32.
[48]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).
[49]
Mic Porter. 2010. A consideration of academic misconduct in the creative disciplines: From inspiration to imitation and acceptable incorporation. Emerge 2 (2010), 1--16.
[50]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748--8763.
[51]
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-Shot Text-to-Image Generation. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8821--8831.
[52]
Mohammad Rasouli, Tao Sun, and Ram Rajagopal. 2020. Fedgan: Federated generative adversarial networks for distributed data. arXiv preprint arXiv:2006.07228 (2020).
[53]
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecny, Sanjiv Kumar, and H Brendan McMahan. 2020. Adaptive federated optimization. arXiv preprint arXiv:2003.00295 (2020).
[54]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684--10695.
[55]
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22500--22510.
[56]
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems 35 (2022), 36479--36494.
[57]
Thomas Schäper, J Nils Foege, Stephan Nüesch, and Sebastian Schäfer. 2021. Determinants of idea sharing in crowdsourcing: evidence from the automotive industry. R&D Management 51, 1 (2021), 101--113.
[58]
Fengrui Shi, Zhijin Qin, Di Wu, and Julie McCann. 2018. Mpcstoken: Smart contract enabled fault-tolerant incentivisation for mobile p2p crowd services. In 2018 IEEE 38Th international conference on distributed computing systems (ICDCS). IEEE, 961--971.
[59]
Yujun Shi, Jian Liang, Wenqing Zhang, Vincent YF Tan, and Song Bai. 2022. Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning. arXiv preprint arXiv:2210.00226 (2022).
[60]
Ben Shneiderman. 2008. Creativity support tools: A grand challenge for HCI researchers. In Engineering the user interface: From research to practice. Springer, 1--9.
[61]
Laurent Sifre and Stéphane Mallat. 2014. Rigid-motion scattering for texture classification. arXiv preprint arXiv:1403.1687 (2014).
[62]
Santiago Silva, Boris A Gutman, Eduardo Romero, Paul M Thompson, Andre Altmann, and Marco Lorenzi. 2019. Federated learning in distributed medical databases: Meta-analysis of large-scale subcortical brain data. In 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, 270--274.
[63]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[64]
Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. 2022. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 (2022).
[65]
V Srinivasan and Amaresh Chakrabarti. 2010. Investigating novelty--outcome relationships in engineering design. AI EDAM 24, 2 (2010), 161--178.
[66]
Ming Tao, Hao Tang, Fei Wu, Xiao-Yuan Jing, Bing-Kun Bao, and Changsheng Xu. 2022. Df-gan: A simple and effective baseline for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16515--16525.
[67]
K Varaprasada Rao and Sandeep Kumar Panda. 2022. A design model of copyright protection system based on distributed ledger technology. In Computer Communication, Networking and IoT: Proceedings of 5th ICICC 2021, Volume 2. Springer, 127--141.
[68]
Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Roman Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. 2022. Clipasso: Semantically-aware object sketching. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1--11.
[69]
Di Wu, He Xu, Zhongkai Jiang, Weiren Yu, Xuetao Wei, and Jiwu Lu. 2021. EdgeLSTM: Towards deep and sequential edge computing for IoT applications. IEEE/ACM Transactions on Networking 29, 4 (2021), 1895--1908.
[70]
Di Wu, Zhiwang Yu, Nan Ma, Jianan Jiang, Yuetian Wang, Guixiang Zhou, Hanhui Deng, and Yi Li. 2023. StyleMe: Towards Intelligent Fashion Generation with Designer Style. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1--16.
[71]
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. 2022. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs. In International Conference on Learning Representations.
[72]
Han Yan, Haijun Zhang, Linlin Liu, Dongliang Zhou, Xiaofei Xu, Zhao Zhang, and Shuicheng Yan. 2022. Toward intelligent design: An ai-based fashion designer using generative adversarial networks aided by sketch and rendering generators. IEEE Transactions on Multimedia (2022).
[73]
Qiang Yang, Anbu Huang, Lixin Fan, Chee Seng Chan, Jian Han Lim, Kam Woh Ng, Ding Sheng Ong, and Bowen Li. 2023. Federated Learning with Privacy-preserving and Model IP-right-protection. Machine Intelligence Research 20, 1 (2023), 19--37.
[74]
Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2022. Retrieval-Augmented Multimodal Language Modeling. arXiv preprint arXiv:2211.12561 (2022).
[75]
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. 2022. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789 (2022).
[76]
Theodore Zamenopoulos and Katerina Alexiou. 2018. Co-design as collaborative research. Bristol University/AHRC Connected Communities Programme.
[77]
Chaoning Zhang, Chenshuang Zhang, Sheng Zheng, Yu Qiao, Chenghao Li, Mengchun Zhang, Sumit Kumar Dam, Chu Myaet Thwal, Ye Lin Tun, Le Luang Huy, et al. 2023. A complete survey on generative ai (aigc): Is chatgpt from gpt-4 to gpt-5 all you need arXiv preprint arXiv:2303.11717 (2023).
[78]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586--595.
[79]
Min Zhao, Rongzhen Wang, Fan Bao, Chongxuan Li, and Jun Zhu. 2023. ControlVideo: Adding Conditional Control for One Shot Text-to-Video Editing. arXiv preprint arXiv:2305.17098 (2023).
[80]
Tianyue Zheng, Ang Li, Zhe Chen, Hongbo Wang, and Jun Luo. 2023. AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving. arXiv preprint arXiv:2302.08646 (2023).

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 8, Issue CSCW2
CSCW
November 2024
5177 pages
EISSN:2573-0142
DOI:10.1145/3703902
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2024
Published in PACMHCI Volume 8, Issue CSCW2

Permissions

Request permissions for this article.

Check for updates

Badges

  • Best Paper

Author Tags

  1. fashion design
  2. federated learning
  3. generative AI tool
  4. generative adversarial network
  5. sketch generation

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 229
    Total Downloads
  • Downloads (Last 12 months)229
  • Downloads (Last 6 weeks)102
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media