MCCP: multi-modal fashion compatibility and conditional preference model for personalized clothing recommendation
Authors: 
Yunzhu Wang, Li Liu, Xiaodong Fu, Lijun LiuAuthors Info & Claims
Yunzhu Wang, Li Liu, Xiaodong Fu, Lijun LiuAuthors Info & ClaimsPages 9621 - 9645
Abstract
Personalized clothing recommendation remains challenging due to the richness of fashion item representations, the non-uniqueness of fashion compatibility relationship and the complicated conditions of user preference. To address these problems, a novel model combining Multi-modal Fashion Compatibility and Conditional Preference (MCCP) is proposed. Firstly, we extract and fuse the multi-modal features (visual and textual) to comprehensively represent fashion items which can learn item-to-item compatibility and items-to-item compatibility. Secondly, we define conditional preference by dividing user-item interaction data into preference conditions and constructing conditional weight branch to learn preference degrees. Finally, we jointly train all of them based on Bayesian Personalized Ranking (BPR) to offer personalized and fashionable recommendations for user. We create a dataset WEAR-U including user label information and fashion data. Extensive experiment results on WEAR-U verify the effectiveness of the proposed model MCCP.
References
[1]
Bettaney EM, Hardwick SR, Zisimopoulos O, Chamberlain BP (2020) Fashion outfit generation for e-commerce. In: Joint European conference on machine learning and knowledge Discovery in Databases, Springer, pp 339–354
[2]
Chaidaroon S, Fang Y, Xie M, Magnani A (2019) Neural compatibility ranking for text-based fashion matching. In: Proceedings of the 42nd International ACM SIGIR conference on research and development in information retrieval, pp 1229–1232
[3]
Chen K, Franko K, Sang R (2021) Structured model pruning of convolutional networks on tensor processing units. arXiv:2107.04191
[4]
Chen L, He Y (2018) Dress fashionably: Learn fashion collocation with deep mixed-category metric learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 32
[5]
Chen W, Huang P, Xu J, Guo X, Guo C, Sun F, Li C, Pfadler A, Zhao H, Zhao B (2019) Pog: personalized outfit generation for fashion recommendation at alibaba ifashion. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2662–2670
[6]
Cheng Z, Chang X, Zhu L, Kanjirathinkal RC, and Kankanhalli M Mmalfm: Explainable recommendation by leveraging reviews and images ACM Trans Inf Syst (TOIS) 2019 37 2 1-28
[7]
Cheng WH, Song S, Chen CY, Hidayati SC, and Liu J Fashion meets computer vision: a survey ACM Comput Surv (CSUR) 2021 54 4 1-41
[8]
Cui Z, Li Z, Wu S, Zhang XY, Wang L (2019) Dressing as a whole: Outfit compatibility learning based on node-wise graph neural networks. In: The World Wide Web conference, pp 307–317
[9]
de Barros Costa E, Rocha HJB, Silva ET, Lima NC, Cavalcanti J (2017) Understanding and personalising clothing recommendation for women. In: World conference on information systems and technologies, Springer, pp 841–850
[10]
Dong X, Song X, Feng F, Jing P, Xu XS, Nie L (2019) Personalized capsule wardrobe creation with garment and user modeling. In: Proceedings of the 27th ACM international conference on multimedia, pp 302–310
[11]
Dong X, Wu J, Song X, Dai H, Nie L (2020) Fashion compatibility modeling through a multi-modal try-on-guided scheme. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 771–780
[12]
Han X, Song X, Yin J, Wang Y, Nie L (2019) Prototype-guided attribute-wise interpretable scheme for clothing matching. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 785–794 (2019)
[13]
Han X, Wu Z, Jiang YG, Davis LS (2017) Learning fashion compatibility with bidirectional lstms. In: Proceedings of the 25th ACM international conference on multimedia, pp 1078–1086
[14]
Han X, Song X, Yao Y, Xu XS, and Nie L Neural compatibility modeling with probabilistic knowledge distillation IEEE Trans Image Process 2019 29 871-882
[15]
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
[16]
He R, McAuley J (2016) Vbpr: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI conference on artificial intelligence, vol 30
[17]
Hidayati SC, Hsu CC, Chang YT, Hua KL, Fu J, Cheng WH (2018) What dress fits me best? fashion recommendation on the clothing style for personal body shape. In: Proceedings of the 26th ACM international conference on multimedia, pp 438–446
[18]
Hsieh CY, Li YM (2019) Fashion recommendation with social intelligence on personality and trends. In: 2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI), IEEE, pp 85–90
[19]
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 675–678
[20]
Kang WC, Kim E, Leskovec J, Rosenberg C, McAuley J (2019) Complete the look: Scene-based complementary product recommendation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10532–10541
[21]
Laenen K, Moens MF (2020) Attention-based fusion for outfit recommendation. In: Fashion Recommender Systems, Springer, pp 69–86
[22]
Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 29
[23]
Li X, Wang X, He X, Chen L, Xiao J, Chua TS (2020) Hierarchical fashion graph network for personalized outfit recommendation. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 159–168 (2020)
[24]
Li Z, Cui Z, Wu S, Zhang X, Wang L (2019) Semi-supervised compatibility learning across categories for clothing matching. In: 2019 IEEE international conference on multimedia and expo (ICME), IEEE, pp 484–489
[25]
Lin YL, Tran S, Davis LS (2020) Fashion outfit complementary item retrieval. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3311–3319
[26]
Song Lin J, X, Gan T, Yao Y, Liu W, Nie L, (2021) Paintnet: a shape-constrained generative framework for generating clothing from fashion model. Multimed Tools Appl 80:17183–17203
[27]
Liu J, Song X, Chen Z, and Ma J Neural fashion experts: i know how to make the complementary clothing matching Neurocomputing 2019 359 249-263
[28]
Liu S, Feng J, Song Z, Zhang T, Lu H, Xu C, Yan S (2012) Hi, magic closet, tell me what to wear! In: Proceedings of the 20th ACM international conference on Multimedia, pp 619–628
[29]
Lu Z, Hu Y, Jiang Y, Chen Y, Zeng B (2019) Learning binary code for personalized fashion recommendation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10562–10570
[30]
McAuley J, Targett C, Shi Q Van Den Hengel A (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp 43–52
[31]
Meng L, Feng F, He X, Gao X, Chua TS (2020) Heterogeneous fusion of semantic and collaborative information for visually-aware food recommendation. In: Proceedings of the 28th ACM international conference on multimedia, pp 3460–3468
[32]
Rendle S, Freudenthaler C, and Gantner Z Schmidt-Thieme L 2012 Bayesian personalized ranking from implicit feedback Bpr arXiv:1205.2618
[33]
Sagar D, Garg J, Kansal P, Bhalla S, Shah RR, Yu Y (2020) Pai-bpr: personalized outfit recommendation scheme with attribute-wise interpretability. In: 2020 IEEE sixth international conference on multimedia big data (BigMM), IEEE, pp 221–230
[34]
Sanchez-Riera J, Lin JM, Hua KL, Cheng WH, Tsui AW (2017) I-stylist: finding the right dress through your social networks. In: International conference on multimedia modeling, Springer, pp 662–673
[35]
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Sci
[36]
Song X, Feng F, Liu J, Li Z, Nie L, Ma J (2017) Neurostylist: Neural compatibility modeling for clothing matching. In: Proceedings of the 25th ACM international conference on Multimedia, pp 753–761
[37]
Song X, Han X, Li Y, Chen J, Xu XS, Nie L (2019) Gp-bpr: Personalized compatibility modeling for clothing matching. In: Proceedings of the 27th ACM international conference on multimedia, pp 320–328
[38]
Sun GL, Cheng ZQ, Wu X, and Peng Q Personalized clothing recommendation combining user social circle and fashion style consistency Multimed Tools Appl 2018 77 14 17731-17754
[39]
Sun GL, He JY, Wu X, Zhao B, and Peng Q Learning fashion compatibility across categories with deep multimodal neural networks Neurocomputing 2020 395 237-246
[40]
Vasileva MI, Plummer BA, Dusad K, Rajpal S, Kumar R, Forsyth D (2018) Learning type-aware embeddings for fashion compatibility. In: Proceedings of the European conference on computer vision (ECCV), pp 390–405
[41]
Wang T, Xu X, Yang Y, Hanjalic A, Shen HT, Song J (2019) Matching images and text with multi-modal tensor fusion and re-ranking. In: Proceedings of the 27th ACM international conference on multimedia, pp 12–20
[42]
Yang X, Song X, Feng F, Wen H, Duan LY, and Nie L Attribute-wise explainable fashion compatibility modeling ACM Trans Multimed Comput Commun Appl (TOMM) 2021 17 1 1-21
[43]
Yu LF, Yeung SK, Terzopoulos D, and Chan TF Dressup!: outfit synthesis through automatic optimization ACM Trans Graph 2012 31 6 134-1
[44]
Zhang H, Huang W, Liu L, and Chow TW Learning to match clothing from textual feature-based compatible relationships IEEE Transactions on Industrial Informatics 2019 16 11 6750-6759
[45]
Zhou X, Guo G, Sun Z, and Liu Y Multi-facet user preference learning for fine-grained item recommendation Neurocomputing 2020 385 258-268
[46]
Zhu N, Cao J, Liu Y, Yang Y, Ying H, Xiong H (2020) Sequential modeling of hierarchical user intention and preference for next-item recommendation. In: Proceedings of the 13th international conference on web search and data mining, pp 807–815
Index Terms
- MCCP: multi-modal fashion compatibility and conditional preference model for personalized clothing recommendation
Index terms have been assigned to the content through auto-classification.
Recommendations
Collaborative recommendation model based on multi-modal multi-view attention network: Movie and literature cases
AbstractThe existing collaborative recommendation models that use multi-modal information emphasize the representation of users’ preferences but easily ignore the representation of users’ dislikes. Nevertheless, modelling users’ dislikes ...
Highlights- We propose an attention-based multi-modal multi-view recommendation model.
- Our ...
Personalized clothing recommendation combining user social circle and fashion style consistency
With the rapid expansion of social networks and fashion websites, clothing recommendation has attracted more attention of researchers, since various web data bring opportunities for recommender systems to solve the problems of cold start and sparsity. ...
Effects of Personalized and Aggregate Top-N Recommendation Lists on User Preference Ratings
Prior research has shown a robust effect of personalized product recommendations on user preference judgments for items. Specifically, the display of system-predicted preference ratings as item recommendations has been shown in multiple studies to bias ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 24 June 2023
Accepted: 22 April 2023
Revision received: 01 August 2022
Received: 30 July 2021
Author Tags
Qualifiers
- Research-article
Funding Sources
- the national natural science foundation of china
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Jan 2025