research-article

Modeling Multi-Relational Connectivity for Personalized Fashion Matching

Authors:

Zhiyong ChengAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 7047 - 7055

https://doi.org/10.1145/3581783.3612583

Published: 27 October 2023 Publication History

Abstract

Personalized fashion matching task aims to predict the compatible fashion items given available ones for specific users through the effective modeling of the third-order interaction patterns among the user and item pairs. To achieve this, previous methods separately model two key components, user-item and item-item relationships, which ignore the inherent correlations between them and lead to undesirable performance. With a new perspective, this paper proposes to formulate the personalized item matching as the multi-relational connectivity and apply a single-component translation operation to model the targeted third-order interactions. With user-item-item interactions naturally constructing a multi-relational graph, we further device two graph learning modules to enhance the translation-based matching approach from two perspectives,C ontext and Path. The proposed method, named CP-TransMatch, has been tested with extensive experiments on three benchmark fashion datasets and proven effective. It sets the new SOTA for the personalized fashion matching task.

References

[1]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In NeurIPS. 2787--2795.

[2]

Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. 2019. POG: Personalized outfit generation for fashion recommendation at alibaba ifashion. In KDD. ACM, 2662--2670.

[3]

Guillem Cucurull, Perouz Taslakian, and David Vazquez. 2019. Context-aware visual compatibility prediction. In CVPR. IEEE, 12617--12626.

[4]

Yujuan Ding, Zhihui Lai, PY Mok, and Tat-Seng Chua. 2023. Computational Technologies for Fashion Recommendation: A Survey. arXiv preprint arXiv:2306.03395 (2023).

[5]

Yujuan Ding, Yunshan Ma, Wai Keung Wong, and Tat-Seng Chua. 2021. Leveraging Two Types of Global Graph for Sequential Fashion Recommendation. In ICMR. ACM, 73--81.

[6]

Yujuan Ding, PY Mok, Yunshan Ma, and Yi Bin. 2023. Personalized fashion outfit generation with user coordination preference learning. Information Processing & Management 60, 5 (2023), 103434.

Digital Library

[7]

Yujuan Ding, PY Mok, Xun Yang, and Yanghong Zhou. 2022. Modeling Field-Level Factor Interactions for Fashion Recommendation. In ICME. IEEE, 1--6.

[8]

Weili Guan, Fangkai Jiao, Xuemeng Song, Haokun Wen, Chung-Hsing Yeh, and Xiaojun Chang. 2022. Personalized Fashion Compatibility Modeling via Metapathguided Heterogeneous Graph Learning. In SIGIR. ACM, 482--491.

[9]

Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2022. A Survey on Knowledge Graph-Based Recommender Systems. IEEE Trans. Knowl. Data Eng. 34, 8 (2022), 3549--3568.

[10]

Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S. Davis. 2017. Learning Fashion Compatibility with Bidirectional LSTMs. In ACM Multimedia. ACM, 1078--1086.

[11]

Ruining He, Wang-Cheng Kang, and Julian J. McAuley. 2017. Translation-based Recommendation. In RecSys. ACM, 161--169.

[12]

Ruining He and Julian J. McAuley. 2016. VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback. In AAAI. AAAI Press, 144--150.

[13]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR. ACM, 639--648.

[14]

Yang Hu, Xi Yi, and Larry S. Davis. 2015. Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach. In ACM Multimedia. ACM, 129--138.

[15]

Wang-Cheng Kang, Mengting Wan, and Julian J. McAuley. 2018. Recommendation Through Mixtures of Heterogeneous Item Relationships. In CIKM. ACM, 1143--1152.

[16]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 6 (2017), 84--90.

Digital Library

[17]

Yang Li, Yadan Luo, and Zi Huang. 2020. Graph-Based Relation-Aware Representation Learning for Clothing Matching. In ADC (Lecture Notes in Computer Science, Vol. 12008). Springer, 189--197.

Digital Library

[18]

Zhi Li, Bo Wu, Qi Liu, Likang Wu, Hongke Zhao, and Tao Mei. 2021. Learning the Compositional Visual Coherence for Complementary Recommendations. In IJCAI. AAAI Press, Article 489, 8 pages.

[19]

Shuiying Liao, Yujuan Ding, and PY Mok. 2023. Recommendation of Mix-and- Match Clothing by Modeling Indirect Personal Compatibility. In ICMR. ACM, 560--564.

[20]

Yen-Liang Lin, Son Dinh Tran, and Larry S. Davis. 2020. Fashion Outfit Complementary Item Retrieval. In CVPR. Computer Vision Foundation / IEEE, 3308--3316.

[21]

Zhi Lu, Yang Hu, Yan Chen, and Bing Zeng. 2021. Personalized Outfit Recommendation With Learnable Anchors. In CVPR. Computer Vision Foundation / IEEE, 12722--12731.

[22]

Zhi Lu, Yang Hu, Yunchao Jiang, Yan Chen, and Bing Zeng. 2019. Learning Binary Code for Personalized Fashion Recommendation. In CVPR. Computer Vision Foundation / IEEE, 10562--10570.

[23]

Yunshan Ma, Xun Yang, Lizi Liao, Yixin Cao, and Tat-Seng Chua. 2019. Who, where, and what to wear? extracting fashion knowledge from social media. In ACM Multimedia. 257--265.

[24]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. AUAI Press, 452--461.

Digital Library

[25]

Dikshant Sagar, Jatin Garg, Prarthana Kansal, Sejal Bhalla, Rajiv Ratn Shah, and Yi Yu. 2020. PAI-BPR: Personalized Outfit Recommendation Scheme with Attribute-wise Interpretability. In BigMM. IEEE, 221--230.

[26]

Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019. GP-BPR: Personalized Compatibility Modeling for Clothing Matching. In ACM Multimedia. ACM, 320--328.

[27]

Guang-Lu Sun, Jun-Yan He, Xiao Wu, Bo Zhao, and Qiang Peng. 2020. Learning fashion compatibility across categories with deep multimodal neural networks. Neurocomputing 395 (2020), 237--246.

[28]

Mariya I Vasileva, Bryan A Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. 2018. Learning type-aware embeddings for fashion compatibility. In ECCV. Springer, 390--405.

[29]

Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015. Learning visual clothing style with heterogeneous dyadic cooccurrences. In ICCV. IEEE, 4642--4650.

[30]

Hongwei Wang, Hongyu Ren, and Jure Leskovec. 2021. Relational Message Passing for Knowledge Graph Completion. In KDD. ACM, 1697--1707.

[31]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR. ACM, 165--174.

[32]

An Yan, Chaosheng Dong, Yan Gao, Jinmiao Fu, Tong Zhao, Yi Sun, and Julian J. McAuley. 2022. Personalized Complementary Product Recommendation. In WWW (Companion Volume). ACM, 146--151.

[33]

Xun Yang, Xiaoyu Du, and Meng Wang. 2020. Learning to match on graph for fashion compatibility modeling. In AAAI, Vol. 34. AAAI Press, 287--294.

[34]

Xun Yang, Xiangnan He, Xiang Wang, Yunshan Ma, Fuli Feng, Meng Wang, and Tat-Seng Chua. 2019. Interpretable fashion matching with rich attributes. In SIGIR. ACM, 775--784.

[35]

Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. 2019. TransNFCM: Translation-Based Neural Fashion Compatibility Modeling. In AAAI. AAAI Press, 403--410.

[36]

Xuewen Yang, Dongliang Xie, Xin Wang, Jiangbo Yuan, Wanying Ding, and Pengyun Yan. 2020. Learning Tuple Compatibility for Conditional Outfit Recommendation. In ACM Multimedia. ACM, 2636--2644.

Cited By

Yu MMa YWu LCheng KLi XMeng LChua TGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Smart Fitting Room: A One-stop Framework for Matching-aware Virtual Try-OnProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658064(184-192)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658064
Ding YFan WHuang XLi QChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Large Language Models for Graph LearningCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3641300(1643-1646)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3641300
Su ZChen YZhang FWang RZhou FLin G(2024)DMAP: Decoupling-Driven Multi-Level Attribute Parsing for Interpretable Outfit CollocationIEEE Transactions on Multimedia10.1109/TMM.2024.340254126(9988-10000)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3402541

Index Terms

Modeling Multi-Relational Connectivity for Personalized Fashion Matching
1. Applied computing
  1. Electronic commerce
    1. Online shopping

Recommendations

Computational Technologies for Fashion Recommendation: A Survey
Fashion recommendation is a key research field in computational fashion research and has attracted considerable interest in the computer vision, multimedia, and information retrieval communities in recent years. Due to the great demand for applications, ...
Personalized Recommendation Algorithm Using User Demography Information
WKDD '09: Proceedings of the 2009 Second International Workshop on Knowledge Discovery and Data Mining

Personalized recommendation systems are web-based systems that aim at predicting a user’s interest on available products and services by relying on previously rated items and dealing with the problem of information and product overload. User demography ...
Neural fashion experts: I know how to make the complementary clothing matching
Abstract
Clothing has gradually become the beauty enhancing product, while the harmonious clothing matching is critical for a suitable outfit. The existing clothing matching techniques mainly rely on the visual features but overlook the textual ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Innovation and Technology Fund
Shandong Project towards the Integration of Education and Industry
National Natural Science Foundation of China
Research Grants Council of Hong Kong

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
187
Total Downloads

Downloads (Last 12 months)187
Downloads (Last 6 weeks)16

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu MMa YWu LCheng KLi XMeng LChua TGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Smart Fitting Room: A One-stop Framework for Matching-aware Virtual Try-OnProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658064(184-192)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658064
Ding YFan WHuang XLi QChua TNgo CKumar RLauw HKa-Wei Lee R(2024)Large Language Models for Graph LearningCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3641300(1643-1646)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3641300
Su ZChen YZhang FWang RZhou FLin G(2024)DMAP: Decoupling-Driven Multi-Level Attribute Parsing for Interpretable Outfit CollocationIEEE Transactions on Multimedia10.1109/TMM.2024.340254126(9988-10000)Online publication date: 2024
https://doi.org/10.1109/TMM.2024.3402541

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents