Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3477495.3532038acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning

Published: 07 July 2022 Publication History

Abstract

Fashion Compatibility Modeling (FCM) is a new yet challenging task, which aims to automatically access the matching degree among a set of complementary items. Most of existing methods evaluate the fashion compatibility from the common perspective, but overlook the user's personal preference. Inspired by this, a few pioneers study the Personalized Fashion Compatibility Modeling (PFCM). Despite their significance, these PFCM methods mainly concentrate on the user and item entities, as well as their interactions, but ignore the attribute entities, which contain rich semantics. To address this problem, we propose to fully explore the related entities and their relations involved in PFCM to boost the PFCM performance. This is, however, non-trivial due to the heterogeneous contents of different entities, embeddings for new users, and various high-order relations. Towards these ends, we present a novel metapath-guided personalized fashion compatibility modeling, dubbed as MG-PFCM. In particular, we creatively build a heterogeneous graph to unify the three types of entities (i.e., users, items, and attributes) and their relations (i.e., user-item interactions, item-item matching relations, and item-attribute association relations). Thereafter, we design a multi-modal content-oriented user embedding module to learn user representations by inheriting the contents of their interacted items. Meanwhile, we define the user-oriented and item-oriented metapaths, and perform the metapath-guided heterogeneous graph learning to enhance the user and item embeddings. In addition, we introduce the contrastive regularization to improve the model performance. We conduct extensive experiments on the real-world benchmark dataset, which verifies the superiority of our proposed scheme over several cutting-edge baselines. As a byproduct, we have released our source codes to benefit other researchers.

Supplementary Material

MP4 File (SIGIR22-fp081.mp4)
Presentation video regarding "Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning". In this video, we propose to fully explore the related entities (users, items, and attributes) and their relations involved in Personalized Fashion Compatibility Modeling (PFCM) to boost the PFCM performance. In particular, We define a heterogeneous graph to creatively unify three types of entities and relations in the PFCM context. And we then present a metapath-guided personalized compatibility modeling scheme to perform heterogeneous graph learning. It adopts the pre-defined metapaths to explore the high order relations among various entities, and hence strengthen the user and item embeddings. Last but not least, we derive users? embeddings via fusing their interacted items and introduce a contrastive regularization to improve embedding learning.

References

[1]
Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, and Joonseok Lee. 2019. NGCN: Multi-scale Graph Convolution for Semi-supervised Node Classification. In Proceedings of the Conference on Uncertainty in Artificial Intelligence. AUAI Press, 841--851.
[2]
Suthee Chaidaroon, Yi Fang, Min Xie, and Alessandro Magnani. 2019. Neural Compatibility Ranking for Text-based Fashion Matching. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 1229--1232.
[3]
Guillem Cucurull, Perouz Taslakian, and David Vázquez. 2019. Context-Aware Visual Compatibility Prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 12617--12626.
[4]
Zeyu Cui, Zekun Li, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Dressing as a Whole: Outfit Compatibility Learning Based on Node-wise Graph Neural Networks. In Proceedings of the World Wide Web Conference. ACM, 307--317.
[5]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics. ACL, 4171--4186.
[6]
Xue Dong, Xuemeng Song, Fuli Feng, Peiguang Jing, Xin-Shun Xu, and Liqiang Nie. 2019. Personalized Capsule Wardrobe Creation with Garment and User Modeling. In Proceedings of the International ACM Conference on Multimedia. ACM, 302--310.
[7]
Xue Dong, Jianlong Wu, Xuemeng Song, Hongjun Dai, and Liqiang Nie. 2020. Fashion Compatibility Modeling through a Multi-modal Try-on-guided Scheme. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 771--780.
[8]
Yuxiao Dong, Nitesh V. Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable Representation Learning for Heterogeneous Networks. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 135--144.
[9]
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Yihong Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. In Proceedings of the World Wide Web Conference. ACM, 417--426.
[10]
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. In Proceedings of the World Wide Web Conference. ACM, 2331--2341.
[11]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 855--864.
[12]
Weili Guan, Haokun Wen, Xuemeng Song, Chung-Hsing Yeh, Xiaojun Chang, and Liqiang Nie. 2021. Multimodal Compatibility Modeling via Exploring the Consistent and Complementary Correlations. In Proceedings of the International ACM Conference on Multimedia. ACM, 2299--2307.
[13]
Xianjing Han, Xuemeng Song, Jianhua Yin, Yinglong Wang, and Liqiang Nie. 2019. Prototype-guided Attribute-wise Interpretable Scheme for Clothing Matching. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 785--794.
[14]
Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S. Davis. 2017. Learning Fashion Compatibility with Bidirectional LSTMs. In Proceedings of the International ACM Conference on Multimedia. ACM, 1078--1086.
[15]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 770--778.
[16]
Lu Jiang, Shoou-I Yu, Deyu Meng, Yi Yang, Teruko Mitamura, and Alexander G. Hauptmann. 2015. Fast and Accurate Content-based Semantic Search in 100M Internet Videos. In Proceedings of the International ACM Conference on Multimedia. ACM, 49--58.
[17]
Di Jin, Cuiying Huo, Chundong Liang, and Liang Yang. 2021. Heterogeneous Graph Neural Network via Attribute Completion. In Proceedings of the World Wide Web Conference. ACM, 391--400.
[18]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 1--15.
[19]
Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, and Tat-Seng Chua. 2020. Hierarchical Fashion Graph Network for Personalized Outfit Recommendation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 159--168.
[20]
Zhi Lu, Yang Hu, Yunchao Jiang, Yan Chen, and Bing Zeng. 2019. Learning Binary Code for Personalized Fashion Recommendation. In IEEE Conference on Computer Vision and Pattern Recognition,. IEEE, 10562--10570.
[21]
Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 43--52.
[22]
Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 1--12.
[23]
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 1--17.
[24]
Dikshant Sagar, Jatin Garg, Prarthana Kansal, Sejal Bhalla, Rajiv Ratn Shah, and Yi Yu. 2020. PAI-BPR: Personalized Outfit Recommendation Scheme with Attribute-wise Interpretability. In IEEE International Conference on Multimedia Big Data. IEEE, 221--230.
[25]
Chuan Shi, Binbin Hu, Wayne Xin Zhao, and Philip S. Yu. 2019. Heterogeneous Information Network Embedding for Recommendation. IEEE Transactions on Knowledge and Data Engineering 31, 2 (2019), 357--370.
[26]
Xue Song, Jingjing Chen, Zuxuan Wu, and Yu-Gang Jiang. 2021. Spatial-temporal graphs for cross-modal text2video retrieval. IEEE Transactions on Multimedia (2021).
[27]
Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, and Liqiang Nie. 2018. Neural Compatibility Modeling with Attentive Knowledge Distillation. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 5--14.
[28]
Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017. NeuroStylist: Neural Compatibility Modeling for Clothing Matching. In Proceedings of the International ACM Conference on Multimedia. ACM, 753--761.
[29]
Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019. GP-BPR: Personalized Compatibility Modeling for Clothing Matching. In Proceedings of the International ACM Conference on Multimedia. ACM, 320--328.
[30]
Yizhou Sun and Jiawei Han. 2012. Mining heterogeneous information networks: principles and methodologies. Synthesis Lectures on Data Mining and Knowledge Discovery 3, 2 (2012), 1--159.
[31]
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale Information Network Embedding. In Proceedings of the World Wide Web Conference. ACM, 1067--1077.
[32]
Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David A. Forsyth. 2018. Learning Type-Aware Embeddings for Fashion Compatibility. In Proceedings of the European Conference on Computer Vision. Springer, 405--421.
[33]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Proceedings of the Conference on Neural Information Processing Systems. MIT Press, 5998--6008.
[34]
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the International Conference on Learning Representations. OpenReview.net, 1--12.
[35]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous Graph Attention Network. In Proceedings of the World Wide Web Conference. ACM, 2022--2032.
[36]
Yuying Xing, Zhao Li, Pengrui Hui, Jiaming Huang, Xia Chen, Long Zhang, and Guoxian Yu. 2020. Link Inference via Heterogeneous Multi-view Graph Neural Networks. In Proceedings of the International Conference on Database Systems for Advanced Applications. Springer, 698--706.
[37]
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V. Chawla. 2019. Heterogeneous Graph Neural Network. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 793--803.
[38]
Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, and Tat-Seng Chua. 2013. Attribute-augmented Semantic Hierarchy: Towards Bridging Semantic Gap and Intention Gap in Image Retrieval. In Proceedings of the International ACM Conference on Multimedia. ACM, 33--42.
[39]
Jintao Zhang and Quan Xu. 2021. Attention-aware Heterogeneous Graph Neural Network. Big Data Mining and Analytics 4, 4 (2021), 233--241.
[40]
Huan Zhao, Quanming Yao, Jianda Li, Yangqiu Song, and Dik Lun Lee. 2017. Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks. In Proceedings of the International ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 635--644.

Cited By

View all
  • (2024)Swarm Self-supervised Hypergraph Embedding for RecommendationACM Transactions on Knowledge Discovery from Data10.1145/363805818:4(1-19)Online publication date: 13-Feb-2024
  • (2024)Improving Issue-PR Link Prediction via Knowledge-Aware Heterogeneous Graph LearningIEEE Transactions on Software Engineering10.1109/TSE.2024.340844850:7(1901-1920)Online publication date: 1-Jul-2024
  • (2024)Dynamic Relation Graph Learning for Time-Aware Service RecommendationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.332597721:2(1503-1517)Online publication date: Apr-2024
  • Show More Cited By

Index Terms

  1. Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2022
    3569 pages
    ISBN:9781450387323
    DOI:10.1145/3477495
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 July 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. heterogeneous graph neural networks
    2. metapath-guided graph learning
    3. personalized compatibility modeling

    Qualifiers

    • Research-article

    Funding Sources

    • Australian Research Council

    Conference

    SIGIR '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)201
    • Downloads (Last 6 weeks)15
    Reflects downloads up to 16 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Swarm Self-supervised Hypergraph Embedding for RecommendationACM Transactions on Knowledge Discovery from Data10.1145/363805818:4(1-19)Online publication date: 13-Feb-2024
    • (2024)Improving Issue-PR Link Prediction via Knowledge-Aware Heterogeneous Graph LearningIEEE Transactions on Software Engineering10.1109/TSE.2024.340844850:7(1901-1920)Online publication date: 1-Jul-2024
    • (2024)Dynamic Relation Graph Learning for Time-Aware Service RecommendationIEEE Transactions on Network and Service Management10.1109/TNSM.2023.332597721:2(1503-1517)Online publication date: Apr-2024
    • (2024)DMAP: Decoupling-Driven Multi-Level Attribute Parsing for Interpretable Outfit CollocationIEEE Transactions on Multimedia10.1109/TMM.2024.340254126(9988-10000)Online publication date: 2024
    • (2024)Textual Enhanced Adaptive Meta-Fusion for Few-Shot Visual RecognitionIEEE Transactions on Multimedia10.1109/TMM.2023.329573126(2408-2418)Online publication date: 1-Jan-2024
    • (2024)Multi-Modal Structure-Embedding Graph Transformer for Visual Commonsense ReasoningIEEE Transactions on Multimedia10.1109/TMM.2023.327969126(1295-1305)Online publication date: 1-Jan-2024
    • (2024)Learning to Agree on Vision Attention for Visual Commonsense ReasoningIEEE Transactions on Multimedia10.1109/TMM.2023.327587426(1065-1075)Online publication date: 1-Jan-2024
    • (2024)Heterogeneous-Grained Multi-Modal Graph Network for Outfit RecommendationIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.33581908:2(1788-1799)Online publication date: Apr-2024
    • (2024)Multimodal High-Order Relationship Inference Network for Fashion Compatibility Modeling in Internet of Multimedia ThingsIEEE Internet of Things Journal10.1109/JIOT.2023.328560111:1(353-365)Online publication date: 1-Jan-2024
    • (2024)Unifying heterogeneous and homogeneous relations for personalized compatibility modelingKnowledge-Based Systems10.1016/j.knosys.2024.111560290:COnline publication date: 2-Jul-2024
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media