Outfit Recommendation using Graph Neural Networks via Visual Similarity
Authors: 
Diana Zagidullina and Ilya MakarovAuthors Info & Claims
Diana Zagidullina and Ilya MakarovAuthors Info & ClaimsDecember 2021
Pages 208 - 222
Abstract
Computer vision plays an important role in the development of the fashion industry. There has been a lot of research done on various fashion recommendations, and determining the compatibility of clothing is a key factor in most of them. Solving this problem can help users buy items that go well with their current wardrobe, and help stores sell multiple clothing items at once. Previous research has mainly focused on learning compatibility between two clothing elements. There are several approaches that take into account the outfit as a whole but they require rich textual data. In this work, we only use images of clothing from the Polyvore dataset to extract visual features. By representing outfits in the form of a graph, we train node embeddings based on graph structure and node features. We then train multi-layer perceptron to classify the set of embeddings representing the outfit. We compare our method with the relevant works in two tasks: outfit compatibility prediction and fill-in-the-blank. Our approach showed the best result among approaches that use only images on the first task and showed state-of-the-art result on the second task.
References
[1]
Ananyeva M, Makarov I, and Pendiukhov M Bychkov I, Kalyagin VA, Pardalos PM, and Prokopyev O GSM: inductive learning on dynamic graph embeddings Network Algorithms, Data Mining, and Applications 2020 Cham Springer 85-99
[2]
Andreeva E, Ignatov DI, Grachev A, Savchenko AV, et al. van der Aalst WMP et al. Extraction of visual features for recommendation of products via deep learning Analysis of Images, Social Networks and Texts 2018 Cham Springer 201-210
[3]
Averchenkova A et al. Bychkov I, Kalyagin VA, Pardalos PM, Prokopyev O, et al. Collaborator recommender system Network Algorithms, Data Mining, and Applications 2020 Cham Springer 101-119
[4]
Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Nips, vol. 14, pp. 585–591 (2001)
[5]
Brand, M.: Continuous nonlinear dimensionality reduction by kernel eigenmaps. In: IJCAI, pp. 547–554. Citeseer (2003)
[6]
Chen, L., He, Y.: Dress fashionably: learn fashion collocation with deep mixed-category metric learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1 (2018)
[7]
Cucurull, G., Taslakian, P., Vazquez, D.: Context-aware visual compatibility prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12617–12626 (2019)
[8]
Cui, Z., Li, Z., Wu, S., Zhang, X.Y., Wang, L.: Dressing as a whole: outfit compatibility learning based on node-wise graph neural networks. In: The World Wide Web Conference, pp. 307–317 (2019)
[9]
Deerwester S, Dumais ST, Furnas GW, Landauer TK, and Harshman R Indexing by latent semantic analysis J. Am. Soc. Inf. Sci. 1990 41 6 391-407
[10]
Demochkin K, Savchenko AV, et al. van der Aalst WMP et al. Multi-label image set recognition in visually-aware recommender systems Analysis of Images, Social Networks and Texts 2019 Cham Springer 291-297
[11]
Gerasimova, O., Makarov, I.: Higher school of economics co-authorship network study. In: 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), pp. 1–4. IEEE (2019)
[12]
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 855–864 (2016)
[13]
Hamilton, W.L., Ying, R., Leskovec, J.: Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 1025–1035 (2017)
[14]
Han, X., Wu, Z., Jiang, Y.G., Davis, L.S.: Learning fashion compatibility with bidirectional lstms. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1078–1086 (2017)
[15]
He, R., McAuley, J.: VBPR: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1 (2016)
[16]
He, R., Packer, C., McAuley, J.: Learning compatibility across categories for heterogeneous item recommendation. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 937–942. IEEE (2016)
[17]
Hsiao, W.L., Grauman, K.: Creating capsule wardrobes from fashion images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7161–7170 (2018)
[18]
Hu, Y., Yi, X., Davis, L.S.: Collaborative fashion recommendation: a functional tensor factorization approach. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 129–138 (2015)
[19]
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
[20]
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
[21]
Kiselev D, Makarov I, et al. van der Aalst WMP et al. Prediction of new itinerary markets for airlines via network embedding Analysis of Images, Social Networks and Texts 2020 Cham Springer 315-325
[22]
Kruskal, J.B.: Multidimensional scaling. Sage (1978)
[23]
Li, X., Wang, X., He, X., Chen, L., Xiao, J., Chua, T.S.: Hierarchical fashion graph network for personalized outfit recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 159–168 (2020)
[24]
Li Y, Cao L, Zhu J, and Luo J Mining fashion outfit composition using an end-to-end deep learning approach on set data IEEE Trans. Multimedia 2017 19 8 1946-1955
[25]
Liang X, Lin L, Yang W, Luo P, Huang J, and Yan S Clothes co-parsing via joint image segmentation and labeling with application to clothing retrieval IEEE Trans. Multimedia 2016 18 6 1175-1186
[26]
Lin Y, Ren P, Chen Z, Ren Z, Ma J, and De Rijke M Explainable outfit recommendation with joint outfit matching and comment generation IEEE Trans. Knowl. Data Eng. 2019 32 8 1502-1516
[27]
Liu, S., et al.: Hi, magic closet, tell me what to wear! In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 619–628 (2012)
[28]
Ma, Y., Ding, Y., Yang, X., Liao, L., Wong, W.K., Chua, T.S.: Knowledge enhanced neural fashion trend forecasting. In: Proceedings of the 2020 International Conference on Multimedia Retrieval, pp. 82–90 (2020)
[29]
Makarov I, Bulanov O, Gerasimova O, Meshcheryakova N, Karpov I, Zhukov LE, et al. van der Aalst WMP et al. Scientific matchmaker: collaborator recommender system Analysis of Images, Social Networks and Texts 2018 Cham Springer 404-410
[30]
Makarov I, Bulanov O, and Zhukov LE Kalyagin VA, Nikolaev AI, Pardalos PM, and Prokopyev OA Co-author recommender system Models, Algorithms, and Technologies for Network Analysis 2017 Cham Springer 251-257
[31]
Makarov I and Gerasimova O Rojas I, Joya G, and Catala A Link prediction regression for weighted co-authorship networks Advances in Computational Intelligence 2019 Cham Springer 667-677
[32]
Makarov, I., Gerasimova, O.: Predicting collaborations in co-authorship network. In: 2019 14th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), pp. 1–6. IEEE (2019)
[33]
Makarov I, Gerasimova O, Sulimov P, Korovina K, Zhukov LE, et al. van der Aalst WMP et al. Joint node-edge network embedding for link prediction Analysis of Images, Social Networks and Texts 2018 Cham Springer 20-31
[34]
Makarov I, Gerasimova O, Sulimov P, Zhukov LE, et al. van der Aalst WMP et al. Co-authorship network embedding and recommending collaborators via network embedding Analysis of Images, Social Networks and Texts 2018 Cham Springer 32-38
[35]
Makarov, I., Gerasimova, O., Sulimov, P., Zhukov, L.E.: Recommending co-authorship via network embeddings and feature engineering: the case of national research university higher school of economics. In: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries, pp. 365–366. ACM (2018)
[36]
Makarov I, Gerasimova O, Sulimov P, and Zhukov LE Dual network embedding for representing research interests in the link prediction problem on co-authorship networks PeerJ Comput. Sci. 2019 5
[37]
Makarov I, Kiselev D, Nikitinsky N, and Subelj L Survey on graph embeddings and their applications to machine learning problems on graphs PeerJ Comput. Sci. 2021 7
[38]
Makarov I, Korovina K, and Kiselev D Jonnee: joint network nodes and edges embedding IEEE Access 2021 9 144646-144659
[39]
Makarov I, Makarov M, and Kiselev D Fusion of text and graph information for machine learning problems on networks PeerJ Comput. Sci. 2021 7
[40]
McAuley, J., Targett, C., Shi, Q., van Den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52 (2015)
[41]
Mikolov T, Sutskever I, Chen K, Corrado GS, and Dean J Distributed representations of words and phrases and their compositionality Adv. Neural. Inf. Process. Syst. 2013 26 3111-3119
[42]
Revanur, A., Kumar, V., Sharma, D.: Semi-supervised visual representation learning for fashion compatibility. In: Fifteenth ACM Conference on Recommender Systems, pp. 463–472 (2021)
[43]
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
[44]
Rustem, M.K., Makarov, I., Zhukov, L.E.: Predicting psychology attributes of a social network user. In: Proceedings of the Fourth Workshop on Experimental Economics and Machine Learning (EEML 2017), Dresden, Germany, 17–18 September 2017, pp. 1–7. CEUR WP (2017)
[45]
Shih, Y.S., Chang, K.Y., Lin, H.T., Sun, M.: Compatibility family learning for item recommendation and generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 32, no. 1 (2018)
[46]
Song, X., Feng, F., Liu, J., Li, Z., Nie, L., Ma, J.: Neurostylist: neural compatibility modeling for clothing matching. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 753–761 (2017)
[47]
Sun, G.L., Wu, X., Chen, H.H., Peng, Q.: Clothing style recognition using fashion attribute detection. In: Proceedings of the 8th International Conference on Mobile Multimedia Communications, pp. 145–148 (2015)
[48]
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
[49]
Tenenbaum, J.B., De Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
[50]
Vasileva, M.I., Plummer, B.A., Dusad, K., Rajpal, S., Kumar, R., Forsyth, D.: Learning type-aware embeddings for fashion compatibility. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 390–405 (2018)
[51]
Veit, A., Kovacs, B., Bell, S., McAuley, J., Bala, K., Belongie, S.: Learning visual clothing style with heterogeneous dyadic co-occurrences. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4642–4650 (2015)
[52]
Wang, X., Wu, B., Zhong, Y.: Outfit compatibility prediction and diagnosis with multi-layered comparison network. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 329–337 (2019)
[53]
Wold S, Esbensen K, and Geladi P Principal component analysis Chemom. Intell. Lab. Syst. 1987 2 1–3 37-52
[54]
Yamaguchi K, Kiapour MH, Ortiz LE, and Berg TL Retrieving similar styles to parse clothing IEEE Trans. Pattern Anal. Mach. Intell. 2014 37 5 1028-1040
[55]
Yu Y, Hui CL, and Choi TM An empirical study of intelligent expert systems on forecasting of fashion color trend Expert Syst. Appl. 2012 39 4 4383-4389
Index Terms
- Outfit Recommendation using Graph Neural Networks via Visual Similarity
Index terms have been assigned to the content through auto-classification.
Recommendations
Hierarchical Fashion Graph Network for Personalized Outfit Recommendation
SIGIR '20: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information RetrievalFashion outfit recommendation has attracted increasing attentions from online shopping services and fashion communities.Distinct from other scenarios (e.g., social networking or content sharing) which recommend a single item (e.g., a friend or picture) ...
OutfitNet: Fashion Outfit Recommendation with Attention-Based Multiple Instance Learning
WWW '20: Proceedings of The Web Conference 2020Recommending fashion outfits to users presents several challenges. First of all, an outfit consists of multiple fashion items, and each user emphasizes different parts of an outfit when considering whether they like it or not. Secondly, a user’s liking ...
Computational Technologies for Fashion Recommendation: A Survey
Fashion recommendation is a key research field in computational fashion research and has attracted considerable interest in the computer vision, multimedia, and information retrieval communities in recent years. Due to the great demand for applications, ...
Comments
Information & Contributors
Information
Published In
Dec 2021
357 pages
ISBN:978-3-031-16499-6
DOI:10.1007/978-3-031-16500-9
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 16 December 2021
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in