survey

Computational Technologies for Fashion Recommendation: A Survey

Authors:

Yujuan Ding,

Zhihui Lai,

P.Y. Mok, and

Tat-Seng ChuaAuthors Info & Claims

ACM Computing Surveys, Volume 56, Issue 5

Article No.: 121, Pages 1 - 45

https://doi.org/10.1145/3627100

Published: 25 November 2023 Publication History

Get Access

Abstract

Fashion recommendation is a key research field in computational fashion research and has attracted considerable interest in the computer vision, multimedia, and information retrieval communities in recent years. Due to the great demand for applications, various fashion recommendation tasks, such as personalized fashion product recommendation, complementary (mix-and-match) recommendation, and outfit recommendation, have been posed and explored in the literature. The continuing research attention and advances impel us to look back and in-depth into the field for a better understanding. In this article, we comprehensively review recent research efforts on fashion recommendation from a technological perspective. We first introduce fashion recommendation at a macro level and analyze its characteristics and differences with general recommendation tasks. We then clearly categorize different fashion recommendation efforts into several sub-tasks and focus on each sub-task in terms of its problem formulation, research focus, state-of-the-art methods, and limitations. We also summarize the datasets proposed in the literature for use in fashion recommendation studies to give readers a brief illustration. Finally, we discuss several promising directions for future research in this field. Overall, this survey systematically reviews the development of fashion recommendation research. It also discusses the current limitations and gaps between academic research and the real needs of the fashion industry. In the process, we offer a deep insight into how the fashion industry could benefit from the computational technologies of fashion recommendation.

A Appendix

Table A.1.

NO.	Paper	Where	When	Method	Input	Dataset	Evaluation
1	Fashion Coordinates Recommender System Using Photographs from Fashion Magazines [61]	IJCAI	2011	1	2	18	1
2	Large Scale Visual Recommendations from Street Fashion Images [62]	KDD	2014	1	3	20	7
3	Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences [158]	ICCV	2015	2	1	29	2,7
4	Learning Compatibility across Categories for Heterogeneous Item Recommendation [48]	ICDM	2016	2	3	29	8
5	NeuroStylist: Neural Compatibility Modeling for Clothing Matching [143]	MM	2017	2	3,6	8,10	1,2
6	Compatibility Family Learning for Item Recommendation and Generation [135]	AAAI	2018	2	3	2,29	2
7	Neural Compatibility Modeling with Attentive Knowledge Distillation [142]	MM	2018	2	3,6	3	2
8	Fashion Sensitive Clothing Recommendation Using Hierarchical Collocation Model [197]	MM	2018	2	2	32,33	1
9	Learning Type-aware Embeddings for Fashion Compatibility [157]	ECCV	2018	2	3,6	6,7	3,6
10	Learning to Match on Graph for Fashion [174]	AAAI	2019	3	3,6	29	1
11	TransNFCM: Translation-based Neural Fashion Compatibility Modeling [176]	AAAI	2019	3	3,6	2,3	1, 2
12	Interpretable Fashion Matching with Rich Attributes [175]	SIGIR	2019	/	3,6,8	24	1
13	Context-aware Visual Compatibility Prediction [18]	CVPR	2019	3	3	2,29	3,6
14	Learning Similarity Conditions without Explicit Supervision [148]	ICCV	2019	2	3,6	2,7	3,6
15	Improving Outfit Recommendation with Co-supervision of Fashion Generation [92]	WWW	2019	2	1,4	3,8	1,2
16	Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation [91]	TKDE	2019	/	1,4	3,8	1,8
17	Toward Explainable Fashion Recommendation [149]	WACV	2020	/	2	5	7
18	Fashion Compatibility Modeling through a Multi-modal Try-on-guided Scheme [31]	SIGIR	2020	/	3	41	1,6
19	Fashion Outfit Complementary Item Retrieval [93]	CVPR	2020	2	1,7	7	3,6

Table A.1. Summary of Mainstream Research on Complementary Fashion Recommendation

Methods, input, and evaluation settings refer to Table 2, and “/” denotes other methods that are not listed in the table. Dataset refers to Table A.3.

Table A.2.

NO.	Paper	Where	When	Method	Input	Dataset	Evaluation
1	Recommending Outfits from Personal Closet [150]	ICCV(W)	2017	1	3	5	8
2	Learning Fashion Compatibility with Bidirectional LSTMs [42]	MM	2017	2	3	2	3,6
3	Mining Fashion Outfit Composition Using an End-to-end Deep Learning Approach on Set Data [82]	TMM	2017	2	3,5,7	4	6
4	Interpretable Partitioned Embedding for Customized Multi-item Fashion Outfit Composition [36]	ICMR	2018	3	2	13	7
5	Outfit Recommendation with Deep Sequence Learning [66]	BigMM	2018	2	1,4	2	8
6	Outfit Compatibility Prediction and Diagnosis with Multi-layered Comparison Network [165]	MM	2019	/	1,4	12	3,6
7	Dressing as a Whole: Outfit Compatibility Learning Based on Node-wise Graph Neural Networks [19]	WWW	2019	3	3,6	2	3,6
8	Low-rank Regularized Multi-representation Learning for Fashion Compatibility Prediction [67]	TMM	2019	/	2	2	8
9	Towards a Unified Framework for Visual Compatibility Prediction [138]	WACV	2020	3	3,7	2,7	3
10	Learning Tuple Compatibility for Conditional Outfit Recommendation [178]	MM	2020	4	3,7	40	3,6
11	Learning Diverse Fashion Collocation by Neural Graph Filtering [100]	TMM	2020	3	3	2,6,31	3,6
12	Learning the Composition Visual Coherence for Complementary Recommendation [87]	IJCAI	2021	4	3	2,7	3,5

Table A.2. Summary of Mainstream Research on Outfit Compatibility Modeling

Methods, input, and evaluation settings refer to Table 2, and “/” denotes other methods that are not listed in the table. Dataset refers to Table A.3.

A.1 Dataset Summary

A.1.2 Existing Datasets.

We present the organization of the existing datasets in Table A.3, which contains the publication year, the basic statistics, and a brief introduction to each dataset. From the dataset sorting, we can see that more than 30 datasets are proposed in the literature for fashion recommendation tasks, which are quite rich in kind. Although these datasets might be for different sub-tasks, the data options for some specific tasks are still too many. For example, based on Polyvore, around 14 datasets have been constructed, most of them applied for compatibility modeling or outfit recommendation. However, there is still no benchmark for certain tasks to fairly evaluate different approaches. Most existing datasets have been applied only once in evaluating the corresponding methods, which is sometimes unnecessary. Even though some methods want to emphasize certain features or information in addressing certain tasks, they could have expanded or improved the existing datasets rather than proposing a new one, usually introducing too many differences in the new dataset. For example, almost every Polyvore-based dataset has side information on the item category, but such information can be different, being fine-grained in some datasets (Li’s Polyvore categorizes items into 333 categories) but coarse in others (only 5 categories). Such differences in details can also affect the evaluation of different methods.

Table A.3.

No.	Dataset	Year	#User	#Item	#Outfit	Item Info	Description	Applied in
Polyvore
1	“Sets” on Polyvore [59]	2015	150	83k	NA	C	Outfit = Top + Bottom + Shoes.	[59]
2	Maryland Polyvore [42]	2017	NA	16k	22k	C, T	Outfit lengths are varied and less than 8.	[18, 19, 42, 66, 67, 87, 115, 138, 148, 176]
3	FashionVC [143]	2017	248	29k	21k pairs	C, I, T	Outfit = Top + Bottom.	[84, 91, 92, 143, 176]
4	Li’s Polyvore [82]	2017	NA	368k	195k	C, T	Outfit length is 4; Category taxonomy is unstructured.	[82]
5	Polyvore 409 [150]	2017	NA	644k	409k	C, I, T	Outfit lengths are varied; Likes and comments of outfits are given.	[149, 150]
6	Polyvore Outfits-D [157]	2018	NA	32k	175k	C, T	Outfit lengths are varied and less than 16; Categories are fine-grained.	[157]
7	Polyvore Outfits [157]	2018	NA	365k	68k	C, T	Outfit lengths are varied and less than 19; Categories are fine-grained.	[18, 72, 80, 87, 93, 100, 138, 148, 157]
8	ExpFashion [91]	2018	NA	51k	201k	C	Outfit = Top + Bottom; Each outfit has several comments.	[91]
9	Capsule [56]	2018	NA	7,478	3,759	C, I, H	Each item is labeled with the season, occasion and function to wear it.	[56, 72]
10	FashionVC+ [94]	2019	248	29k	21k	C, I, T	Outfit = Top + Bottom + Shoes	[94]
11	Polyvore-U [106]	2019	630	205k	150k	C, I	Outfit = Top + Bottom + Shoes; Four versions of data in total and the largest version is shown here.	[105, 106]
12	Polyvore-T [165]	2019	NA	84k	20k	C, I, T, V	Each outfit contains 3–5 items belonging to main categories.	[165]
13	Feng’s Polyvore [35]	2019	NA	7.53M	1.56M	C, I, V	Each outfit is associated with the number of likes.	[35]
14	Polania’s Polyvore [122]	2019	NA	NA	14k	/	Each outfit is composed of at least 2 items.	[122]
15	Lin’s Polyvore [90]	2020	150	159k	66k	/	Over 68% of fashion outfits are liked by one user only.	[90]
16	EVALUATION3 [198]	2020	NA	NA	18k pairs	I, A	Each top-bottom pair is labeled for its fashionability (good/bad/normal) by fashion experts with reasons.	[198]
17	Moosaei’s Polyvore [114]	2020	NA	256k	50k	I, T	Included users have over 100k followers; Each outfit contains at least two items and five, on average.	[114]
Other fashion communities (Instagram, Flickr, Pinterest, chictopia, etc)
18	Magazine [61]	2011	NA	NA	14,813	NA		[61]
19	WOW (Flickr) [99]	2012	NA	34k	24k	A, O	It is specifically for occasion-oriented clothing recommendation.	[99]
20	Fashion-136K (street) [62]	2014	8,357	NA	136k	I, H, B, L	It contains street photos in various poses and complex backgrounds.	[62]
21	Fashion144k (Chictopia) [137]	2015	14,287	NA	144k	I, T, H	Each post contains an outfit containing 3.22 fashion items, on average.	[137]
22	Street Fashion (street) [197]	2018	NA	NA	1M pairs	/	The fashion pairs are generated from 100k street photographs.	[197]
23	FashionKE (Instagram) [109]	2019	NA	NA	81k	C, A, O	It contains full-body human image.	[109, 159]
24	Lookastic (Lookastic) [175]	2019	NA	15k	31k	C, I, A	The dataset is divided into two parts: women and men.	[175]
25	Fashion Takes Shape (Chictopia) [130]	2019	180	NA	18k	C, I	Each image is labeled with the body shape (average or above average) of the user.	[130]
26	Shop the Look (Pinterest) [69]	2019	NA	38k	72k	C, I	Outfit = Top + Bottom; Both the product and scene-product pair images are provided; Product is labeled with bounding box in images.	[69]
27	Fashionist [159, 160]	2020	NA	NA	2893	C, A, O	It contains fashion images with natural backgrounds.	[159, 160]
28	SocialMediaRec (Lookbook) [193]	2020	2293	NA	229k	C, T, H	Included users have over 7k fans and 100 selfie posts; Age, likes, and fans of users are given.	[193]
Amazon
29	Clothing, Shoes and Jewelry [68, 110]	2015	NA	1.5M (2014) 2.7M (2018)	NA	T, I, C	It provides detailed user–item interaction information including reviewer, rating and time; Price and brand information is partially given.	[17, 18, 45, 47, 54, 68, 110, 135, 158, 174, 181]
30	bodyFashion [30]	2019	12k	76k	NA	C, I, T	User-item interaction records with size and rating on items.	[30]
31	Amazon Fashion Dataset [100]	2020	NA	NA	60k	C, O, S	This dataset is specifically for fashion collocation.	[100, 146]
Taobao
32	Taobao	2017	NA	61k	406k pairs	C	Item pairs are matched by the fashion experts.	[192, 197]
33	OSA (Taobao, JD, etc) [197]	2018	NA	20k	NA	/	Each item is described with two–three images of different human postures in various background.	[197]
34	iFashion [13]	2019	3.6M	4.5M	127k	C, I, T	Each outfit has at least four items; Each user interacts with over 40 outfits.	[13, 81, 90]
Other e-commerce platforms (IQON, Net-A-Porter, etc)
35	E-commerce matching (Net-A-Porter, etc) [97]	2018	NA	50k	68k pairs	I, T	Images for items are multi-view; Item descriptions covers information of category, brand, price etc; Items are matched by the fashion experts.	[97]
36	Style4Body -Shape[52]	2018	270	NA	348k	I	Body measurements of different users are given.	[50, 52]
37	IQON (IQON) [115]	2018	NA	200k	89k	I	Each outfit is liked for at least 50 times.	[115]
38	IQON3000 (IQON) [144]	2019	3,568	672k	309k	C, I, A, T	Each outfit has the price information and the number of likes.	[105, 128, 144, 186]
39	Zhou (Zalora, ASOS, etc) [196]	2019	NA	NA	NA	C, I, T	It contains full-body, half-body, product and detailed images, as well as the brand and price information.	[196]
40	IQON (IQON) [178]	2020	NA	NA	29k	C, I	Each outfit has been liked by over 70 people; Categories are fine-grained.	[178]
41	FOTOS (SSENSE) [31]	2020	NA	11k	20k	I, T	Each outfit is worn by one model in standard front-view pose.	[31]
42	VIBE (Birdsnest) [57]	2020	68	1957	NA	I, T	It contains dresses worn by models, different body shapes and the measurements are given.	[57]
43	[160]	2020	NA	NA	2893	C, I, A, O	User information such as age and gender are given.	[159, 160]

Table A.3. Existing Datasets for Fashion Recommendation Grouped by Data Sources

The numbers and quantities are reported approximately. k denotes thousands and M denotes millions. Abbreviation is used to denote different item information as well, C for Category, T for Textual description, including titles, I for Image, V for Votes, A for Attributes, O for Occasion, L for Location, S for Style, and H for Hashtag or other tags. NA denotes not applicable, and / denotes the information is not given in the original papers.

We can conclude that there is so far no satisfactory benchmark dataset for any sub-task in fashion recommendation. For personalized fashion recommendation, most methods have been developed based on the Amazon Review dataset [68, 110] (No. 23 in Table A.3) and are therefore more consistent in their development and more comparable. However, only applying one dataset somehow makes the results biased and unable to comprehensively evaluate the methods. Even though it is hard to find another dataset so far that has been specifically proposed for personalized fashion recommendation, we can create one based on the available extensive datasets. The iFashion dataset (No. 26 in Table A.3) was originally for the outfit recommendation task. But it provides the sequences of the user’s successive clicking on items, offering real-world user–item interaction data with another type of behavior and from another source, which can serve as a great complement for the Amazon dataset (No. 23) for the personalized fashion recommendation task.

For compatibility learning or complementary recommendation, as we mentioned above, although the available datasets are many, few of them have been applied more than once. In fact, most datasets are from three main sources: Polyvore, Taobao, and IQON, which have different characteristics, such as different outfit styles (western, Chinese, and Japanese). By combining the three sources of data, more solid benchmark datasets can be developed.

References

[1]

G. Mohammed Abdulla and Sumit Borar. 2017. Size recommendation system for fashion e-commerce. In KDD Workshop on Machine Learning Meets Fashion.

Abstract

A Appendix

A.1 Dataset Summary

A.1.2 Existing Datasets.

References

Cited By

Index Terms

Recommendations

Personalized Fashion Recommendation with Visual Explanations based on Multimodal Attention Network: Towards Visually Explainable Recommendation

Modeling Multi-Relational Connectivity for Personalized Fashion Matching

Personalized fashion outfit generation with user coordination preference learning

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

A.1.1 Data Sources.

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Full Text

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations