Synthesis Lectures on Information Concepts, Retrieval, and Services
3 total works
Nowadays, fashion has become an essential aspect of people's daily life. As each outfit usually comprises several complementary items, such as a top, bottom, shoes, and accessories, a proper outfit largely relies on the harmonious matching of these items. Nevertheless, not everyone is good at outfit composition, especially those who have a poor fashion aesthetic. Fortunately, in recent years the number of online fashion-oriented communities, like IQON and Chictopia, as well as e-commerce sites, like Amazon and eBay, has grown. The tremendous amount of real-world data regarding people's various fashion behaviors has opened a door to automatic clothing matching.
Despite its significant value, compatibility modeling for clothing matching that assesses the compatibility score for a given set of (equal or more than two) fashion items, e.g., a blouse and a skirt, yields tough challenges: (a) the absence of comprehensive benchmark; (b) comprehensive compatibility modeling with the multi-modal feature variables is largely untapped; (c) how to utilize the domain knowledge to guide the machine learning; (d) how to enhance the interpretability of the compatibility modeling; and (e) how to model the user factor in the personalized compatibility modeling. These challenges have been largely unexplored to date.
In this book, we shed light on several state-of-the-art theories on compatibility modeling. In particular, to facilitate the research, we first build three large-scale benchmark datasets from different online fashion websites, including IQON and Amazon. We then introduce a general data-driven compatibility modeling scheme based on advanced neural networks. To make use of the abundant fashion domain knowledge, i.e., clothing matching rules, we next present a novel knowledge guided compatibility modeling framework. Thereafter, to enhance the model interpretability, we put forward a prototype wise interpretable compatibility modeling approach. Following that, noticing the subjective aesthetics of users, we extend the general compatibility modeling to the personalized version. Moreover, we further study the real-world problem of personalized capsule wardrobe creation, aiming to generate a minimum collection of garments that is both compatible and suitable for the user. Finally, we conclude the book and present future research directions, such as the generative compatibility modeling, virtual try-on with arbitrary poses, and clothing generation.
Despite its significant value, compatibility modeling for clothing matching that assesses the compatibility score for a given set of (equal or more than two) fashion items, e.g., a blouse and a skirt, yields tough challenges: (a) the absence of comprehensive benchmark; (b) comprehensive compatibility modeling with the multi-modal feature variables is largely untapped; (c) how to utilize the domain knowledge to guide the machine learning; (d) how to enhance the interpretability of the compatibility modeling; and (e) how to model the user factor in the personalized compatibility modeling. These challenges have been largely unexplored to date.
In this book, we shed light on several state-of-the-art theories on compatibility modeling. In particular, to facilitate the research, we first build three large-scale benchmark datasets from different online fashion websites, including IQON and Amazon. We then introduce a general data-driven compatibility modeling scheme based on advanced neural networks. To make use of the abundant fashion domain knowledge, i.e., clothing matching rules, we next present a novel knowledge guided compatibility modeling framework. Thereafter, to enhance the model interpretability, we put forward a prototype wise interpretable compatibility modeling approach. Following that, noticing the subjective aesthetics of users, we extend the general compatibility modeling to the personalized version. Moreover, we further study the real-world problem of personalized capsule wardrobe creation, aiming to generate a minimum collection of garments that is both compatible and suitable for the user. Finally, we conclude the book and present future research directions, such as the generative compatibility modeling, virtual try-on with arbitrary poses, and clothing generation.
Graph Learning for Fashion Compatibility Modeling
by Weili Guan, Xuemeng Song, Xiaojun Chang, and Liqiang Nie
Published 3 November 2022
This book sheds light on state-of-the-art theories for more challenging outfit compatibility modeling scenarios. In particular, this book presents several cutting-edge graph learning techniques that can be used for outfit compatibility modeling. Due to its remarkable economic value, fashion compatibility modeling has gained increasing research attention in recent years. Although great efforts have been dedicated to this research area, previous studies mainly focused on fashion compatibility modeling for outfits that only involved two items and overlooked the fact that each outfit may be composed of a variable number of items. This book develops a series of graph-learning based outfit compatibility modeling schemes, all of which have been proven to be effective over several public real-world datasets. This systematic approach benefits readers by introducing the techniques for compatibility modeling of outfits that involve a variable number of composing items. To deal with the challenging task of outfit compatibility modeling, this book provides comprehensive solutions, including correlation-oriented graph learning, modality-oriented graph learning, unsupervised disentangled graph learning, partially supervised disentangled graph learning, and metapath-guided heterogeneous graph learning. Moreover, this book sheds light on research frontiers that can inspire future research directions for scientists and researchers.
This book systemically presents key concepts of multi-modal hashing technology, recent advances on large-scale efficient multimedia search and recommendation, and recent achievements in multimedia indexing technology. With the explosive growth of multimedia contents, multimedia retrieval is currently facing unprecedented challenges in both storage cost and retrieval speed. The multi-modal hashing technique can project high-dimensional data into compact binary hash codes. With it, the most time-consuming semantic similarity computation during the multimedia retrieval process can be significantly accelerated with fast Hamming distance computation, and meanwhile the storage cost can be reduced greatly by the binary embedding. The authors introduce the categorization of existing multi-modal hashing methods according to various metrics and datasets. The authors also collect recent multi-modal hashing techniques and describe the motivation, objective formulations, and optimization steps for context-aware hashing methods based on the tag-semantics transfer.