Understanding the rich world of outfits: a study of fashion compatibility, latent style, and outfit behavior

Many computer vision applications in the fashion domain require solving tasks where complex relationships between images, such as the notion of item compatibility, are being learned. We take a metric learning approach to representing compatibility between pairs of items. First, we introduce a model that learns compatibility relationships in dedicated embedding subspaces dependent on item type, which results in significant gains on established fashion compatibility prediction tasks. Second, we present a method for learning a richer notion of compatibility across multiple compatibility conditions whose contributions are learned as a latent variable, which provides better performance on established tasks while requiring fewer embedding subspaces to be learned. Third, we make the first published attempt at diagnosing the salient features of a pair of items that make them compatible, and linking them to human-interpretable concepts. Finally, we demonstrate that our representation of outfits enables diverse, novel, and practically-useful visual search queries for the fashion domain, and results in semantically-meaningful style summaries with several directions for future work.LimitedAuthor requested closed access (OA after 2yrs) in Vireo ETD syste