Bringing multimodality to Amazon visual search system
Published in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2024
Pure image-to-image matching in visual search suffers from false positives caused by matching local visual patterns. This paper adds image-text alignment losses, drawing on vision-language pretraining, as constraints on the image-to-image matching loss in deep metric learning, and applies the approach to the Amazon visual search system.