UI Concepts for Vector Search

In this article we explore novel UI/UX concepts for implementing vector search in online image search applications. We also look at backend implementations to support these UI elements and explore techniques such as semantic filtering, weighted query combinations, custom instructions, and search as recommendation. The ability to compose queries as a combination of multiple elements via mathematical operations is a property of vector search that often goes under-utilised, these simple operations can turn vector search systems into semantic filters, recommenders, or even search ensembles. However, in the real world it is unreasonable to expect an end user to be aware of any of these mechanisms, the challenge becomes one of how these can be presented in at intuitive and usable manner. This article demonstrates some of the many ways this can be achieved by drawing influence from existing UX patterns and repurposing emerging UI patterns from LLM applications.