SearchAI
Semantic search for artworks using multimodal embeddings, enabling intuitive exploration beyond keywords.
Example output
Examples of artworks retrieved through semantic search queries combining visual and textual meaning.
What this project illustrates
-
Building end-to-end semantic search systems from data ingestion to UI
-
Leveraging multimodal embeddings to move beyond keyword-based search
-
Designing scalable search architectures for large visual datasets
-
Bridging machine learning, search infrastructure, and user experience
Context & intent
Context
Traditional search engines rely heavily on keywords and metadata, which limits discovery in large visual collections.
In art databases, users often struggle to express what they are looking for using precise terms, especially for abstract or conceptual queries.
This creates friction between user intent and available content.
​
Approach
SearchAI uses multimodal embeddings to represent both artworks and user queries in a shared vector space.
By encoding visual and textual information, the system retrieves artworks based on semantic similarity, rather than exact keyword matches.
Users can search using natural language queries such as moods, styles, or abstract concepts.
​
Value
-
Enable intuitive, natural-language search over visual content
-
Improve discovery in large-scale artwork collections
-
Retrieve conceptually related results, not just literal matches
-
Combine visual and textual understanding in a single search flow
-
Provide a foundation for scalable semantic retrieval systems
Project management & key decisions
-
Choosing a multimodal embedding approach to align text queries and images in the same representation space
-
Favoring semantic similarity search over traditional keyword indexing
-
Designing a cloud-native architecture to support scalability and deployment
-
Structuring the project end-to-end, from data scraping to UI, to reflect real product constraints
Tech stack
-
Frontend: React, Netlify
-
Backend: FastAPI, Python
-
ML: Multimodal embedding model (text + image vectors)
-
Search engine: Vector-based semantic search
-
Infrastructure: Docker, Google Cloud Run, Artifact Registry, Google Cloud Storage
-
Data source: The MET Museum Open Access API
Scientific background
SearchAI builds on research in representation learning and multimodal models, showing that text and images can be embedded into a shared semantic space.
These embeddings make it possible to measure conceptual similarity across modalities and enable retrieval of visually or semantically related content beyond surface-level descriptions.