01 — THE CHALLENGE
A useful model starts with the right question.
Traditional metadata search breaks down when tags are missing, inconsistent or too literal. The goal was to make a visual catalog searchable through meaning and appearance.
02 — Multimodal Search · Retrieval
SearchAI
Find an image with an idea, not a filename.
Try the live demo↗PROJECT OVERVIEW
A multimodal retrieval engine that connects natural-language intent with visual content through shared text-image representations.
Project anatomy
From visual intent to the closest artworks.
The engine projects text and images into a shared CLIP space, then queries a vector index enriched with catalog filters and metadata.
Core pipeline
Select a step to understand how the data is transformed.Selected step
Artworks, descriptions, authors, categories, dates and image URLs are prepared as searchable documents.
Implemented features
Visible capabilities are connected to the technical components that make them possible.Natural-language search
Descriptions of a mood, material or period become visual queries.
Shared text-image space
Similarity relies on CLIP instead of keyword overlap alone.
Literal search
A keyword mode remains available when exact wording is relevant.
Filters and facets
Category, author, period and other metadata refine semantic ranking.
Real catalog
The demonstration uses artworks and metadata from The Metropolitan Museum of Art.
Response measurement
Hit count and processing time are exposed with every search.
Stack by responsibility
Each technology is placed within its role in the system.React · Axios
Input, filters and results grid.
FastAPI · Pydantic
Search contract, validation and typed response.
CLIP · PyTorch
Text and image encoding in a shared space.
Meilisearch
Vector index, keyword search, filters and facets.
Google Cloud Storage
Document loading, settings and request logging.
Docker · Cloud Run
Reproducible API and indexing-engine deployment.
Multimodal search engineBased on the current codebase · Simplified representation
From technology to use
Describe an intent and follow the semantic ranking.
The demonstration lets you search for an artwork in everyday language, then shows how proximity between the query and images organizes the results.
Educational demonstration: data is local and some computations are simulated or accelerated.04 — OUTCOME AND IMPACT
“Users can describe a visual idea in natural language or start from an image and retrieve semantically aligned content across a large catalog.”