Semantic Search

Definition

Semantic search is an information retrieval approach that understands the intent and contextual meaning behind a query, returning results that are conceptually relevant rather than lexically identical. By converting both queries and documents into dense vector embeddings, semantic search enables similarity-based retrieval that bridges vocabulary gaps—finding relevant content even when the exact query words don't appear in the documents. This is foundational to modern RAG systems and knowledge-grounded AI agents.

Engineering Context

Semantic search is what makes RAG work. Instead of finding documents that contain the exact words in a query, semantic search finds documents that are conceptually related. Implemented using cosine similarity or dot product between query embeddings and document embeddings. Hybrid search (combining dense embeddings + BM25 keyword matching) typically outperforms either alone. In production, approximate nearest-neighbor (ANN) algorithms like HNSW enable sub-millisecond search across millions of vectors. Reranking with a cross-encoder model as a second pass further improves result relevance before injection into the LLM context.

Related Terms