Adding AI search to your website: semantic search and RAG implementation guide

Adding AI search to your website: semantic search and RAG implementation guide

Here is a typical pain point: a documentation site with 400+ tutorials, where searching "how to change the page title" returns nothing — because the pages literally say "modify the title tag." Keyword search matches on the literal string, so users must use the exact wording to hit anything, while real users phrase things all kinds of ways. AI semantic search encodes "text" into "vectors" and looks for nearest neighbors in semantic space — even with completely different phrasing, it returns content that means the same thing. Below is an actionable walkthrough from principles to implementation to tool selection.

Semantic vs keyword search

To be clear: semantic search doesn't replace full-text search. In practice the two are combined into "hybrid search" — run BM25 for keyword hits, then fuse with vector results using weighted scores. Recall is noticeably better than either approach alone.

Architecture

Three core components, all required:

1. Embedding model: converts text into ~1536-dimension vectors (text-embedding-3-small); the more similar the text, the closer the vectors;
2. Vector database: stores vectors, builds indexes, runs nearest-neighbor queries;
3. LLM generation (optional): organizes retrieved results into a natural-language answer — i.e. RAG.

What people overlook is chunking. Embedding a whole page in one go dilutes the vector; instead split by paragraph or 500-1000 characters with 50-100 characters of overlap, and attach metadata like title and URL so context isn't lost during retrieval.

Implementation options

Option 1: Vector DB semantic search (recommended)

Using OpenAI embeddings + Chroma, the code is short and good for a proof of concept:

from openai import OpenAI
import chromadb

client = OpenAI()
chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection("website_content")

# 1. Index documents
def index_content(docs):
    for i, doc in enumerate(docs):
        response = client.embeddings.create(
            model="text-embedding-3-small",
            input=doc["text"]
        )
        collection.add(
            embeddings=[response.data[0].embedding],
            documents=[doc["text"]],
            metadatas=[{"title": doc["title"], "url": doc["url"]}],
            ids=[f"doc_{i}"]
        )

# 2. Search
def search(query, n_results=5):
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=query
    )
    results = collection.query(
        query_embeddings=[response.data[0].embedding],
        n_results=n_results
    )
    return results

A few engineering notes: the embeddings API is rate-limited, so batch indexing needs concurrency control; when data updates, use upsert (overwrite by same id) rather than repeated add, or the vector store fills up with stale versions.

Option 2: RAG search (retrieve + generate)

On top of semantic search, let an LLM answer based on the retrieved content, with sources cited to avoid hallucination:

def rag_search(query):
    # 1. Retrieve relevant content
    context_docs = search(query)
    context = "\n\n".join(context_docs["documents"][0])
    
    # 2. Generate an answer
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Answer the user's question based on the following content. If the content is not enough to answer, say so honestly.\n\nRelevant content:\n{context}"},
            {"role": "user", "content": query}
        ]
    )
    return response.choices[0].message.content

The key to RAG is returning the "sources" alongside the answer so users can click through and verify — the standard pattern for enterprise knowledge-base Q&A; for the integration approach, see website AI chatbot integration.

Frontend integration

// Search frontend component
class AISearch {
    constructor(inputEl, resultEl) {
        this.input = inputEl;
        this.results = resultEl;
        this.debounceTimer = null;
        
        this.input.addEventListener('input', (e) => {
            clearTimeout(this.debounceTimer);
            this.debounceTimer = setTimeout(() => {
                this.search(e.target.value);
            }, 300);
        });
    }
    
    async search(query) {
        if (query.length < 2) return;
        
        const response = await fetch('/api/search', {
            method: 'POST',
            headers: { 'Content-Type': 'application/json' },
            body: JSON.stringify({ query })
        });
        
        const data = await response.json();
        this.renderResults(data);
    }
    
    renderResults(data) {
        this.results.innerHTML = data.map(item => `
            <a href="${item.url}" class="search-result">
                <h4>${item.title}</h4>
                <p>${item.excerpt}</p>
            </a>
        `).join('');
    }
}

The frontend debounces for 300ms so every keystroke doesn't fire a request; in production, add a backend cache layer (cache identical queries for 5-10 minutes) to keep costs down.

Vector DB comparison

Selection advice: start small with Chroma or pgvector at zero cost; once you hit millions of vectors or need an SLA and elasticity, consider the managed Pinecone/Qdrant tiers. If your site already runs PostgreSQL, adding the pgvector extension is the least-effort route — no new components.

Cost estimate

With text-embedding-3-small, 1M tokens cost about $0.02 — indexing a 500-page site at ~2,000 characters per page costs under $1 in total; embedding at query time is nearly negligible. The real cost center is LLM generation tokens in RAG scenarios. All told, a small-to-mid site's semantic search usually lands within a few dollars a month.

How to measure whether search got better

"Is it accurate?" shouldn't be a gut feeling. Before launch, prepare 50-100 real user questions as a test set and measure the hit rate of "is the correct answer within the top 3 results" as your iteration baseline. After launch, track three signals: zero-result rate (share of searches returning nothing), click-through on search results, and the share of searches that immediately spawn a new query — the lower that last one, the better the results. Run the test set weekly, and compare against the baseline after every model or chunking change. That's how semantic search keeps getting better.

16IDC Takeaway

AI semantic search lets a site's content be discovered far more fully: on content-heavy sites (docs, blogs, knowledge bases) user satisfaction typically runs 40-60% higher than with traditional search, because users stop having to "guess the keyword." Start with Chroma or pgvector, get "index + retrieve" working, then layer on RAG and hybrid search. For more AI-related applications, browse the related articles here.

Reference: OpenAI Embeddings docs https://platform.openai.com/docs/guides/embeddings; Chroma docs https://docs.trychroma.com/