Embeddings and keyword search

Search that looks at words, and search that looks at vectors, fail in different places. Keyword search finds the string you typed. An embedding search finds text that a model placed near your query in a space of numbers. Teams pick one, then discover they needed both, usually after a confident answer cited the wrong page.

What a keyword index does

A keyword index stores terms. The query "invoice 1044" finds documents that contain those tokens, subject to stemming, stop words, and the analyzer you configured. If the document says "bill number 1044" and your query says "invoice," a plain keyword search can miss it. If the document says "invoice 1044" in a footer on every page, the search can return the whole archive.

That strictness is the feature. Error codes, SKUs, names, and quoted phrases are keyword problems. You want the string, or you want to know it is absent.

What an embedding is

An embedding model reads a piece of text and emits a vector, a fixed list of numbers. Texts the model considers similar get vectors that point in similar directions. "Car" and "automobile" can land near each other without sharing letters. The index stores those vectors. At query time you embed the question and ask for the nearest stored vectors.

The embedding model and the chat model are often different models. Changing the chat model does not rebuild the index. Changing the embedding model does. Old vectors and new vectors are not comparable. You re-embed the collection when you switch.

Where the neighbor is wrong

Nearness is a clue about the embedding model's training, and it is a weak clue about your files. "Bank" beside a river and "bank" as a firm can sit near each other. A policy from 2019 can outrank the policy from this year because the wording is closer to the question. A chunk that says "do not do X" can rank beside a chunk that says "do X" because the words overlap.

The search result is a passage to read, or to hand to a model as source text. It is not a verdict. Retrieval before the model writes is the step where that passage gets pasted into the prompt. If you skip the reading, the generator will speak in the voice of whichever neighbor came back.

Chunks decide what can be found

Long pages are split before they are embedded. A split that cuts a sentence from its heading produces a vector for a fragment. The question was about the heading. The fragment does not contain it, so the nearest-neighbor search never had a chance.

A page that was never passed through the embedding model is absent. There is no fuzzy match that recovers a file you forgot to index. Keyword and vector indexes have the same blind spot here: what you did not store, you cannot retrieve.

A useful index for ordinary company text is often both. Keywords for the identifier. Vectors for the paraphrase. A person, or a careful prompt, still has to check that the passage answers the question that was asked.

A split you can test on your own files

Take one real question and one identifier. "What is our return window?" is a paraphrase. "RMA-4418" is a string. Run both against the keyword index and against the vector index, and write down the top three titles, not the generated paragraph.

On the paraphrase, the vector index often surfaces a policy that never uses the words "return window," and the keyword index often surfaces a page that happens to contain "return" in a footer. On the identifier, the keyword index should find the row that contains RMA-4418. If the vector index returns a nearby code instead, that is the failure mode to remember: nearness is not the string.

Then look at the chunk, not the title. If the heading is right and the body is the previous section, the split cut the page in the wrong place. Fix the chunk. Do not switch chat models and hope the generator repairs a bad neighbor.

Keep a note of which embedding model produced the vectors. The day you change it, the old vectors are a different space. Queries embedded with the new model against the old index return neighbors that are meaningless. Re-embed the collection as one job, or keep two indexes until the new one is full.

Keyword and vector search also share an operations problem that has nothing to do with meaning. A file added to the folder and never sent through the indexer is invisible to both. The generator will still answer. The log of retrieved ids is how you notice that the folder and the index have drifted. If the id you expected is absent from that log, stop tuning the prompt. The page was never a candidate.