When keyword search beats a vector match

Two people can mean the same thing in different ways

A support agent types a part number into a knowledge base. Another asks why a pump loses pressure after a restart. The first query depends on exact characters. The second may need a document that describes the same problem using different words.

Keyword search is useful when the wording itself carries the identity of the answer. Vector search represents text as numbers intended to capture aspects of meaning. It can help when a question and a useful passage use different vocabulary. Neither method is automatically better for the whole collection.

Consider parts AX-140 and AX-141. Their manuals may be almost identical, but a small installation difference can matter. A semantically similar result for the wrong part is not a near success. It is the wrong answer. Conversely, an exact keyword query may miss a useful troubleshooting guide because the customer says "cuts out" and the manual says "intermittent shutdown".

Combine candidate lists, not incomparable scores

A hybrid pipeline runs both kinds of retrieval and combines the documents they propose. The first lists are candidates, meaning possible answers that still need to be ordered and selected. Keep document identifiers attached so a result found by both branches can be recognised as the same source.

Raw keyword and vector scores usually have different meanings and scales. Adding them directly can make one branch dominate for reasons unrelated to quality. Rank-based fusion avoids that particular problem by using a document's position in each list. Reciprocal rank fusion is one such method, and Azure AI Search documents how it applies it to parallel queries.

Add another ranking stage only when it helps

A reranker examines the question together with candidate text and produces a new ordering. It can improve the final selection, but it cannot recover a document that neither retrieval branch supplied. Increasing reranking sophistication before checking candidate coverage often adds cost without fixing the real problem.

Keep a keyword-only baseline. Compare it with vector-only and combined retrieval using the same questions and source snapshot. Split results by query type, especially exact identifiers, abbreviations, natural-language questions and questions with no supported answer.

For a small collection of well-named documents, improved fields and keyword handling may be enough. For a varied collection with inconsistent terminology, semantic retrieval may add useful coverage. Let those observations decide whether a second branch is worth operating.

Make incorrect confidence visible

A hybrid system nearly always finds something that looks related. That does not mean the collection contains the requested answer. A question about an unsupported product can retrieve the closest manual and invite the model to fill in the gaps.

Inspect the selected source before judging the fluency of the response. Does it name the correct product, version and condition? Can the answer point to the passage that supports its main claim? Include these checks in the test set rather than relying on a single average relevance score.

The architecture is successful when it improves the reader's ability to find the right material. More retrieval stages, larger vectors and a longer context are implementation choices, not evidence of that outcome.