Query Rewriting

Goal

Rewrite a user's question when the document collection uses different vocabulary, preserve the user's meaning, and compare retrieval before and after the rewrite instead of assuming a longer query is better.

Users do not always phrase questions in the same vocabulary as the documents. User:

It dies too fast after the patch.

Document:

Battery duration regression after software update.

A retrieval system can rewrite the query toward corpus vocabulary. But rewriting creates a new risk: query drift.

Start with a vocabulary mismatch

A user writes:

It dies too fast after the patch.

The support documents use very different words:

Battery duration regression after software update.

A literal keyword search may miss the useful document even though the two sentences describe the same problem. One option is to create a second search query such as:

battery duration regression after software update

That transformation is query rewriting. The rewritten text is not the user's new request. It is a retrieval tool that tries to express the same information need in words that match the collection.

This step can help, but it can also damage the search. If a rewrite changes “XR-8” to “XR-9,” adds a product the user never mentioned, or turns “after the patch” into “before the patch,” retrieval may become confidently wrong. Keep the original query, log the rewrite, and judge the change by whether relevant documents actually move into useful positions.

Expansion can add useful synonyms

A simple rule-based expansion might map:

patch → software update
dies too fast → short battery duration

This can help lexical retrieval while preserving the core intent. But expansions should be domain-reviewed. If “reset” means factory reset in one product and session reset in another, adding the wrong synonym can hurt.

Rewriting should not invent missing constraints

User:

What's the return window?

A rewrite should not silently add:

for enterprise customers in Finland

unless those constraints came from trusted session state. Query rewriting may normalize wording, expand known terms, or resolve explicit context. It should not invent user attributes or desired filters.

Evaluate original versus rewritten retrieval

For a held-out set, compare:

original query recall@k
rewritten query recall@k

Also inspect regressions. A rewrite strategy that improves average recall but breaks exact product-code queries may need a conditional rule. For example:

if exact identifier present:
    preserve it exactly
else:
    allow semantic expansion

Log both queries

When debugging, store:

• original user query;
• rewritten retrieval query;
• rewrite method/version;
• retrieved chunk IDs.

Without the original, you cannot tell whether the rewrite changed intent.

Rewrite drift is an evaluation target

A rewrite can be fluent and still be wrong. Suppose the user asks:

Which Cedar policy applies to contractors?

A rewrite such as:

Cedar employee policy

may retrieve many relevant-looking documents while dropping the contractor constraint. This is query drift: retrieval improves for a different question. A useful evaluation stores the original query, rewrite, retrieved IDs, and expected relevant documents. Then it can detect cases where the rewrite increased apparent similarity but changed intent.

Multiple rewrites can improve recall, but increase cost

Some systems generate several query variants and union their candidate sets. That can help when the corpus uses different vocabulary from the user. But every extra variant adds retrieval work and can introduce more distractors.

Evaluate whether multi-query expansion improves relevant-document recall enough to justify the larger candidate set and latency. More queries are not automatically better.

A rewrite is useful only if it preserves the user's information need

Query rewriting can add missing vocabulary, resolve shorthand from conversation context, or split a broad question into retrieval-friendly subqueries. But every rewrite is also a chance to change meaning. A system that silently turns “Can I return this?” into “What is the 30-day return policy?” may exclude an exception the user actually needed.

Keep the original query beside the rewritten form and evaluate them as a pair. Good tests include ambiguous references, product identifiers, negation, and questions where the correct action is to ask for clarification rather than invent context. When a rewrite helps retrieval, the trace should show which new terms or decomposition caused the relevant evidence to appear.

Predict

What is query drift?

A rewrite changes the retrieval intent away from the user's actual requestThe vector index uses a different row order after rebuildingA user submits the same query twice

Run the rewriting Lab

The Lab rewrites informal words into the vocabulary used by the policy documents, then measures how many words the query shares with one policy chunk.

1. Click Run once. rewrite_query returns the query unchanged, so rewritten equals original and one check fails.
2. Complete the TODO: replace each whole word that appears in synonyms, and leave every other word—especially identifiers such as XR-417—exactly as it was.
3. Click Run again. You should see rewritten: XR-417 price policy for employee and overlap with policy chunk (original -> rewritten): 2 -> 4. The rewrite doubled the shared words, and the product code survived.
4. Add a wrong synonym on purpose: change "rules": "policy", to "rules": "banana",.
5. Click Run. The rewritten query becomes XR-417 price banana for employee, and the overlap falls to 3. One bad dictionary entry made retrieval worse. The Lab should report Result: experiment ran because the original synonym baseline changed while the rewrite invariants still pass.
6. Keep the original and rewritten queries side by side in your notes; that record is how you catch a rewrite regression later. Press Reset afterward, after copying your function.

Loading lab…

Quick Check

1. What is the main goal of query rewriting?

Improve retrieval match while preserving user intentReplace access-control filters with a better queryMake every query more descriptive, even if the rewrite changes the user's original intent

2. Why preserve exact identifiers during rewriting?

They may carry lexical evidence that semantic paraphrasing could loseExact identifiers skip authorization checksExact identifiers should be paraphrased into natural language so dense retrieval can match their meaning better

3. Which log record lets you tell whether rewriting changed intent or retrieval?

Original query, rewritten query, method/version, and retrieval resultsThe rewritten query after discarding the originalThe final answer and score without either query

0 of 3 questions answered.

Explain it back

Rewrite one vague query into corpus-friendly vocabulary. Then identify one term you would preserve exactly and one evaluation that would detect query drift.

Key Takeaways

• Query rewriting is an upstream retrieval intervention.
• Better vocabulary match must not change user intent.
• Exact identifiers often need preservation.
• Rewrites should not invent missing constraints.
• Compare original and rewritten retrieval on the same held-out cases.

Next Lesson

Next, construct a grounded model prompt that clearly separates retrieved evidence from trusted application instructions.

References

• Lewis et al., Retrieval-Augmented Generation.
• Thakur et al., BEIR.