HyDE: Searching With an Answer That Might Be Wrong

Barakaeli Lawuo, Aug 27, 2026

In December 2022, Luyu Gao, Xueguang Ma, Jimmy Lin and Jamie Callan posted a retrieval paper with an unusual footnote: no models were trained or fine-tuned in making it.¹ They took two models off the shelf. One was InstructGPT (the text-davinci-003 model), a GPT-3 model tuned to follow instructions.^1,4 The other was Contriever, a text encoder trained without any relevance labels.^1,2 On the TREC Deep Learning 2019 web search queries, Contriever alone scored 44.5 nDCG@10. With their method, HyDE, the same Contriever scored 61.3. A Contriever fine-tuned on the MS MARCO training data, which is richly labeled for exactly this kind of search, scored 62.1.¹

The setting matters for reading that number. It is zero-shot: HyDE never saw a labeled query-document pair from any test set, and the authors counted the instruction tuning inside InstructGPT as the only supervision anywhere in the system.¹ They tested on 11 query sets: TREC DL19 and DL20 for web search, six low-resource sets from the BEIR benchmark, and Swahili, Korean, Japanese and Bengali from Mr. TyDi.^1,7,6 The trick is one sentence long. Ask a language model to write a passage that answers the question, then search with the embedding of that passage instead of the question.

This post follows one query through that pipeline, using the example the HyDE paper itself prints in its Figure 1.

Why the query needed a stand-in

Dense retrieval turns a query and every document into vectors and ranks documents by the inner product between them. Written out, the score is $\mathrm{sim}(q, d) = \langle \mathrm{enc}_q(q), \mathrm{enc}_d(d) \rangle$, where the two encoders map text to vectors of the same size.¹ The hard part, the HyDE authors argue, sits inside that equation. Both encoders have to land in one space where inner product means relevance, and without relevance judgments to fit, they call learning that space intractable.¹ A relevance label is a human mark saying this document answers this query. MS MARCO's passage task has 503 thousand training queries,⁵ but MS MARCO restricts commercial use, and many real search setups have nothing like it.¹

Contriever came from the other direction. Gautier Izacard and colleagues at Meta AI trained a BERT-base encoder with contrastive learning on documents alone, drawn half from Wikipedia and half from CCNet web text.² To make a training pair, they cut two random spans out of the same document, deleted 10% of the tokens in each, and taught the model to put those two spans close together and spans from other documents far apart.² They call this independent cropping, and they point out that it is symmetric: both sides of a pair come from the same distribution of text.² One encoder embeds queries and documents alike, and a text's vector is the average of the last layer's hidden states.²

So Contriever learned what makes two pieces of text similar as passages, without ever being taught what makes a passage answer a question. On BEIR it beat BM25 on 11 of 15 datasets by Recall@100, but still trailed BM25 on nDCG@10.² BM25 is the classic keyword-weighting method that scores documents by shared terms. HyDE's move is to stop asking Contriever to compare a question with a passage at all. It turns the question into a passage first, so the search becomes document against document, the one comparison Contriever was trained for.¹

Step one: the query and the instruction

The query in the paper's figure is a real web-search-style question, typed the way people type: how long does it take to remove wisdom tooth.¹ No capital letter, no question mark, missing an article. Before it reaches any index, HyDE wraps it in an instruction. For the web search sets the appendix gives the exact prompt:¹

Each dataset got its own version with a different quantifier. SciFact asks for "a scientific paper passage to support/refute the claim," FiQA for "a financial article passage," TREC-NEWS for "a news passage about the topic," and Mr. TyDi for a passage in Swahili, Korean, Japanese or Bengali "to answer the question in detail."¹ The instruction is the only place a task description enters the system. The encoder is the same for every dataset.¹

Step two: a passage that nobody checked

InstructGPT then writes the passage, sampled at temperature 0.7, the OpenAI playground default.¹ For the wisdom tooth query, the snippet the paper shows reads: "It usually takes between 30 minutes and two hours to remove a wisdom tooth..."¹ That is the hypothetical document. The authors are blunt about what it is: not real, able to contain factual errors, and likely to be ungrounded.¹ They only need it to look like a relevant document. In their framing, the language model captures relevance by producing an example of it, which moves relevance modeling out of the encoder and into a generator that follows instructions.¹

This changes what the encoder sees. The raw query is nine words of search shorthand. The draft is fluent prose with a duration, a procedure name and the vocabulary a dental page would use. Whether its two-hour figure is right does not matter yet.

Step three: averaging drafts into one query vector

A generator samples, so it could write a different passage each time. HyDE treats the query vector as an expected value over those samples. It writes $g(q, \mathrm{INST})$ for the instruction model's output, calls the Contriever encoder $f$, and estimates the vector by drawing $N$ documents and averaging their embeddings.¹ The query itself can go into the average as one more hypothesis:¹

Term by term: $q$ is the query text and $\mathrm{INST}$ the dataset's instruction. $\hat{d}_k$ is the $k$-th generated document, and the hat marks it as generated, not taken from the corpus. $N$ is how many are sampled. $f$ is the unchanged Contriever encoder, the same function that embedded every real document ahead of time. $f(q)$ is the query's own embedding, added so the average has $N+1$ members, which is why the fraction is $1/(N+1)$. The result $\hat{v}_q$ is a single vector of the same size as every document vector.¹

Averaging comes with an assumption that the paper states outright. Taking a plain expectation assumes the distribution of query vectors is uni-modal, which the authors gloss as "the query is not ambiguous."¹ The arXiv version does not say in its text what value of $N$ the experiments used.

Step four: the nearest real passage

HyDE takes the inner product between $\hat{v}_q$ and every document vector in the corpus and returns the closest ones.¹ The experiments ran this through the Pyserini toolkit with a standard maximum inner product search index, no special index, no training.¹ For the wisdom tooth query, the figure's retrieved real document reads: "How wisdom teeth are removed... Some ... a few minutes, whereas others can take 20 minutes or longer...."¹

Put the two snippets side by side. The draft says 30 minutes to two hours. The real passage says a few minutes to 20 minutes or longer. My reading is that the draft got the specific numbers wrong and still landed on the right document, which is the behavior the authors designed for. They describe the encoder's dense bottleneck as a lossy compressor: squeezing a passage into one vector filters out the extra, hallucinated details, and the nearest-neighbor search grounds what is left in real documents.¹ The figure shows the same pattern in Korean. The generated passage dates human use of fire to about 8 million years ago; the retrieved passage says 1.42 million years, in the time of Homo erectus.¹

Note what never got computed. At no point did the system score the query against a document. The authors point this out themselves: with HyDE's factorization, query-document similarity is no longer modeled or computed, and retrieval is split into a generation task and a document-document similarity task.¹

What happened across the 11 query sets

nDCG@10

A score for the top 10 results that rewards putting the most relevant documents highest. Contriever's authors describe it as suited to rankings shown to people.²

Recall@k

The share of all relevant documents that appear anywhere in the top k results. Contriever's authors use it for retrievers that feed systems like question answering models, which read hundreds of documents and ignore their order.²

MRR@100

The average over queries of 1 divided by the rank of the first relevant result, counting only the top 100. Mr. TyDi's baselines are tuned for it, and HyDE reports it for Mr. TyDi.^6,1

The 2019 track judged 43 test queries over a corpus of 8.8 million passages.⁵ On DL19 HyDE matched the fine-tuned Contriever on precision and beat it on recall@1k, 88.0 against 83.6. On DL20 it fell about 10% short on nDCG@10 and map, with similar recall.¹ Without HyDE, plain Contriever lost to BM25 on both years. With it, HyDE beat BM25 by about 10 points of nDCG@10 each year.¹

The low-resource BEIR sets told the same story, with one case worth dwelling on. On TREC-COVID, Contriever scored 27.3 nDCG@10 and BM25 59.5. HyDE brought Contriever to 59.3, so BM25 stayed ahead there, but only by 0.2.¹ The Contriever paper had already given a likely reason for the weak baseline: its training data was collected before the COVID outbreak.² The encoder did not change, so the gain came from a generator that could write COVID-shaped text for it to embed.

The generator's size mattered. The authors swapped InstructGPT for smaller instruction models, keeping Contriever fixed.¹

Every generator helped, and larger ones helped more. The authors add a caution that the Cohere model was experimental and undocumented at the time, so training techniques may explain part of the gap, not size alone.¹

Query2doc puts the question back in

Three months later Liang Wang, Nan Yang and Furu Wei at Microsoft Research published Query2doc, which uses the same kind of generated passage in a different way.³ Instead of a zero-shot instruction, they few-shot prompt text-davinci-003: the instruction "Write a passage that answers the given query:" followed by 4 labeled query-passage pairs sampled from MS MARCO training data, sampled at temperature 1 with at most 128 tokens.³ Their Figure 1 shows the output for "when was pokemon green released": a passage saying it came out in Japan on February 27th, 1996.³

Then they keep the query. For BM25, the new query is the original repeated 5 times and concatenated with the pseudo-document, because the passage is much longer than the query and would otherwise swamp its terms. For dense retrievers, it is the query, a [SEP] token and the passage.³ Unlike HyDE, the dense retrievers in Query2doc were trained with the expanded queries, so this is not zero-shot on that side.³

With no fine-tuning, BM25 plus Query2doc rose from 51.2 to 66.2 nDCG@10 on DL19 and from 47.7 to 62.9 on DL20.³ The ablation shows why they kept the query:

Alone, the generated passage did worse than the bare query under BM25; together they did far better. The authors read this as the two being complementary.³ In the same paper's main table, HyDE appears at the 61.3 and 57.9 it reported for DL19 and DL20, below BM25 plus Query2doc.³ Keep the caveat in mind: that comparison mixes a zero-shot dense method with a keyword method whose prompt contains labeled examples. For trained dense retrievers the gains were smaller. DPR went from 33.7 to 35.1 MRR@10 on the MS MARCO dev set, while SimLM and E5, which already distill from a cross-encoder reranker, gained 0.4 and 0.8.³

Query2doc also states a criticism of HyDE directly: HyDE implicitly assumes the ground-truth document and the pseudo-document express the same meaning in different words, which may not hold for some queries.³

Where the drafts go wrong

Hallucinated specifics that do not wash out. The wisdom tooth example shows the encoder ignoring a wrong number. Query2doc shows the other side. For "who sings monk theme song," the generated passage correctly named Randy Newman and "It's a Jungle Out There," but said the song had been the theme since the 2002 premiere. It was used from season two, in 2003.³ The authors call such errors subtle and hard to verify, and a serious obstacle to building trustworthy systems on LLM output.³ Retrieval can still work with that passage. A system that shows the passage to a user, or lets it stand in for retrieval, would pass the error along.

Languages the models know less well. The HyDE authors expected trouble here. A small contrastive encoder gets saturated as the number of languages grows, and a large generator can be under-trained on languages with less data than English or French.¹ On Mr. TyDi, HyDE improved mContriever in all four languages, but in Bengali it scored 41.3 MRR@100 against BM25's 41.8. In Swahili it reached 41.7 while the fine-tuned mContriever reached 51.2.¹ Their hypothesis is that these languages are under-trained in both pre-training and instruction tuning.¹

Weak generators and vague instructions. Query2doc found that texts from smaller models "tend to be shorter and contain more factual errors." The 1.3B babbage model moved BM25 on DL19 only from 51.2 to 52.0.³ HyDE trailed the fine-tuned Contriever most clearly on FiQA and DBPedia, financial posts and entities, and its authors blame under-specified instructions.¹ Query2doc's out-of-domain results were mixed too: with SimLM and E5, SciFact nDCG@10 fell by 2.9 points.³

Cost at query time. Query2doc measured it. Retrieving the top 100 BM25 results took 16 ms of index search. With Query2doc, the LLM call alone took more than 2,000 ms, and the index search rose to 177 ms because the expanded query has many more terms.³ HyDE's own conclusion suggests a way to live with this. Use HyDE when a search system is new and has no logs, then roll out a supervised retriever as logs accumulate, sending only rare and emerging queries to the HyDE backend.¹

The assumption the paper left open

Go back to step three. The average in equation 8 makes sense when every sampled draft is a variation on one meaning. If a query could mean two things, the drafts would split between two neighborhoods of the embedding space, and their mean could fall between them, near neither kind of relevant document. That picture is my reading of the equation; the paper does not test it. What the authors do say is that they simply take the expectation, assuming the distribution is uni-modal and the query is not ambiguous, and that "the study of ambiguous queries and diversity is left to future work."¹