Divergence Engines (Part 1): Escaping the Relevance Trap

A common retrieval path is simple:

embed → find nearest neighbors → re-rank → summarize → generate

An embedding compresses a document into a representation that a search index can compare with the query. The nearest candidates may pass through a re-ranker, then a summarizer, before they reach the response model. Each stage can serve a legitimate purpose. If I need the date of a meeting or the paragraph where I discussed a specific idea, precision is welcome.

I need something else when I am trying to think. The nearest documents may share vocabulary and assumptions; a re-ranker may favor the same well-supported interpretation; a summary may smooth over disagreement. None of these effects is guaranteed by the mere presence of embeddings or a graph. They depend on how candidates are selected, scored, and reused. Still, a pipeline optimized only for the closest answer has little reason to return a weak tie, a contradiction, or a recent idea that has not acquired the familiar vocabulary yet.

Imagine a notebook with three incompatible approaches to the same problem. A query using the vocabulary of one approach brings back two passages from that cluster. The third approach sits in the corpus, but its terms are different enough that it never reaches the re-ranker. The response model cannot weigh the disagreement it was never shown. A later summary may describe the two retrieved passages as “what the notes say.” The omission has become a seemingly coherent account.

GraphRAG can preserve relations that a flat vector index misses, but the graph does not solve this by existing. If ingestion connects only similar nodes and retrieval enters through the nearest match, the system can walk a graph and still remain in the same conceptual district.

We are building architectures that fear surprise.

That is deliberately stronger than the engineering claim. The narrower claim is that many retrieval stacks measure relevance and latency while leaving exploratory value implicit. A system cannot select for a kind of difference it has no way to represent or evaluate.

The feedback loop matters more to me than any single bad result:

If the next query borrows language from the last answer, the nearest results can become nearer still. If a memory system saves that answer as a compact account of earlier material, later retrieval may select the compact account instead of the sources, and any lost distinctions are harder to recover. This is a plausible failure mode, not a theorem about every RAG system. Users change course; corpora change; ranking rules differ. The point is that similarity has a compounding path while divergence often has to be introduced deliberately.

There are at least two versions of the loop. One runs through the person: I see an answer, adopt its terms, and use them in my next prompt. The other runs through the system: it stores a summary or inferred memory and later returns that compact version in place of the messier source. They can reinforce one another. Even if nothing is written back to a database, the conversation itself can narrow the next query. And when a memory system does write back, the source may need an explicit path back into view.

I have felt a version of this while using ChatGPT’s memory feature: accumulated “memories” can make responses gravitate toward the same centers or mix up concerns that mattered separately. I wrote about those observations in my field notes. I cannot infer the product’s internal retrieval design from that experience, but it makes the design question concrete. What does a memory system keep distinct? Which connections does it invite? When can it admit that the familiar answer is getting in the way?

Retrieval is cognition’s border control: it quietly decides which parts of a corpus are worth seeing. When that filter keeps favoring proximity, we may get a polished version of the thought we already brought in. Useful, sometimes. Maddening when we came looking for an opening.

It reminds me of talking to someone who keeps saying, “What I hear you saying is…” and then hands back a cleaner version of my own thought. On a bad day, that is exactly the half-assed AI answer I click “accept” on because I need to move on. The irritation is personal, but the architectural question is wider: which possible disagreements disappeared before the model had a chance to respond?

When I think about thinking, I picture a movement between following associations and testing what they amount to. While reading a paper, I may skim its figures, follow an unfamiliar citation, compare a claim with an older one, then return to the argument and decide what survives. The detour is part of the work. So is the return.

AI memory systems are often better at the return: storing, retrieving, compressing, answering. Their exploratory mechanisms are usually left to prompts or chance. I want to be able to ask a retrieval system for useful differences: a weak connection, a claim that disputes this one, an idea from a neighboring field, or a gap in the material it holds. Those are proposals for operations, not a claim that every search should wander.

I am wary of romanticizing the wandering. A factual question deserves a factual answer, and an irrelevant detour wastes attention. But a system that treats every detour as error leaves the user to perform the exploratory work alone. I keep thinking of the movement when reading: sample a few signals, pursue a strange association, test it against the material, then compress only after something has changed. Current memory products promise the compression. The pursuit is mostly ours.

The distinction matters. A contradiction should be tied to a claim and its evidence, not inserted for theatrical balance. A bridge should have some traceable relationship to the question, not merely come from a distant cluster. “Surprise me” is too cheap a specification. The system needs a way to preserve relevance while making room for what relevance ranking tends to bury.

I think of these as divergence primitives: associative drift along weaker ties, cross-domain bridges, contradiction surfacing, temporal novelty, and negative-space mapping. Some can be approximated with today’s tools. Others depend on how a memory system represents claims, time, provenance, and relationships. Part 2 takes up those implementation choices.

At the user level, I sometimes sample more of the possibility space by rerunning a prompt, comparing different models, or setting up a simulation with perspectives that would not normally share a room. The “Archaeologies of Latent Futures” approach, inspired by Karan Malhotra’s WorldSim prompt hack, treats a model as a space for provisional scenarios: a future encyclopedia entry, a group with conflicting expertise, a timeline that went another way. The output is material to examine, not evidence that the imagined world exists.

These tactics can unsettle a stale answer. They cannot change what a retrieval layer repeatedly chooses to show the model. That choice belongs in the infrastructure. We can test whether a system surfaces a defensible contradiction, a meaningful bridge, or a source the first query would have missed. We can also test whether any of those differences change the next question the person asks. That is a more interesting measure than producing a slightly faster version of the answer they expected.

There is an incentive problem here, too. Precision has familiar measures: latency, recall, whether the answer looks satisfactory. A useful difference may slow the interaction down before it improves anything. If we only count how quickly the system returns a plausible answer, exploratory retrieval will look like a mistake. I do not yet know the right measure for discovery. I do know that a system can be excellent at retrieval under its existing metrics while becoming increasingly poor at showing us what we did not know to ask.