Similar is not true: how our agents use a knowledge graph

A while back, one of our own agents drafted marketing copy describing a service list we had already retired. The old list sat in a reference document, the document sat in a vector store, and the store did exactly what vector stores do: it returned the chunk most similar to the question. The stale version of a fact is maximally similar to the question that should surface the new one. Nothing was technically broken, and that is the problem.

This failure has a shape, and once you see it you find it everywhere agents run on plain retrieval-augmented generation. Embeddings measure similarity. They do not know what is currently true, who superseded what, or that a number from March stopped being the number in June. If your agent answers questions about a living business from a pile of chunks, it will eventually state something that was true with total confidence.

The fix that is working for us is what people have started calling graph engineering: treating agent memory as an engineered graph of facts rather than a bag of similar text. This post describes what our own fleet runs today, with the real numbers, and the parts we have not built yet.

What a memory graph actually is

Strip the buzzword and it is four commitments:

Entities and typed relations, not chunks. "Lyra", "the Strix box", "the vision endpoint" are nodes. "Replaced", "runs on", "depends on" are edges. Our agents' shared graph currently tracks 193 entities and 270 typed relations, and every one of the 1,310 memory entries our fleet has filed links back into it.

Time on every fact. A fact carries when it was written and, ideally, when it stopped being true. The research frontier here is the temporal knowledge graph: Zep (arXiv:2501.13956) is the cleanest published architecture, with explicit validity intervals on every edge.

Invalidation, not overwrites. When a fact changes, the old edge is marked invalid and a new one is written. The history stays queryable. Overwriting in place is how you lose the ability to answer "what changed and when", which is the question that matters most in a post-mortem.

Recency-weighted recall. Not all memory ages at the same speed. The scoring idea comes from the generative-agents work at Stanford (Park et al., arXiv:2304.03442): retrieval combines relevance with recency and importance instead of relevance alone.

What we run, with numbers

Our CRM copilot Lyra is the most instrumented example. Three layers, in the order we reach for them:

Live tools first. For volatile facts, revenue, the current plan, this month's pipeline, the agent does not retrieve at all. It calls a tool that reads the system of record at answer time. A nightly job regenerates the figures; the tool reads whatever is current. Retrieval of a cached copy of a number is how agents quote last quarter with conviction.

A synced, rolled-up knowledge base second. Every night the KB indexes the project docs and the last 48 hours of git commits as dated changelog entries. Dailies older than 30 days get rolled up into monthly summaries by a local model and the dailies are deleted, so the store grows by understanding rather than by volume.

Recency-weighted hybrid retrieval third. Full-text and vector search are combined, then decayed by content type with different half-lives: changelog entries fade with a 30-day half-life, conversation memory at 90 days, reference documentation at 365. A floor stops old-but-foundational documents from vanishing entirely. The weights are boring and that is the point; the failure they prevent, an old fact outranking its replacement, is expensive.

Alongside Lyra, the agents on our own box share the graph described above, and they follow a written protocol: before answering about any person, project or number, query the graph first. If unsure, say so and check. Wrong is worse than slow.

The five rules we now hold every agent to

1. Volatile facts come from live tools, never from retrieval. If a number can change weekly, reading a stored copy of it is a bug.

2. Every stored fact gets a date and links. An undated memory is a rumour. An unlinked one is a dead end for the next agent that finds it.

3. Change means invalidate then add. The old fact is marked ended, the new one written beside it. Nothing is silently overwritten.

4. Retrieval is recency-weighted by content type. Changelogs age fast, reference material slowly, and the ranking must know the difference.

5. Check before speaking. An agent that states a fact about the business without consulting the graph is guessing, however fluent the guess.

What we have not built yet

Full bitemporal validity, a facts table carrying both when a fact became true and when we learned it stopped, in the style Zep formalised, is designed but not yet in production here. We are running the simpler layers for a few weeks first and letting the regression harness tell us what the extra machinery is worth. When it lands we will publish what it changed, including if the answer is "little".

If you are evaluating this space, the current crop worth reading beyond Zep is Mem0 and Letta. All of it runs happily against local models: everything described in this post executes on one desk-side box, and no memory leaves it.