A corpus changes on a few pages. Its compressed memory needs an update. My initial thought was to replay old training examples near the changed pages, preserving the facts most likely to depend on the revision.
That is a reasonable story about the document graph. It turned out to be a poor account of where forgetting occurred in the tested trained cache. At a fixed replay budget, locality-aware selection was indistinguishable from random selection. Even an oracle graph did not rescue the idea.
The local edit and the distributed representation
Cartridges compress a corpus into a trained KV cache using self-study examples. The original method amortizes offline training across later queries. My study asks what happens when the corpus changes repeatedly and the cache is refreshed rather than rebuilt.
The locality policy spends replay on the mention-graph neighborhood of changed pages. This assumes the vulnerable old knowledge is concentrated near the edit. But the update changes shared trained parameters. Interference can spread across the representation even when the semantic revision is local.
The distinction is easy to miss. A dependency graph can correctly describe which answers should change while failing to predict which unchanged answers the optimizer will damage. Semantic dependence and optimization interference are different relations.
A result that defeated the mechanism
On three held-out synthetic worlds with repeated revisions, locality replay differed from random by only +0.4 points on unchanged facts, with an interval spanning zero. Its dependent multi-hop comparison was also inconclusive. The oracle-selection control mattered because it weakened the explanation that the graph was merely too noisy.
Diagnostics instead pointed to coverage. Forgetting was distributed, and preservation tracked how many pages replay touched. A later test on fresh worlds used replay stratified to cover every unchanged page and found a benefit over random on unchanged facts. Locality again tied random.
These are results in versioned synthetic wikis using the tested model and update protocol. They do not establish that locality never matters in a real corpus or a different representation. They do establish that the original local-repair story was not supported in this setting.
Updating can be worse than remaining stale
At the tested small replay budget, every refresh was worse overall than not updating. Warm-started full replay was the only refresh that beat the stale cache, and all cartridges remained well below retrieval over current text.
That is an uncomfortable but necessary baseline. A system that changes an old answer to the correct new answer may simultaneously damage enough unaffected answers to lose overall. Reporting only revised facts would make the refresh look useful while concealing its maintenance cost.
The full accounting should include update training, replay generation, retained examples, inference, and the frequency of future queries. A compressed representation can be excellent for a stable corpus and expensive to maintain under churn. The deployment’s revision rate helps determine whether the amortization ever pays back.
What I would keep from the failed idea
I would keep the distinction between changed facts, dependent facts, and unrelated facts. It makes the failure interpretable. I would abandon the assumption that graph distance alone predicts training damage.
A follow-up could compare a semantic dependency graph with an empirical interference map measured from small updates. The map would need to save more than it costs to estimate, and a simple coverage policy remains a serious baseline. More diagnostic machinery is not automatically a better refresh system.
The broader memory question is how to change one part of a compressed representation while preserving the rest. This experiment suggests that identifying the intended change is only half the job. The representation also determines where the unintended changes can travel.