Freshness and drift: how the index keeps itself from lying

Immutable revisions, the append-only drift ledger, the checksum guard, stale marks, and "what changed in the knowledge" archives — RAGRAF's central guarantee.

The problem the mechanics solve #

An index is a cache. A cache that goes stale silently is worse than no cache: the agent makes decisions on facts about a code state that no longer exists. The classic failure scenario:

The index was built on Monday.
On Tuesday a developer renamed a function and dropped a table.
On Wednesday the agent asks "who calls the old function" — receives an honest, proven, stale answer and builds an edit on it.

RAGRAF does not promise "the index is always fresh" — that is expensive and unattainable during active work. It promises something else: staleness is always visible. An answer either confirms freshness, or lists what diverged, or refuses. Silently serving the past as the present does not exist in the architecture (anti-requirement #1, principle P2).

Four mechanisms #

1. Immutable revision #

The index is built as a snapshot: corpus_hash = sha1(sorted list of (path, sha256)) pins the exact corpus state. Within a revision, facts are neither edited nor appended — only a new revision. Consequence: every answer is reproducible — index_rev in the envelope uniquely determines the fact set.

2. Append-only drift ledger #

While the agent works, code gets edited. Every edit is registered by the ledger, not by the index:

Field	Purpose
`entry_id`, `rev`	binding to the revision the ledger lives against
`change`	`create` / `modify` / `delete` — what happened to the file
`path`, `detected_by`, `detected_at`	where, noticed by whom (guard / explicit call), when
`old_sha256`, `new_sha256`	exact content versions
`affected_nodes`, `affected_symbols`	which nodes and symbols are affected
`dedupe_key` (UNIQUE)	re-registering the same event creates no duplicates

The ledger is the only place where changes between revisions live. It is never deleted or overwritten: the divergence history is part of the product's audit trail.

3. The checksum guard #

A cheap check built into every request: corpus_hash is recomputed (or compared against the state cache) and matched against the index revision. Divergence → the ledger grows, and the answer envelope receives:

JSON

"drift": { "files": 3, "symbols": 7, "entries": 4 },
"stale_warnings": [
  { "path": "src/Pipeline/PipelineEngine.php",
    "reason": "modify after rev-0007",
    "suggest": "live_read" }
]

The action policy is configurable (index.reindex_on_drift): warn (mark and serve), auto (reindex transparently), fail (refuse — for strict pipelines). Facts touched by the ledger are marked stale in the results themselves — stale cannot be confused with missing: they are different answers.

4. Ledger archive and the "what changed in the knowledge" report #

On reindex the ledger does not vanish without a trace — it is archived into drift_archive together with the revision that closed it, and turned into a human-readable report: which files diverged, which symbols were affected, what the index "learned anew". This is rare among RAG systems: the ability to answer "what changed in the knowledge about the code between Monday and Wednesday" — not from git history, but from the history of facts. Deduplication and archiving instead of endless accumulation is a direct takeaway from AdaRepair-Mem ("more memory ≠ better").

How it looks in answers #

Three honest states instead of one "answer":

State	What the agent sees	Example
Fresh	`drift.files = 0`, no warnings	a normal answer
Diverged	`stale_warnings` with `suggest` + `stale` marks on affected facts	"function found, but its file was edited after the revision — re-read via live_read"
Impossible	refusal with a reason (code 3)	the ledger contradicts the state; the requested operation is incompatible with the mode

suggest is mandatory: a warning without an instruction is half the value. Options: live_read (read the file directly — always available), reindex (rebuild the revision).

Verified by protocols, not by promise #

The mechanics are protected by acceptance tests fixed in docs/architecture.md §8.5:

M5-T2 — the ledger loses no events under parallel edits of one file: dedupe_key and registration order verified by scenario.
M5-T3 — a stale fact cannot be served as fresh by any operation: end-to-end check across all graph-reading operations.
M5-T4 — the ledger archive restores the full divergence picture of a closed revision.
M6 — 100% of code facts carry verifiable path:line (the verification script references actually extracted artifacts).

Boundaries of the mechanics #

Honesty requires naming what the mechanics do not do:

No real-time tracking of intra-revision changes — the guard fires on a request or an explicit command; no file watcher ships (a future direction, see future).
No delta repair of the CPG layer — Joern has no incremental rebuild; after edits touching interprocedural links, slice is more honestly rebuilt wholesale.
corpus_hash hashes the file list, not the directory tree — renaming a directory without changing files yields the same content hashes but different paths; the guard compares paths too.
No protection against edits of .ragraf/ itself — index files live next to the code; their integrity is the file system's and the owner's responsibility (hence .ragraf/ goes into .gitignore, not into the repository).

Why this is the WOW #

Compare with plain RAG: chunks get reindexed "eventually", search returns fragments with no notion of age, and no answer ever says "I am based on yesterday's code". RAGRAF makes freshness part of every answer's contract — drift, stale_warnings, index_rev and capability are always present, even when everything is fine. The agent (and the human reading its report) always knows how much a fact can be trusted. That is the difference between "search over code" and a knowledge base.

Next: the 22 operations reference — what to ask and how to read answers →