Problem space: why an agent cannot be trusted with memory of code

Three symptoms everyone recognizes #

Anyone who has handed an LLM agent an edit in an existing project has seen this firsthand.

Symptom 1 — the agent cannot find the place #

It answers confidently about a function, but verification shows: the function lives in another file, and the described one is a namesake. In a 392-file repository, a name grep returns dozens of matches — and the right one is not first.

Measured on the pilot corpus: rg located files but not places — 34 candidates for SuperAdmin, 44 for z_e(, 64 for namespace BatteryService. Finding a file is easy; finding the place in the file is not.

Symptom 2 — the agent does not know the DB schema #

A column was named "like the variable in the neighboring module" — AuthorUserID. No such column exists. This surfaces at runtime, not at read time:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'AuthorUserID' in 'field list'

In the journals the column was called InitiatorID, and UpdatedBy existed in exactly one table out of five. The code was already written when the error surfaced. This very case produced the write_check operation — validating the write plan against the schema before execution.

Symptom 3 — the agent does not know it is stale #

The nastiest one. The index was built, the code was then edited — the agent receives facts about a state that no longer exists. Telling "no such function" apart from "the function appeared after indexing" is impossible from the answer alone. A decision is made on that answer, and it is wrong.

What measurements say, not opinions #

Navigation is the dominant failure cause #

The VulnGym study (cited in the Positive Technologies article on graphs in code audit): on tasks where the source-to-sink chain is longer than nine steps, the best agent found 22.58% of vulnerabilities, and in 56.5% of failures it either never found the needed file (26.8%) or found the file but not the line (29.7%).

More than half of failures are not about analysis. They are about navigation — exactly the part RAGRAF moves out of the model into an external index.

Free-form graph queries are error generators #

LLMxCPG (USENIX Security '25) measured how base models generate CPGQL queries against a code graph:

The fine-tuned model is perfect — because it was trained for exactly this. Nobody trained your arbitrary agent. Hence architecture decision P4: no query language at all, only fixed operations. Details in architecture.

Adjacent literature the design leans on #

• ALIBI — comment injections suppress detection in >90% of cases → free text is stored separately from structural facts (text_chunk vs node/edge).
• "Words Speak…" — wording shifts verdicts by 33.2% → answer-wording neutrality is a requirement (FT-23).
• Repoformer — the right not to search yields up to 70% speedup → an honest "not found" beats an approximate answer (principle P8).
• LCIP (arXiv:1512.07435) — a measured ceiling for impact analysis: precision 69% / recall 79% / F1 55% (Java, mutation simulation). Used only as an order-of-magnitude reference for metric M2 — never as a "report target".
• AdaRepair-Mem — "more memory ≠ better" → the drift ledger is archived and deduplicated, never accumulated forever.

Why existing tools do not close the gap #

RAGRAF's position: take the structural precision of a graph and the semantic reach of vectors, wrap them in a fixed, verifiable, freshness-aware contract of 22 operations, and store everything in SQLite files — zero services. The optional Joern layer adds interprocedural reachability, and its absence is reported, never hidden.

Five anti-requirements: the problem the index has no right to create #

An index is a cache. A cache that lies is worse than no cache:

1. Silent staleness — last week's facts served as current. → Immutable revision + append-only drift ledger + stale_warnings in every envelope + checksum-guarded auto-reindex. Mechanics in freshness and drift.
2. Hallucinated links inside the index — a plausible edge "because the database says so". → Every edge carries evidence path:line, provenance (stated/derived) and confidence; heuristic edges are a separate kind and a separate answer block.
3. Unverifiable answers — the agent cannot check what it received. → 100% of code facts carry path:line; metric M6 verifies references automatically.
4. Capability masquerade — "slice returned empty" vs "slice is impossible in this mode". → capability: full|degraded in every envelope; refusal with code 3 and a reason; silent degradation is structurally impossible.
5. Secrets leaking into the index — config and .env values inside vectors. → secrets_exclude is exclusion, not masking: files are never opened; telemetry stores argument hashes and a 60-character digest.

These five anti-requirements shaped the architecture more strongly than any feature. Each maps to a checkable mechanism — see architecture and the acceptance protocols (metrics M5/M6, test M3-T2).

What this produced in practice #

One real feature increment (the /agreements/ document-flow section: payment, contract and vacation journals), 46 operation calls, 0 runtime errors:

Neither a compiler nor a "page opens" test catches that class. What does: an exact schema in the index, static rights tracing, and checks under a real role. Full breakdowns in benchmarks and future directions.

Boundaries: what RAGRAF is not #

• Not a code rewriter. RAGRAF answers; the agent edits. No autonomous patching.
• Not a vulnerability scanner. Taint analysis is not a release goal (the CPG layer makes it possible later — see future).
• Not a service. No daemons, no network endpoints except the loopback panel. Three SQLite files in <project>/.ragraf/.
• Not an oracle. Every answer lists its limits ("static analysis; dynamics not covered"), and honest overall verifiability across all operations is 24.4% — because schema facts have no line numbers by nature. We do not draw fake references for a prettier report.

Next: architecture — three SQLite files and zero infrastructure →