Architecture: three SQLite files and zero infrastructure

The model in one paragraph #

RAGRAF snapshots a corpus into an immutable revision (rev-NNNN): property/AST graph, FTS5 lexicon, 384-dimensional vectors, DB schema from migrations, and optional interprocedural calls from Joern — all in three SQLite files inside <project>/.ragraf/. Reads go through a facade of 22 fixed operations (no query language); every answer is wrapped in an envelope (index_rev, capability, drift, provenance, limits, stale_warnings, freshness). While the agent edits code, the index does not change: deltas accumulate in an append-only drift ledger, affected facts get marked stale, and a checksum guard reindexes automatically (or via reindex), archiving the ledger into a "what changed in the knowledge" report.

Eight principles #

Each is derived from a measured defect, not from taste. They are not revisited without revisiting the architecture.

Three stores #

Separated by lifecycle: graph and lexicon are large and rebuildable; control is small and never deleted.

Why chunk_text is a separate table rather than an FTS column: an eighth column would silently shift all bm25() weights, and sqlite3 excludes UNINDEXED columns from the index — text would not be readable by column number. The answer must return a fragment in one call (search --with-text), and a second-step address must resolve without touching disk (code --entry-ids).

Identifiers = content #

Determinism is a requirement (NFR-5), so every identifier is a content hash, never time or traversal order:

node_id     = sha1(kind | fqn | path | start_line)      ← the string component is mandatory
edge_uid    = sha1(src | dst | kind | evidence_path | evidence_line)
chunk_id    = sha1(content_sha256 | chunk_kind | model) ← deliberately WITHOUT revision and path
corpus_hash = sha1(sorted list of (path, sha256))

Revision-independence of chunk_id is the main economy of rebuilds: unchanged chunks keep their embeddings across revisions. Two builds of the same revision must produce identical identifier sets (test M1-T4). No UUIDv4, no timestamps inside identifiers.

Provenance: stated vs derived #

The provenance column is mandatory and takes exactly two values:

• stated — the edge is proven by the analyzer: an exact declaration/call line, an information_schema row. confidence = 1.0.
• derived — inferred by heuristic or statistics (dynamic call, name coincidence, history inference). confidence from a calibratable table: 0.60 unambiguous heuristic · 0.40 ambiguous · 0.30 history/embedding inference.

calls_heuristic is a separate edge kind: heuristics are admitted into the graph (otherwise dynamic PHP yields no links at all) but never silently mixed with proven ones — impact --mode hard returns only stated. Source conflicts are resolved by priority (information_schema > migrations > CPG > SCIP > tree-sitter) and recorded in coverage as conflict.* — never swallowed.

The indexing pipeline #

0. Version and availability checks (Joern/JDK, PHP, tree-sitter) → capabilities
1. scan    — file selection, exclusions, sha256, languages      → corpus_hash
2. ast     — tree-sitter: symbol boundaries, tokens, calls, SQL literals
3. resolve — name resolution: exact → name+file → honest refusal
4. schema  — tables/columns/FKs from information_schema and migrations/*.sql
5. cpg     — Joern: interprocedural calls; failure → degraded (indexing does not die)
6. lexvec  — FTS5 + embeddings (reuse by chunk_sha256)
7. coverage, conflicts, duplicates → coverage; invariants → rev.state = 'ready'

The graph and lexicon loading flows are independent: the graph does not wait for vectors. An honest detail: in the shipped code the AST walk is sequential (the documented "parallel parse, 8 cores/16 threads" is a goal, not a fact; closing the gap is in future directions, R-PERF-1).

The answer envelope #

A single format for every answer — requirement FT-16: the agent always sees what it is trusting.

{
  "op": "impact",
  "index_rev": "rev-0007",
  "capability": "full",
  "drift": { "files": 3, "symbols": 7, "entries": 4 },
  "provenance": "index",
  "limits": ["static analysis; dynamics not covered"],
  "stale_warnings": [
    { "path": "src/Pipeline/PipelineEngine.php",
      "reason": "modify after rev-0007",
      "suggest": "live_read" }
  ],
  "result": [ "…" ]
}

provenance values: index (index only) · live (file reads only) · mixed (each record marked). stale_warnings is populated automatically for every ledger-affected path; suggest is mandatory. How to read the envelope — in the operations reference.

Capability and degradation #

No embedding model → lexical search works, the envelope reports it. No free port → the server names the cause. Silent degradation does not exist (src/ragraf/cpg.py:9-11).

Layers and processes #

Components are separate processes, not libraries inside the agent's process:

Performance contract (NFR-4) and reality #

Schema-tier operations are practically free: measured means schema 0.5 ms · symbol 0.7 ms · table_usage 0.9 ms · write_check 3.3 ms.

Security and secrets #

• Secrets are excluded, not masked: files matching secrets_exclude never reach text_chunk, vectors, or logs (FT-39/40, sample-based test M3-T2).
• db.dsn_from indexes the fact "this symbol references a config", without values.
• Telemetry stores sha1 of the argument + a 60-character digest instead of full query texts.
• Admin panel: loopback only, mutations POST+token (secrets.compare_digest), project-config writes validated with a .bak copy. Known gap: GET /api/token hands the token to any local client — documented honestly in the panel article and the project's SECURITY.md.
• shell=True is absent everywhere except the documented Joern .bat launch (manual quoting; hardening — R-SEC-4).

Where the numbers come from #

Every "measured" claim in the documentation points to an artifact: bench/results/*.json (M2 runs rev-0012 and rev-0025 published), the usage_call journal (per corpus), stage timings in index_status. Estimates (tokens ≈ chars/4, lines ≈ chars/45) are marked "estimate" wherever they appear — the coefficients themselves were measured on two corpora.

Next: freshness and drift — the mechanics that keep the index from lying →