core/agent
8
0
Fork 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions

docs(openbrain): alignment audit 2026-04-24 — PARTIAL verdict

Browse source 
Section-by-section verdict with file:line evidence against
RFC-OPENBRAIN.md. Headline gaps:

- org scoping not persisted on writes (migration, model fillable,
  remember action all only handle project)
- supersede path leaves Qdrant/Elastic stale
- forget path leaves Elastic stale
- brain:reindex missing --org/--project/--stale/--dry-run/
  --elastic-only flags
- MCP tool schemas don't expose org for write/list
- Resilience uneven: brain_list skips circuit breaker;
  BrainService HTTP layer has no retry/circuit

Verdict: PARTIAL. Queueing pipeline, Qdrant/Ollama/Elastic
plumbing, and MCP tool surface are in place; scoping +
index-consistency-on-mutation are the follow-up work.

Closes tasks.lthn.sh/view.php?id=54

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Virgil <virgil@lethean.io>
This commit is contained in:
Snider 2026-04-24 05:36:59 +01:00
parent 4c1fa56d17
commit 8f3b39983a
1 changed files with 27 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
docs/AUDIT-openbrain-20260424.md Normal file
View file

@ -0,0 +1,27 @@
<!-- SPDX-License-Identifier: EUPL-1.2 -->
# OpenBrain Alignment Audit — 2026-04-24
## Summary
`docs/RFC-AGENT-PIPELINE.md:193-203` only requires OpenBrain to exist as a queryable knowledge base for non-actionable findings; `docs/php-agent/RFC.openbrain-design.md:1-12` redirects all implementation detail to `../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md`. Against that superseding RFC, the PHP implementation is materially in place: MariaDB/Qdrant/Ollama/Elasticsearch plumbing exists, `EmbedMemory` is queued, `brain:reindex` exists, and MCP `remember`/`recall`/`forget`/`list` tools are present (`php/Services/BrainService.php:106-121`, `php/Jobs/EmbedMemory.php:17-60`, `php/Console/Commands/BrainReindexCommand.php:13-53`, `php/Mcp/Tools/Agent/Brain/BrainRemember.php:18-102`, `php/Mcp/Tools/Agent/Brain/BrainRecall.php:19-119`, `php/Mcp/Tools/Agent/Brain/BrainForget.php:18-78`, `php/Mcp/Tools/Agent/Brain/BrainList.php:18-81`). The remaining drift is concentrated in write-side `org` scoping, index consistency on supersede/forget, incomplete reindex options, and uneven resilience.
## Section-by-section
- §1 Architecture (Postgres + Qdrant + Ollama + Elasticsearch): PARTIAL — `BrainService::remember()` writes MariaDB first and queues indexing (`php/Services/BrainService.php:106-121`); `recall()` embeds the query, searches Qdrant, then hydrates `BrainMemory` rows from MariaDB (`php/Services/BrainService.php:130-210`); `EmbedMemory` upserts Qdrant and indexes Elasticsearch (`php/Jobs/EmbedMemory.php:32-60`); Elasticsearch search/aggregation helpers exist (`php/Services/BrainService.php:263-323`, `php/Services/BrainService.php:421-570`). Drift: `forget()` deletes from MariaDB + Qdrant only, not Elasticsearch (`php/Services/BrainService.php:213-222`), and the Elastic document omits `agent_id`, `source`, and `created_at` from the RFC schema (`../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:261-280`, `php/Services/BrainService.php:488-500`).
- §2 Scoping (workspace/org/project filters): PARTIAL — workspace scoping is enforced in service/model code (`php/Services/BrainService.php:140-141`, `php/Models/BrainMemory.php:114-137`), and service-side Qdrant/Elastic filters support `org` and `project` (`php/Services/BrainService.php:448-480`, `php/Services/BrainService.php:530-554`). Drift: the write path does not accept or persist `org` (`../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:61-108`, `php/Actions/Brain/RememberKnowledge.php:82-91`, `php/Models/BrainMemory.php:68-80`, `php/Migrations/0001_01_01_000008_create_brain_memories_table.php:28-46`), and MCP recall/list schemas expose `project` but not `org` (`php/Mcp/Tools/Agent/Brain/BrainRecall.php:59-87`, `php/Mcp/Tools/Agent/Brain/BrainList.php:41-67`).
- §3 Async embedding (EmbedMemory job + queue worker): PARTIAL — the core async path matches the RFC: new memories start with `indexed_at = null`, then `EmbedMemory` is dispatched (`php/Services/BrainService.php:106-121`), and the job is queueable with retries/backoff and marks `indexed_at` after Qdrant + Elasticsearch indexing (`php/Jobs/EmbedMemory.php:17-60`). Drift: the supersedes path deletes the old MariaDB row but does not dispatch `DeleteFromIndex`, even though the RFC requires index cleanup for superseded memories (`../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:121-137`, `php/Services/BrainService.php:110-119`, `php/Jobs/DeleteFromIndex.php:16-35`).
- §4 Re-index artisan command: PARTIAL — `brain:reindex` exists and dispatches `EmbedMemory` jobs in chunks (`php/Console/Commands/BrainReindexCommand.php:13-53`). Drift: the command only supports `--all` and `--chunk`, and only distinguishes `all` vs `indexed_at IS NULL`; RFC options for `--org`, `--project`, `--stale`, `--dry-run`, and `--elastic-only` are not present (`../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:199-246`, `../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:651-669`, `php/Console/Commands/BrainReindexCommand.php:15`, `php/Console/Commands/BrainReindexCommand.php:27-32`).
- §5 MCP tools (remember/recall/forget/list): PARTIAL — all four MCP tools exist, are workspace-gated, and delegate to the expected actions (`php/Mcp/Tools/Agent/Brain/BrainRemember.php:24-102`, `php/Mcp/Tools/Agent/Brain/BrainRecall.php:25-119`, `php/Mcp/Tools/Agent/Brain/BrainForget.php:24-78`, `php/Mcp/Tools/Agent/Brain/BrainList.php:24-80`). Drift: `brain_remember` has no `org` input (`php/Mcp/Tools/Agent/Brain/BrainRemember.php:41-83`), `brain_recall` exposes neither `org` nor keyword-boost parameters even though the service can accept them (`php/Mcp/Tools/Agent/Brain/BrainRecall.php:42-91`, `php/Services/BrainService.php:130-137`), and `brain_list` has no `org` filter (`php/Mcp/Tools/Agent/Brain/BrainList.php:41-67`).
- §6 Circuit breaker / resilience: PARTIAL — MCP tool-level circuit breaker support exists in `AgentTool::withCircuitBreaker()` (`php/Mcp/Tools/Agent/AgentTool.php:310-330`), and `brain_remember`, `brain_recall`, and `brain_forget` use it (`php/Mcp/Tools/Agent/Brain/BrainRemember.php:95-101`, `php/Mcp/Tools/Agent/Brain/BrainRecall.php:109-117`, `php/Mcp/Tools/Agent/Brain/BrainForget.php:72-76`). Queue jobs also retry with backoff (`php/Jobs/EmbedMemory.php:21-26`, `php/Jobs/DeleteFromIndex.php:20-25`). Drift: `brain_list` is not circuit-broken (`php/Mcp/Tools/Agent/Brain/BrainList.php:70-79`), and `BrainService` HTTP calls are timeout-only and fail fast without retry/circuit logic (`php/Services/BrainService.php:45-49`, `php/Services/BrainService.php:77-85`, `php/Services/BrainService.php:151-153`, `php/Services/BrainService.php:271-274`, `php/Services/BrainService.php:315-318`, `php/Services/BrainService.php:586-589`, `php/Services/BrainService.php:606-609`).
- §7 Qdrant auth (api-key): IMPLEMENTED — the service reads a configured Qdrant API key, attaches it as an `api-key` header, and routes all Qdrant reads/writes through that helper (`php/Services/BrainService.php:23-39`, `php/Services/BrainService.php:55-65`, `php/Services/BrainService.php:143-149`, `php/Services/BrainService.php:229-235`, `php/Services/BrainService.php:581-584`, `php/Services/BrainService.php:601-604`).
## Remaining gaps
- `org` scoping is not persisted on writes: the table schema has no `org` column, the model is not fillable for `org`, and the remember action only forwards `project` (`php/Migrations/0001_01_01_000008_create_brain_memories_table.php:28-46`, `php/Models/BrainMemory.php:68-80`, `php/Actions/Brain/RememberKnowledge.php:82-91`).
- Superseding a memory removes the old row in MariaDB without removing its Qdrant/Elasticsearch entries (`php/Services/BrainService.php:110-119`, `php/Jobs/DeleteFromIndex.php:16-35`).
- Forget removes MariaDB + Qdrant data but leaves Elasticsearch stale (`php/Services/BrainService.php:213-222`).
- Elastic documents do not include the full RFC metadata set and use a fixed `brain_memories` index name (`../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:261-280`, `../images/developer/spec/project/lthn/ai/RFC-OPENBRAIN.md:675-687`, `php/Services/BrainService.php:21`, `php/Services/BrainService.php:488-500`).
- `brain:reindex` is missing RFC scoping and mode flags (`php/Console/Commands/BrainReindexCommand.php:15`, `php/Console/Commands/BrainReindexCommand.php:27-32`).
- MCP tool schemas still expose `project`-only scoping for write/list and do not expose `org` across the tool surface (`php/Mcp/Tools/Agent/Brain/BrainRemember.php:41-83`, `php/Mcp/Tools/Agent/Brain/BrainRecall.php:42-91`, `php/Mcp/Tools/Agent/Brain/BrainList.php:41-67`).
- Resilience is uneven: three brain tools use `withCircuitBreaker`, `brain_list` does not, and `BrainService` itself has no retry/circuit layer (`php/Mcp/Tools/Agent/Brain/BrainRemember.php:95-101`, `php/Mcp/Tools/Agent/Brain/BrainRecall.php:109-117`, `php/Mcp/Tools/Agent/Brain/BrainForget.php:72-76`, `php/Mcp/Tools/Agent/Brain/BrainList.php:70-79`, `php/Services/BrainService.php:45-49`).
## Verdict
PARTIAL