Commit graph

18 commits

Author SHA1 Message Date
Snider
385b89b3eb fix(agent/brain): cap remember()/recall()/forget() input field sizes
Bound input field sizes against memory/DB/Qdrant bloat (DoS-by-self):
- content: 65536 bytes via mb_strlen
- tags: max 100 entries; each tag max 128 chars
- agent_id, type: 64 chars each
- project, org: 128 chars each
- supersedes_id: ULID-shape only

validateRememberInput() throws InvalidArgumentException at every entry
point (remember, recall, forget) before any DB or upstream call. Field-
specific error messages so callers know which field violated.

Pest covers good-path, content-too-long, tags-array-too-large, tag-
length, exact-boundary cases.

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=1001
2026-04-25 18:58:41 +01:00
Snider
dea64f4099 fix(agent/brain): walk supersede chain to current head + cycle guard
remember() now resolves a stale supersedes_id to the current live head
before writing — when X has been superseded by Y, a retried call with
supersedes_id=X automatically links the new memory to Y instead of
silently dropping the supersede.

- Walk the chain from supplied supersedes_id to find the active head
- Cap the walk at depth 100 (cycle/runaway protection)
- Throw RuntimeException("Detected cycle while resolving supersede chain")
  on detected cycle, BEFORE any DB write
- Throw InvalidArgumentException("Superseded memory not found") when
  the original supersedes_id never existed
- deleteSupersededMemory no longer silently no-ops once the resolved
  head is expected to exist

Pest coverage extended:
- Direct chain link (X exists, succeeds with X→linked)
- Retry path (X→Y, then retry on X produces Z→Y, walks chain)
- Never-existed target (graceful error)
- Synthetic X↔Y cycle (caps walk + throws, no writes leak)

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=316
2026-04-25 18:42:52 +01:00
Snider
167ce9783e fix(agent/brain): authorise org against MCP context at every entry point
remember(), recall(), forget(), and elasticSearch() now resolve the
allowed-orgs set from the authenticated request context (mcp_workspace_context),
preferring explicit authorised_orgs/authorized_orgs, falling back to the
authenticated workspace's org/slug. A mismatched org throws
AuthorizationException BEFORE any Qdrant/Elasticsearch call or destructive
DB action — closes the horizontal-priv-escalation vector where an MCP
client could recall/remember/forget memories scoped to ANY org by
setting org="other-org" in the request body.

Pest coverage in OrgScopingTest covers good path, unauthorised recall
(asserts no HTTP), cross-org forget (asserts no DB delete), unauthorised
remember (asserts no embed/index jobs).

Note: BrainList free-form org filter is a separate ticket — outside this
lane's allowlist.

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=312
2026-04-25 18:32:19 +01:00
Snider
a1a0981b06 fix(agent/brain): retryableHttp narrows retryable set + 6-attempt budget
retryableHttp() now retries only 408 (Request Timeout), 429 (Too Many
Requests), and 503 (Service Unavailable). 500-and-other-5xx fail
immediately so the circuit-breaker registers them as a single
failure rather than smearing across retry attempts. Retry-After
honoured (numeric + HTTP-date), capped reasonably.

Attempt budget bumped to 6 so a burst of 5 transient 503s can recover
within ONE circuit-permitted call — the original concern from #311.

Note: CircuitBreaker is already applied OUTSIDE the logical Brain
operation by the MCP tool layer, not around each HTTP retry. The
nesting report was stale at this code shape; the real drift was the
retryableHttp() retry set + budget.

Pest coverage in CircuitBreakerTest:
- Recovered 503 burst → circuit stays closed, no failure registered
- Exhausted 503 burst → ONE breaker failure (not five)
- 429 + Retry-After 1 → sleeps 1s, no breaker failure
- 500 → immediate breaker failure, no retry

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=311
2026-04-25 18:14:40 +01:00
Snider
b6565263f3 fix(agent/brain): lock forget+supersede paths against late index writes
Cache::lock keyed by memory id wraps the delete path in BrainService::
forget(); supersede cleanup in remember() lifted to the same idiom.
forget() now ALWAYS queues DeleteFromIndex on a successful delete
(was previously skipped when indexed_at was null — left late writes
from stale preloaded models a window to land entries after the
underlying memory was gone).

Index write paths (qdrantUpsert / elasticIndex) re-check that the
memory row still exists before writing — defence-in-depth against any
future caller that holds a stale model reference past a forget.

Pest coverage extended in SupersedeForgetIndexCleanupTest:
- never-indexed forget queues cleanup
- late stale-model index writes are skipped after forget
- never-indexed supersede cleanup queues deletion
- late stale-model index writes are skipped after supersede

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=999
2026-04-25 18:04:55 +01:00
Snider
6832d40587 fix(agent/brain): batch — org maxLength + retry semantics + forget index cleanup
Codex 5.5 batch lane processed 8 brain Mantis tickets. 4 implemented,
1 stale, 3 deferred.

Tickets implemented:
- #313 — MCP schemas (BrainRemember/Recall/List): org field maxLength=128 with runtime validation; recall filter.org also bounded; pest test coverage added
- #314 — BrainList: removed withCircuitBreaker('brain') from DB-only handler; CircuitBreakerTest updated to assert no breaker call
- #315 — BrainService.retryableHttp(): now retries 408 (request-timeout), 429 (rate-limit), and 5xx; honours Retry-After header; focused retry tests added
- #326 — BrainService.forget(): dispatches DeleteFromIndex only when row has indexed_at (was unconditional); SupersedeForgetIndexCleanupTest covers never-indexed case

Tickets stale-fixed: #316 (RememberKnowledge already rejects missing/deleted supersedes target before dangling retry)
Tickets deferred: #121 (cross-surface audit), #311 (retry-inside-breaker architectural redesign), #312 (no authoritative org claim in MCP request context yet)

Co-authored-by: Codex <noreply@openai.com>
Closes tasks.lthn.sh/view.php?id=313
Closes tasks.lthn.sh/view.php?id=314
Closes tasks.lthn.sh/view.php?id=315
Closes tasks.lthn.sh/view.php?id=326
2026-04-25 14:55:40 +01:00
Snider
c616ff1e32 fix(brain): close openbrain audit gaps — org scoping + index cleanup + reindex flags + MCP schemas + circuit layer
Closes the 5 PARTIAL items flagged in docs/AUDIT-openbrain-20260424.md.

- Gap A (org scoping persisted on writes): new migration adds `org`
  nullable+indexed column to brain_memories; BrainMemory fillable;
  RememberKnowledge action forwards org; BrainService::remember
  persists it.

- Gap B (supersede/forget Elastic cleanup): BrainService::forget
  dispatches DeleteFromIndex (handles both Qdrant + Elastic); supersede
  path dispatches cleanup for the old memory id before replacing it.
  DeleteFromIndex itself untouched — already handled both indexes.

- Gap C (brain:reindex flags): --org, --project, --stale (null OR
  >14d old), --dry-run (count+stop), --elastic-only added to the
  artisan command.

- Gap D (MCP schemas expose org): brain_remember, brain_recall,
  brain_list now accept `org` in input schema + forward into
  action/service.

- Gap E (resilience uneven): brain_list now wrapped in
  withCircuitBreaker('brain', ...) matching the pattern used by
  BrainRemember/Recall/Forget. BrainService gains retryableHttp()
  helper — 100/300/900ms exponential backoff, retries only on 5xx +
  connection errors, not on 4xx. Qdrant calls route through it;
  Ollama left alone (EmbedMemory job has its own retry).

Tests (Good/Bad/Ugly per gap):
- Feature/Brain/OrgScopingTest.php
- Feature/Brain/SupersedeForgetIndexCleanupTest.php
- Feature/Brain/ReindexFlagsTest.php
- Feature/Mcp/BrainSchemaOrgTest.php
- Feature/Brain/CircuitBreakerTest.php

php -l clean on all 13 files. Pest binary not in this checkout —
CI path validates the full suite.

Closes tasks.lthn.sh/view.php?id=107

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Virgil <virgil@lethean.io>
2026-04-24 08:14:06 +01:00
Snider
4c1fa56d17 fix(brain): wire Qdrant api-key header from BRAIN_QDRANT_API_KEY
BrainService::http() was building a PendingRequest with no auth
header, so when Qdrant has auth enabled (the production lthn.sh
deploy does) every upsert/lookup returned 401. The circuit breaker
logged the 401 via Cache::store('file'), which was the red-herring
cache-write error chased in the first #97 iteration.

Changes:
- BrainService loads + trims a Qdrant api key from
  config('brain.qdrant.api_key') in the constructor.
- New qdrantHttp() helper returns a PendingRequest with the
  api-key header when the key is non-empty, or the plain client
  otherwise. Ollama + Elasticsearch call sites still use http()
  (separate auth shapes).
- php/config.php adds a brain.qdrant.api_key entry reading
  env('BRAIN_QDRANT_API_KEY').
- Good/Bad/Ugly Pest tests cover: configured key → header sent,
  unset → header absent, empty-string → header absent.

Closes tasks.lthn.sh/view.php?id=97

Co-authored-by: Codex <noreply@openai.com>
Co-Authored-By: Virgil <virgil@lethean.io>
2026-04-24 05:21:15 +01:00
Snider
a13f4c4bbd feat(brain): add GET /v1/brain/tags + /v1/brain/scopes
Two introspection endpoints for OpenBrain:
- GET /v1/brain/tags — ES terms aggregation over tags.keyword, returns
  {tag: count} pairs for UI filter chips
- GET /v1/brain/scopes — composite aggregation over {org, project},
  returns the scope hierarchy present in the index

Sits under the existing brain.read-auth group in Routes/api.php. New
BrainService helpers for aggregation shape; reuses the elasticSearch
HTTP path added in #59.

Pest coverage with Http::fake for ES.

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=65
2026-04-23 13:52:50 +01:00
Snider
6da45637f5 feat(brain): extend /v1/brain/recall with org + keywords + boost_keywords
Hybrid recall: vector (Qdrant) + keyword (ES) with dedupe-by-id and
score re-ranking via configurable boost multiplier.

- org: string → adds to Qdrant must-filter + ES filter
- keywords: array<string> → when present, ES full-text hits merge into
  results; without keywords, path stays purely vector
- boost_keywords: array<string> → each matched boost-keyword amplifies
  the memory's score by mcp.brain.boost_keywords_multiplier (default 1.5)

BrainService gains a hybridRecall() helper; BrainController::recall()
delegates to it. Existing request fields (query, limit, workspace_id,
project, type) unchanged.

php/tests/Feature/Api/BrainRecallExtendedTest.php — Pest coverage with
Http::fake for both Qdrant + ES, asserting dedupe + boost behaviour.

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=63
2026-04-23 13:51:37 +01:00
Snider
677d890308 feat(brain): make BrainService::remember() async via EmbedMemory job
remember() now writes the brain_memories row with indexed_at=null and
dispatches EmbedMemory::dispatch($memory->id) for async Qdrant + ES
indexing, instead of calling qdrantUpsert() synchronously. Response shape
matches the row state — caller gets the memory immediately, the Job
flips indexed_at once the Qdrant write succeeds.

Superseded rows still soft-delete synchronously (part of the remember
contract, not the indexing path).

php/tests/Feature/Services/BrainServiceRememberTest.php uses Queue::fake()
to assert EmbedMemory is dispatched and BrainService::qdrantUpsert() is
NOT called directly (subclass probe).

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=55
2026-04-23 13:29:30 +01:00
Snider
f223a77daa feat(brain): add Elasticsearch indexing to BrainService
Fills in the elasticIndex/elasticDelete stubs added by #56 and #57, plus a
new elasticSearch() method used by the upcoming /v1/brain/search endpoint
(#64).

- elasticIndex(BrainMemory) → PUT /brain_memories/_doc/{id}
- elasticDelete(string $id) → DELETE /brain_memories/_doc/{id}
- elasticSearch(string $query, array $filters) → POST /brain_memories/_search
- ES URL default http://127.0.0.1:9200 (config override via
  BRAIN_ELASTICSEARCH_URL env var)
- RuntimeException on HTTP failures (same pattern as qdrantUpsert)

php/tests/Feature/Services/BrainServiceElasticTest.php covers Good/Bad/Ugly
for index, delete, and search using Http::fake.

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=59
2026-04-23 13:11:29 +01:00
Snider
8d520adb5e feat(brain): add DeleteFromIndex job
Inverse of the EmbedMemory job (#56): removes a memory from Qdrant (and
the future Elasticsearch index) when brain_forget fires or a memory is
soft-deleted.

- php/Jobs/DeleteFromIndex.php — Laravel Job, 3 retries with backoff
- BrainService: qdrantDelete() private→public and now throws on HTTP
  failure (was silent Log::warning — wouldn't trigger Job retry)
- elasticDelete() stub added (fills in with the ES integration ticket)
- php/tests/Feature/Jobs/DeleteFromIndexTest.php — success + HTTP-failure
  paths via mocked Http

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=57
2026-04-23 12:55:45 +01:00
Snider
4dc5ed8d14 feat(brain): add EmbedMemory job + indexed_at tracking
Implements the async-embedding pipeline's worker side:

- php/Jobs/EmbedMemory.php — Laravel Job that calls BrainService::embed()
  + qdrantUpsert() and sets indexed_at on success
- php/Migrations/…_add_indexed_at_to_brain_memories.php — nullable
  timestamp + index, portable across pgsql/mariadb (hasColumn guard)
- BrainMemory: +indexed_at fillable + datetime cast + PHPDoc
- BrainService: qdrantUpsert() private→public so the Job can use it;
  elasticIndex() stub added (to be implemented by the ES ticket)
- php/tests/Feature/Jobs/EmbedMemoryTest.php — Pest tests for success
  path and Qdrant-failure path

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=56
2026-04-23 12:47:10 +01:00
Snider
bd060c8aa0 feat(brain): add org filter to BrainService::buildQdrantFilter()
Adds an `org` match filter between workspace_id and project in the Qdrant
payload filter chain. Multi-org isolation for OpenBrain memory retrieval.

Co-authored-by: Codex <noreply@openai.com>

Closes tasks.lthn.sh/view.php?id=58
2026-04-23 12:32:58 +01:00
e58986a3b4 revert fcb9c189e5
revert fix(agentic): harden TODO workspace write

Co-Authored-By: Virgil <virgil@lethean.io>
2026-04-23 12:32:57 +01:00
Codex
cbc262add4 fix(agentic): harden TODO workspace write
Co-Authored-By: Virgil <virgil@lethean.io>
2026-04-23 12:32:57 +01:00
Snider
be1130f470 agent updates 2026-03-21 11:10:44 +00:00
Renamed from src/php/Services/BrainService.php (Browse further)