On macOS, /var is a symlink to /private/var. When New() stores the
unresolved root but validatePath() resolves child paths via EvalSymlinks,
the mismatch causes filepath.Rel to produce ".." prefixes — triggering
false SECURITY sandbox escape warnings on every file operation.
Fix: resolve symlinks on the root path in New() so both sides compare
like-for-like. Updates TestNew to compare against resolved paths.
Co-Authored-By: Virgil <virgil@lethean.io>
Wire the marketplace to actually install modules from Git repos, verify
manifest signatures, track installations in the store, and auto-load them
as Workers at startup. A module goes from marketplace entry to running
Worker with Install() + LoadModule().
- Add Store.GetAll() for group-scoped key listing
- Create marketplace.Installer with Install/Remove/Update/Installed
- Export manifest.MarshalYAML for test fixtures
- Wire installer into Service with auto-load on startup (step 8)
- Expose Service.Installer() accessor
- Full integration test: install → load → verify store write → unload → remove
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each module now runs in a real Deno Worker with per-module permission
sandboxing. The I/O bridge relays Worker postMessage calls through the
parent to CoreService gRPC, so modules can access store, files, and
processes without direct network/filesystem access.
- Worker bootstrap (worker-entry.ts): sets up RPC bridge, dynamically
imports module, calls init(core) with typed I/O object
- ModuleRegistry rewritten: creates Workers with Deno permission
constructor, handles LOADING → RUNNING → STOPPED lifecycle
- Structured ModulePermissions (read/write/net/run) replaces flat
string array in Go→Deno JSON-RPC
- I/O bridge: Worker postMessage → parent dispatchRPC → CoreClient
gRPC → response relayed back to Worker
- Test module proves end-to-end: Worker calls core.storeSet() →
Go verifies value in store
40 unit tests + 3 integration tests (Tier 1 boot + Tier 2 bidir + Tier 3 Worker).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Service.OnStartup now creates sandboxed I/O medium, opens SQLite store,
starts gRPC listener on Unix socket, loads .core/view.yml manifest, and
launches Deno sidecar with CORE_SOCKET env var. Full shutdown in reverse.
New files: listener.go (Unix socket gRPC server), runtime/main.ts (Deno
entry point), integration_test.go (full boot with real Deno).
34 tests pass (33 unit + 1 integration).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Service embeds ServiceRuntime[Options] for Core/Opts access.
NewServiceFactory returns factory for core.WithService registration.
Correct Startable/Stoppable signatures with context.Context.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generated Go code from proto. Server implements CoreService with
FileRead/FileWrite/FileList/FileDelete/StoreGet/StoreSet — every
request checked against the calling module's manifest permissions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Service wraps Sidecar for DI registration. OnStartup/OnShutdown hooks
for framework lifecycle integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Module/Index types, ParseIndex from JSON, Search (fuzzy across code/name/
category), ByCategory filter, Find by code. Foundation for git-based
plugin marketplace.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CheckPath (prefix-based), CheckNet (exact match), CheckRun (exact match).
Empty allowed list = deny all. Secure by default.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Process launch with context cancellation, socket directory auto-creation,
channel-based stop synchronization. Uses sleep as fake Deno in tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Options, Permissions with Deno --allow-* flag generation,
DefaultSocketPath with XDG_RUNTIME_DIR support, Sidecar struct.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SQLite-backed KV store with get/set/delete/count/deleteGroup/render.
Extracted from dAppServer object store pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Load() reads .core/view.yml from any directory via io.Medium,
LoadVerified() adds ed25519 signature check. Uses MockMedium for tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign() computes signature over canonical YAML (excluding sign field),
Verify() checks against public key. Tampered manifests are rejected.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Manifest struct, Permissions, Parse() from YAML, SlotNames() helper.
Foundation for Phase 4 module system.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move i18n-validate tool from core/cli internal/tools/ into
pkg/i18n/internal/validate/. Remove bugseti plan docs (now in
core/bugseti repo).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix remaining 187 pkg/ files referencing core/cli → core/go
- Move SDK library code from internal/cmd/sdk/ → pkg/sdk/ (new package)
- Create pkg/rag/helpers.go with convenience functions from internal/cmd/rag/
- Fix pkg/mcp/tools_rag.go to use pkg/rag instead of internal/cmd/rag
- Fix pkg/build/buildcmd/cmd_sdk.go and pkg/release/sdk.go to use pkg/sdk
- Remove all non-library content: main.go, internal/, cmd/, docker/,
scripts/, tasks/, tools/, .core/, .forgejo/, .woodpecker/, Taskfile.yml
- Run go mod tidy to trim unused dependencies
core/go is now a pure Go package suite (library only).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude <developers@lethean.io>
Reviewed-on: #3
Port the standalone lab dashboard (lab.lthn.io) into the core CLI as
pkg/lab/ with collectors, handlers, and HTML templates. The dashboard
monitors machines, Docker containers, Forgejo, HuggingFace models,
training runs, and InfluxDB metrics with SSE live updates.
New command: core lab serve --bind :8080
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ports all remaining LEM pipeline commands from pkg/lem into core ml,
eliminating the standalone LEM CLI dependency. Each command is split
into reusable business logic (pkg/ml/) and a thin cobra wrapper
(internal/cmd/ml/).
New commands: query, inventory, metrics, ingest, normalize, seed-influx,
consolidate, import-all, approve, publish, coverage.
Adds Path(), Exec(), QueryRowScan() convenience methods to DB type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tracks model size at load time and checks Metal active memory after
each generation. If usage exceeds 3× model size, forces double GC
and cache clear as a safety net.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Go wrapper was tracking inter-array references via desc.inputs,
creating chains that kept all intermediate arrays alive across requests.
After 3-4 requests, Metal memory grew to 170GB+ and macOS killed the
process.
Fix: remove desc.inputs/numRefs entirely. MLX-C has its own internal
reference counting — when Go GC finalizes an Array wrapper, it calls
mlx_array_free which decrements the C-side refcount. If the C-side
count reaches 0, Metal memory is freed. Go GC + MLX-C refcounting
together handle all lifecycle management correctly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Go GC cannot see Metal/C memory pressure, so intermediate arrays from
each forward pass accumulated without bound, causing OOM kills after
3-4 requests. Fix: runtime.SetFinalizer on every Array releases C
handles when GC collects them, and runtime.GC() is forced every 4
tokens during generation. Also adds SetMemoryLimit(24GB) as a hard
Metal ceiling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ClearCache() wrapping mlx_clear_cache
- Clear Metal allocator cache every 8 tokens during generation
- Set 16GB cache limit on backend init
- Prevents GPU memory from growing unbounded during inference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gemma 3 tokenizer.json uses [["a","b"],...] format for merges
instead of the ["a b",...] format. Support both.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Supports both multimodal (Gemma3ForConditionalGeneration) and
text-only configs. Resolves weights with language_model. prefix
fallback. Computes head_dim from hidden_size when missing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use pure C callback instead of //export to avoid const char* vs
GoString type mismatch in cgo-generated headers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use _axis/_axes variants for softmax, argmax, topk, sum, mean, squeeze,
concatenate, argpartition
- Fix size_t vs int for count parameters throughout
- Fix int64_t strides in as_strided
- Add mlx_optional_int + mode param to quantized_matmul
- Use mlx_array_new() for null arrays (freqs, key, mask, sinks)
- Fix expand_dims to single-axis signature
- Fix compile callback signature (size_t index)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CGo wrapper for mlx-c providing zero-Python Metal GPU inference.
Includes Gemma 3 model architecture, BPE tokenizer, KV cache,
composable sampling, and OpenAI-compatible serve command.
Build-tagged (darwin && arm64 && mlx) with stubs for cross-platform.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Port LEM scoring/training pipeline into CoreGo as pkg/ml with:
- Inference abstraction with HTTP, llama-server, and Ollama backends
- 3-tier scoring engine (heuristic, exact, LLM judge)
- Capability and content probes for model evaluation
- GGUF/safetensors format converters, MLX to PEFT adapter conversion
- DuckDB integration for training data pipeline
- InfluxDB metrics for lab dashboard
- Training data export (JSONL + Parquet)
- Expansion generation pipeline with distributed workers
- 10 CLI commands under 'core ml' (score, probe, export, expand, status, gguf, convert, agent, worker)
- 5 MCP tools (ml_generate, ml_score, ml_probe, ml_status, ml_backends)
All 37 ML tests passing. Binary builds at 138MB with all commands.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move module identity to our own Forgejo instance. All import paths
updated across 434 Go files, sub-module go.mod files, and go.work.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all exec.Command("gh", ...) calls with the existing pkg/forge
wrapper around the Forgejo Go SDK. BugSETI no longer requires the gh
CLI to be installed.
Changes:
- fetcher: use forge.ListIssues/GetIssue instead of gh issue list/view
- submit: use forge.ForkRepo/CreatePullRequest instead of gh pr create
- seeder: use git clone with forge URL + token auth instead of gh clone
- ghcheck: CheckForge() returns *forge.Client via forge.NewFromConfig()
- config: add ForgeURL/ForgeToken fields (GitHubToken kept for migration)
- pkg/forge: add Token(), GetCurrentUser(), ForkRepo(), CreatePullRequest(),
ListIssueComments(), and label filtering to ListIssuesOpts
Co-Authored-By: Virgil <virgil@lethean.io>
Prevent path traversal in Journal.Append() by validating RepoOwner and
RepoName before using them in file paths. Malicious values like
"../../etc/cron.d" could previously write outside the journal baseDir.
Defence layers:
- Reject inputs containing path separators (/ or \)
- Reject ".." and "." traversal components
- Validate against safe character regex ^[a-zA-Z0-9][a-zA-Z0-9._-]*$
- Verify resolved absolute path stays within baseDir
Closes#46
CVSS 6.3 — OWASP A01:2021-Broken Access Control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass the API key via x-goog-api-key HTTP header instead of the URL
query parameter to prevent credential leakage in proxy logs, web
server access logs, and monitoring systems.
Resolves: #47 (CVSS 5.3, OWASP A09:2021)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements quota enforcement for agents including daily token limits,
daily job limits, concurrent job caps, model allowlists, and global
per-model budgets. Quota recovery returns 50% for failed jobs and
100% for cancelled jobs.
Go: AllowanceService with MemoryStore, AllowanceStore interface, and
25 tests covering all enforcement paths.
Laravel: migration for 5 tables (agent_allowances, quota_usage,
model_quotas, usage_reports, repo_limits), Eloquent models,
AllowanceService, QuotaMiddleware, and REST API routes.
Closes#99
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
