Port the standalone lab dashboard (lab.lthn.io) into the core CLI as
pkg/lab/ with collectors, handlers, and HTML templates. The dashboard
monitors machines, Docker containers, Forgejo, HuggingFace models,
training runs, and InfluxDB metrics with SSE live updates.
New command: core lab serve --bind :8080
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ports all remaining LEM pipeline commands from pkg/lem into core ml,
eliminating the standalone LEM CLI dependency. Each command is split
into reusable business logic (pkg/ml/) and a thin cobra wrapper
(internal/cmd/ml/).
New commands: query, inventory, metrics, ingest, normalize, seed-influx,
consolidate, import-all, approve, publish, coverage.
Adds Path(), Exec(), QueryRowScan() convenience methods to DB type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tracks model size at load time and checks Metal active memory after
each generation. If usage exceeds 3× model size, forces double GC
and cache clear as a safety net.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Go wrapper was tracking inter-array references via desc.inputs,
creating chains that kept all intermediate arrays alive across requests.
After 3-4 requests, Metal memory grew to 170GB+ and macOS killed the
process.
Fix: remove desc.inputs/numRefs entirely. MLX-C has its own internal
reference counting — when Go GC finalizes an Array wrapper, it calls
mlx_array_free which decrements the C-side refcount. If the C-side
count reaches 0, Metal memory is freed. Go GC + MLX-C refcounting
together handle all lifecycle management correctly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Go GC cannot see Metal/C memory pressure, so intermediate arrays from
each forward pass accumulated without bound, causing OOM kills after
3-4 requests. Fix: runtime.SetFinalizer on every Array releases C
handles when GC collects them, and runtime.GC() is forced every 4
tokens during generation. Also adds SetMemoryLimit(24GB) as a hard
Metal ceiling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ClearCache() wrapping mlx_clear_cache
- Clear Metal allocator cache every 8 tokens during generation
- Set 16GB cache limit on backend init
- Prevents GPU memory from growing unbounded during inference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gemma 3 tokenizer.json uses [["a","b"],...] format for merges
instead of the ["a b",...] format. Support both.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Supports both multimodal (Gemma3ForConditionalGeneration) and
text-only configs. Resolves weights with language_model. prefix
fallback. Computes head_dim from hidden_size when missing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use pure C callback instead of //export to avoid const char* vs
GoString type mismatch in cgo-generated headers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use _axis/_axes variants for softmax, argmax, topk, sum, mean, squeeze,
concatenate, argpartition
- Fix size_t vs int for count parameters throughout
- Fix int64_t strides in as_strided
- Add mlx_optional_int + mode param to quantized_matmul
- Use mlx_array_new() for null arrays (freqs, key, mask, sinks)
- Fix expand_dims to single-axis signature
- Fix compile callback signature (size_t index)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CGo wrapper for mlx-c providing zero-Python Metal GPU inference.
Includes Gemma 3 model architecture, BPE tokenizer, KV cache,
composable sampling, and OpenAI-compatible serve command.
Build-tagged (darwin && arm64 && mlx) with stubs for cross-platform.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Port LEM scoring/training pipeline into CoreGo as pkg/ml with:
- Inference abstraction with HTTP, llama-server, and Ollama backends
- 3-tier scoring engine (heuristic, exact, LLM judge)
- Capability and content probes for model evaluation
- GGUF/safetensors format converters, MLX to PEFT adapter conversion
- DuckDB integration for training data pipeline
- InfluxDB metrics for lab dashboard
- Training data export (JSONL + Parquet)
- Expansion generation pipeline with distributed workers
- 10 CLI commands under 'core ml' (score, probe, export, expand, status, gguf, convert, agent, worker)
- 5 MCP tools (ml_generate, ml_score, ml_probe, ml_status, ml_backends)
All 37 ML tests passing. Binary builds at 138MB with all commands.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move module identity to our own Forgejo instance. All import paths
updated across 434 Go files, sub-module go.mod files, and go.work.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all exec.Command("gh", ...) calls with the existing pkg/forge
wrapper around the Forgejo Go SDK. BugSETI no longer requires the gh
CLI to be installed.
Changes:
- fetcher: use forge.ListIssues/GetIssue instead of gh issue list/view
- submit: use forge.ForkRepo/CreatePullRequest instead of gh pr create
- seeder: use git clone with forge URL + token auth instead of gh clone
- ghcheck: CheckForge() returns *forge.Client via forge.NewFromConfig()
- config: add ForgeURL/ForgeToken fields (GitHubToken kept for migration)
- pkg/forge: add Token(), GetCurrentUser(), ForkRepo(), CreatePullRequest(),
ListIssueComments(), and label filtering to ListIssuesOpts
Co-Authored-By: Virgil <virgil@lethean.io>
Prevent path traversal in Journal.Append() by validating RepoOwner and
RepoName before using them in file paths. Malicious values like
"../../etc/cron.d" could previously write outside the journal baseDir.
Defence layers:
- Reject inputs containing path separators (/ or \)
- Reject ".." and "." traversal components
- Validate against safe character regex ^[a-zA-Z0-9][a-zA-Z0-9._-]*$
- Verify resolved absolute path stays within baseDir
Closes#46
CVSS 6.3 — OWASP A01:2021-Broken Access Control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass the API key via x-goog-api-key HTTP header instead of the URL
query parameter to prevent credential leakage in proxy logs, web
server access logs, and monitoring systems.
Resolves: #47 (CVSS 5.3, OWASP A09:2021)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements quota enforcement for agents including daily token limits,
daily job limits, concurrent job caps, model allowlists, and global
per-model budgets. Quota recovery returns 50% for failed jobs and
100% for cancelled jobs.
Go: AllowanceService with MemoryStore, AllowanceStore interface, and
25 tests covering all enforcement paths.
Laravel: migration for 5 tables (agent_allowances, quota_usage,
model_quotas, usage_reports, repo_limits), Eloquent models,
AllowanceService, QuotaMiddleware, and REST API routes.
Closes#99
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds FindByForgejoUser() to Spinner so dispatch matches issues
assigned to Forgejo users (Virgil, Claude, Charon) even when the
agent config key differs (e.g. Hypnos → forgejo_user: Claude).
Searches config key first (direct match), then ForgejoUser field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds pkg/ratelimit for Gemini API rate limiting with sliding window
(RPM/TPM/RPD), persistent state, and token counting. Replaces the
bash agent-runner.sh with a native Go implementation under
`core ai dispatch {run,watch,status}` for local queue processing.
Rate limiting:
- Per-model quotas (RPM, TPM, RPD) with 1-minute sliding window
- WaitForCapacity blocks until capacity available or context cancelled
- Persistent state in ~/.core/ratelimits.yaml
- Default quotas for Gemini 3 Pro/Flash, 2.5 Pro, 2.0 Flash/Lite
- CountTokens helper calls Google tokenizer API
- CLI: core ai ratelimits {show,reset,count,config,check}
Dispatch runner:
- core ai dispatch run — process single ticket from queue
- core ai dispatch watch — daemon mode with configurable interval
- core ai dispatch status — show queue/active/done counts
- Supports claude/codex/gemini runners with rate-limited Gemini
- File-based locking with stale PID detection
- Completion handler updates issue labels on success/failure
Closes#42
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace fmt.Errorf() with structured log.E() errors in agentci, forge,
jobrunner packages. Update PipelineSignal comment to reflect dispatch
fields. Add TODO markers for charmbracelet/ssh migration across all
exec ssh call sites.
Co-Authored-By: Virgil <virgil@lethean.io>
Tickets now carry model (sonnet/haiku/opus) and runner (claude/codex)
fields. agent-runner.sh dispatches to the right backend. Defaults to
claude with sonnet model for cost efficiency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Config-driven agent targets replace hardcoded map so new agents
can be added via CLI instead of recompiling. Includes setup script
for bootstrapping agent machines and CLI commands for management.
- Add pkg/agentci with config types and CRUD (LoadAgents, SaveAgent, etc.)
- Add CLI: core ai agent {add,list,status,logs,setup,remove}
- Add scripts/agent-setup.sh (SSH bootstrap: dirs, cron, prereq check)
- Headless loads agents from ~/.core/config.yaml
- Dispatch ticket includes forgejo_user for dynamic clone URLs
- agent-runner.sh reads username from ticket JSON, not hardcoded
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Dispatch handler matches child issues that need coding (no PR yet,
assigned to a known agent) and SCPs ticket JSON to the agent's
queue directory via SSH. Includes dedup across queue/active/done
and posts dispatch comments on issues.
- Extend PipelineSignal with NeedsCoding, Assignee, IssueTitle, IssueBody
- Extend ForgejoSource to emit signals for unstarted children
- Add DispatchHandler with Match/Execute (SCP ticket delivery)
- Add agent-runner.sh cron-based queue runner for agent machines
- Wire dispatch handler into headless mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Switch Angular from hash-based to path-based routing so each Wails window
(/tray, /main, /settings) loads its correct route. Archive GitHub Actions
workflows to .gh-actions/, update Forgejo deploy registry to dappco.re/osi,
and apply gofmt/alignment fixes across packages.
Co-Authored-By: Virgil <virgil@lethean.io>
Replace all GitHub API and gh CLI dependencies with Forgejo SDK via
pkg/forge. The bash dispatcher burned a week of credit in a day due to
bugs — the jobrunner now talks directly to Forgejo.
- Add forge client methods: CreateIssueComment, CloseIssue, MergePullRequest,
SetPRDraft, ListPRReviews, GetCombinedStatus, DismissReview
- Create ForgejoSource implementing JobSource (epic polling, checklist
parsing, commit status via combined status API)
- Rewrite all 5 handlers to accept *forge.Client instead of shelling out
- Replace ResolveThreadsHandler with DismissReviewsHandler (Forgejo has
no thread resolution API — dismiss stale REQUEST_CHANGES reviews instead)
- Delete pkg/jobrunner/github/ and handlers/exec.go entirely
- Update internal/core-ide/headless.go to wire Forgejo source and handlers
- All 33 tests pass with mock Forgejo HTTP servers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reconcile callers with actual function signatures after merging IO
migration branches. Some functions gained io.Medium params (repos.*),
others kept their original signatures (release.*, cache.*, container.*).
- Add io.Local to repos.LoadRegistry/FindRegistry/ScanDirectory callers
- Remove extra io.Local from release.ConfigExists/LoadConfig/WriteConfig callers
- Fix cache.New call (remove nil Medium arg)
- Add missing IsCPPProject to build discovery
- Add missing fields to mcp.Service struct (subsystems, logger, etc.)
- Add DefaultTCPAddr constant to mcp transport
- Fix node.go interface check (coreio.Medium, not coreio.Node)
- Fix container.linuxkit LoadState/EnsureLogsDir arg counts
- Fix vm templates to use package-level functions
- Remove unused Medium field from DaemonOptions
Co-Authored-By: Virgil <virgil@lethean.io>
