- Add --max-context flag to serve (sliding window, default 4 messages)
to prevent KV-cache explosion on multi-turn conversations
- Pass server max-tokens to chat UI HTML attribute instead of
hardcoded 2048 in JavaScript
- Add chat.js and chat_embed.go for embedded LEM chat UI
Co-Authored-By: Virgil <virgil@lethean.io>
Runs the same prompts through baseline and fine-tuned models, scores
both with the heuristic scorer, and outputs a comparison report with
LEK score deltas and improvement/regression counts.
Uses built-in content probes by default, or custom prompts file.
Co-Authored-By: Virgil <virgil@lethean.io>
Lesson command runs prompts from YAML definitions with state tracking,
sandwich signing, and interactive review mode. Sequence command runs
multiple lessons in order (vertical/strict or horizontal/flexible).
State files enable resume after interruption. Both output chat JSONL
compatible with 'core ml train'.
Co-Authored-By: Virgil <virgil@lethean.io>
Sandwich format wraps seed prompts with KB preamble (axioms framework)
and LEK-1 kernel postfix, then generates responses via local MLX
inference. Outputs chat JSONL compatible with 'core ml train'.
Supports --dry-run mode to output prompts without inference.
Co-Authored-By: Virgil <virgil@lethean.io>
Native MLX LoRA training on Apple Silicon — no Python required.
Reads chat-format JSONL, applies LoRA to target projections,
trains with AdamW + masked cross-entropy loss on assistant tokens.
Usage: core ml train --model-path /path/to/model --data training.jsonl
Co-Authored-By: Virgil <virgil@lethean.io>
Add compiled binary to gitignore. Remove entries for deleted paths
(cmd/bugseti/bugseti, internal/core-ide/core-ide).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove internal/bugseti/ (now core/bugseti repo), cmd/bugseti/
(now core/bugseti/cmd/), and internal/tools/ (i18n-validate
moved to core/go). core/cli internal/ is now empty.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Extract PHP/Laravel commands to `core/php` repo (42 files, standalone module)
- Extract CI/release + SDK commands to `core/ci` repo (10 files)
- Remove `internal/variants/` build tag system entirely
- Move all 30 remaining command packages from `internal/cmd/` to top-level `cmd/`
- Rewrite `main.go` with direct imports — no more variant selection
- PHP and CI are now optional via commented import lines in main.go
Co-authored-by: Claude <developers@lethean.io>
Reviewed-on: #2
Co-authored-by: Charon <charon@lthn.ai>
Co-committed-by: Charon <charon@lthn.ai>
- Change module from forge.lthn.ai/core/go to forge.lthn.ai/core/cli
- Remove pkg/ directory (now served from core/go)
- Add require + replace for forge.lthn.ai/core/go => ../go
- Update go.work to include ../go workspace module
- Fix all internal/cmd/* imports: pkg/ refs → forge.lthn.ai/core/go/pkg/
- Rename internal/cmd/sdk package to sdkcmd (avoids conflict with pkg/sdk)
- Remove SDK library files from internal/cmd/sdk/ (now in core/go/pkg/sdk/)
- Remove duplicate RAG helper functions from internal/cmd/rag/
- Remove stale cmd/core-ide/ (now in core/ide repo)
- Update IDE variant to remove core-ide import
- Fix test assertion for new module name
- Run go mod tidy to sync dependencies
core/cli is now a pure CLI application importing core/go for packages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Claude <developers@lethean.io>
Reviewed-on: #1
Port the standalone lab dashboard (lab.lthn.io) into the core CLI as
pkg/lab/ with collectors, handlers, and HTML templates. The dashboard
monitors machines, Docker containers, Forgejo, HuggingFace models,
training runs, and InfluxDB metrics with SSE live updates.
New command: core lab serve --bind :8080
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ports all remaining LEM pipeline commands from pkg/lem into core ml,
eliminating the standalone LEM CLI dependency. Each command is split
into reusable business logic (pkg/ml/) and a thin cobra wrapper
(internal/cmd/ml/).
New commands: query, inventory, metrics, ingest, normalize, seed-influx,
consolidate, import-all, approve, publish, coverage.
Adds Path(), Exec(), QueryRowScan() convenience methods to DB type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tracks model size at load time and checks Metal active memory after
each generation. If usage exceeds 3× model size, forces double GC
and cache clear as a safety net.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Go wrapper was tracking inter-array references via desc.inputs,
creating chains that kept all intermediate arrays alive across requests.
After 3-4 requests, Metal memory grew to 170GB+ and macOS killed the
process.
Fix: remove desc.inputs/numRefs entirely. MLX-C has its own internal
reference counting — when Go GC finalizes an Array wrapper, it calls
mlx_array_free which decrements the C-side refcount. If the C-side
count reaches 0, Metal memory is freed. Go GC + MLX-C refcounting
together handle all lifecycle management correctly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Go GC cannot see Metal/C memory pressure, so intermediate arrays from
each forward pass accumulated without bound, causing OOM kills after
3-4 requests. Fix: runtime.SetFinalizer on every Array releases C
handles when GC collects them, and runtime.GC() is forced every 4
tokens during generation. Also adds SetMemoryLimit(24GB) as a hard
Metal ceiling.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ClearCache() wrapping mlx_clear_cache
- Clear Metal allocator cache every 8 tokens during generation
- Set 16GB cache limit on backend init
- Prevents GPU memory from growing unbounded during inference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Gemma 3 tokenizer.json uses [["a","b"],...] format for merges
instead of the ["a b",...] format. Support both.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Supports both multimodal (Gemma3ForConditionalGeneration) and
text-only configs. Resolves weights with language_model. prefix
fallback. Computes head_dim from hidden_size when missing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use pure C callback instead of //export to avoid const char* vs
GoString type mismatch in cgo-generated headers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use _axis/_axes variants for softmax, argmax, topk, sum, mean, squeeze,
concatenate, argpartition
- Fix size_t vs int for count parameters throughout
- Fix int64_t strides in as_strided
- Add mlx_optional_int + mode param to quantized_matmul
- Use mlx_array_new() for null arrays (freqs, key, mask, sinks)
- Fix expand_dims to single-axis signature
- Fix compile callback signature (size_t index)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CGo wrapper for mlx-c providing zero-Python Metal GPU inference.
Includes Gemma 3 model architecture, BPE tokenizer, KV cache,
composable sampling, and OpenAI-compatible serve command.
Build-tagged (darwin && arm64 && mlx) with stubs for cross-platform.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Port LEM scoring/training pipeline into CoreGo as pkg/ml with:
- Inference abstraction with HTTP, llama-server, and Ollama backends
- 3-tier scoring engine (heuristic, exact, LLM judge)
- Capability and content probes for model evaluation
- GGUF/safetensors format converters, MLX to PEFT adapter conversion
- DuckDB integration for training data pipeline
- InfluxDB metrics for lab dashboard
- Training data export (JSONL + Parquet)
- Expansion generation pipeline with distributed workers
- 10 CLI commands under 'core ml' (score, probe, export, expand, status, gguf, convert, agent, worker)
- 5 MCP tools (ml_generate, ml_score, ml_probe, ml_status, ml_backends)
All 37 ML tests passing. Binary builds at 138MB with all commands.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move module identity to our own Forgejo instance. All import paths
updated across 434 Go files, sub-module go.mod files, and go.work.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The main.go was removed when Wails3 apps were added to cmd/, breaking
`go build .` for the core CLI. Restore it and update variants/full.go
to include daemon, forge, mcpcmd, prod, and session commands. Drop gitea
(superseded by forge) and unifi (unused).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HubService to the Wails service list and attempt hub registration
at startup when hubUrl is configured. Drains any pending operations
queued from previous sessions.
Co-Authored-By: Virgil <virgil@lethean.io>
Replace no-op stubs with real implementations for queueOp, drainPendingOps,
savePendingOps, and loadPendingOps. Operations are persisted to hub_pending.json
and replayed on next hub connection — 5xx/transport errors are retried, 4xx
responses are dropped as stale. Adds PendingCount() for queue inspection.
Co-Authored-By: Virgil <virgil@lethean.io>
Add Register, Heartbeat, ClaimIssue, UpdateStatus, ReleaseClaim,
and SyncStats methods for hub coordination. ClaimIssue returns
ConflictError on 409 and calls drainPendingOps before mutating.
Co-Authored-By: Virgil <virgil@lethean.io>
Exchange a Forge API token for a hub API key by POSTing to
/api/bugseti/auth/forge. Skips if hub token already cached.
Adds drainPendingOps() stub for future Task 7 use.
Co-Authored-By: Virgil <virgil@lethean.io>
Add doRequest() and doJSON() methods for hub API communication. doRequest
builds full URLs, sets bearer auth and JSON headers, tracks connected
state. doJSON handles status codes: 401 unauthorised, 409 ConflictError,
404 NotFoundError, and generic errors for other 4xx/5xx responses.
Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce HubService struct with types for hub coordination: PendingOp,
HubClaim, LeaderboardEntry, GlobalStats, ConflictError, NotFoundError.
Constructor generates a crypto/rand client ID when none exists. Includes
no-op loadPendingOps/savePendingOps stubs for future persistence.
Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HubURL, HubToken, ClientID, and ClientName fields to Config struct
for agentic portal integration. Include getter/setter methods following
the existing pattern (SetForgeURL, SetForgeToken also added).
Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all exec.Command("gh", ...) calls with the existing pkg/forge
wrapper around the Forgejo Go SDK. BugSETI no longer requires the gh
CLI to be installed.
Changes:
- fetcher: use forge.ListIssues/GetIssue instead of gh issue list/view
- submit: use forge.ForkRepo/CreatePullRequest instead of gh pr create
- seeder: use git clone with forge URL + token auth instead of gh clone
- ghcheck: CheckForge() returns *forge.Client via forge.NewFromConfig()
- config: add ForgeURL/ForgeToken fields (GitHubToken kept for migration)
- pkg/forge: add Token(), GetCurrentUser(), ForkRepo(), CreatePullRequest(),
ListIssueComments(), and label filtering to ListIssuesOpts
Co-Authored-By: Virgil <virgil@lethean.io>
The workspace map previously only cleaned up during Capture() calls,
meaning stale entries would accumulate indefinitely if no new captures
occurred. This adds:
- Background sweeper goroutine (Start/Stop lifecycle) that runs every 5
minutes to evict expired workspaces
- Configurable MaxWorkspaces and WorkspaceTTLMinutes in Config (defaults:
100 entries, 24h TTL) replacing hardcoded constants
- cleanup() now returns eviction count for observability logging
- Nil-config fallback to safe defaults
Fixes#54
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevent path traversal in Journal.Append() by validating RepoOwner and
RepoName before using them in file paths. Malicious values like
"../../etc/cron.d" could previously write outside the journal baseDir.
Defence layers:
- Reject inputs containing path separators (/ or \)
- Reject ".." and "." traversal components
- Validate against safe character regex ^[a-zA-Z0-9][a-zA-Z0-9._-]*$
- Verify resolved absolute path stays within baseDir
Closes#46
CVSS 6.3 — OWASP A01:2021-Broken Access Control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
