Commit graph

415 commits

Author SHA1 Message Date
Snider
bf994fab17 feat: embed CLI locale files, wire i18n ExtraFS
Some checks failed
Deploy / build (push) Failing after 7s
Security Scan / security (push) Successful in 20s
- Added locales/en.json with 90 translation keys for doctor, pkg commands
- Main() embeds CLI locales automatically
- MainWithLocales() accepts additional FSSource for consuming binaries
- Ecosystem packages can ship their own locale files

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-17 00:28:52 +00:00
Snider
2efcbd59ec refactor: move I18nService to go-i18n, simplify log wrapper
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 13s
I18nService now lives in go-i18n as NewCoreService() — any binary can
use it without importing cli. Log convenience functions use go-log
directly. Removed LogService/NewLogService/daemon_cmd wrappers.

Root go.mod: 1 direct forge dep (core/go).

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-16 16:24:12 +00:00
Snider
55b556d1af refactor: split library from binary, remove ecosystem commands
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 13s
core/cli is now a pure library (pkg/cli). The binary moves to
cmd/core/ as a separate sub-module with its own go.mod.

Removed from binary: gocmd (→ lint/go-build), service (→ go-process),
session (→ go-session), module (→ go-scm), plugin (→ go-scm).
Removed from framework: go-crypt, workspace, daemon_cmd.

Root go.mod: 1 direct forge dep (core/go). Cross-compiles CGO_ENABLED=0.

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-16 16:14:40 +00:00
Snider
76cd3a5306 refactor: code quality improvements from Gemini review
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 13s
- Split frame.go: extract built-in components to frame_components.go
- Replace custom indexOf with strings.Index in frame_test.go
- Make prompt.go testable: accept io.Reader via SetStdin, add tests
- Decompose runGoQA: extract emitQAJSON and emitQASummary helpers
- DRY: centralise loadConfig into cmd/config/cmd.go
- Remove hardcoded MACOSX_DEPLOYMENT_TARGET from test/fuzz/cov commands
- Add error assertions to coverage_test.go

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:29:44 +00:00
Snider
48c3f08bcb fix: use Signal(0) for process polling in daemonRunStop
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 12s
Consistent with cmd/service stopDaemon — polls process directly
instead of PID file removal to avoid PID reuse false positives.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:16:20 +00:00
Snider
10a1c8ce07 refactor: remove daemon types moved to go-process, keep Mode/DetectMode
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 20s
Remove PIDFile, HealthServer, Daemon, DaemonOptions, HealthCheck,
Run, and RunWithTimeout from daemon.go — all now live in go-process.
Retain Mode type (ModeInteractive/ModePipe/ModeDaemon), DetectMode(),
IsTTY(), IsStdinTTY(), and IsStderrTTY() as CLI-specific helpers.

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-09 14:14:54 +00:00
Snider
d6cea1dffe feat: add generic AddDaemonCommand with go-process daemon types
Provides a reusable daemon CLI command builder that registers
start/stop/status/run subcommands. Consumers (go-ai, ide, etc.)
call AddDaemonCommand(root, config) with a RunForeground callback
for their business logic. Uses go-process for PID file, health
server, and daemon lifecycle management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 14:05:16 +00:00
Snider
de579ad01d feat: add RangeArgs positional arg validator
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 13s
Wraps cobra.RangeArgs for parity with ExactArgs, MinimumNArgs,
MaximumNArgs — allows go-devops to drop its direct cobra import.

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-09 12:05:05 +00:00
Snider
37a8ae8d31 refactor: swap pkg/framework imports to pkg/core
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 19s
Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-06 14:10:55 +00:00
Snider
f7d72c843b refactor: swap pkg imports to standalone modules
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 16s
- pkg/session → go-session (ParseTranscript now returns ParseStats)
- pkg/workspace → go-io/workspace
- pkg/manifest,marketplace,plugin,repos → go-scm (from prior session)

Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-06 13:48:07 +00:00
Snider
4f1d9a5c3b refactor: swap pkg/{io,log} imports to go-io/go-log
Some checks failed
Deploy / build (push) Failing after 3s
Security Scan / security (push) Successful in 12s
Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-06 12:23:33 +00:00
Snider
945be23adc refactor: use core/go-i18n module instead of core/go/pkg/i18n
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 12s
Co-Authored-By: Virgil <virgil@lethean.io>
2026-03-06 09:09:06 +00:00
Claude
0941ba865f
chore: use min()/max() builtins (Go 1.21+)
All checks were successful
Security Scan / security (pull_request) Successful in 23s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:27:11 +00:00
Claude
c2a57f2227
chore: fmt.Errorf(static) → errors.New
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 16:25:14 +00:00
Snider
fa3a7bcd83 feat(cli): add Go 1.26 iterators and modernise idioms
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 16s
- Add Children() iter.Seq on TreeNode for range-based traversal
- Add RegisteredCommands() iter.Seq on command registry (mutex-safe)
- Add Regions()/Slots() iterators on Composite layout
- Add Tasks()/Snapshots() iterators on TaskTracker (mutex-safe)
- Use strings.FieldsSeq, strings.SplitSeq in parseMultiSelection
- Use range-over-int where applicable

Co-Authored-By: Gemini <noreply@google.com>
Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-23 04:57:24 +00:00
Snider
38765962f8 feat(cli): add Int64Flag, DurationFlag helpers; remove NewPassthrough
Add Int64Flag and DurationFlag to the flag helper set for commands
needing int64 seeds and time.Duration intervals. Remove NewPassthrough
which enabled the anti-pattern of bypassing cobra flag parsing with
stdlib flag.FlagSet.

Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-23 03:32:39 +00:00
Snider
5500c3912c feat(cli): add WithCommands lifecycle pattern for command registration
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 19s
Main() now accepts variadic framework.Option args, allowing commands
to register through the Core lifecycle via WithCommands(). This matches
the pattern from core/go and enables LEM and other consumers to use
the same API.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:31 +00:00
Claude
f54876abb5
test(frame): add message routing edge case tests 2026-02-22 21:21:11 +00:00
Claude
cf6e4700c9
test(frame): add Navigate/Back tests with FrameModel 2026-02-22 21:20:47 +00:00
Claude
d540e5706b
test(frame): add spatial focus navigation tests 2026-02-22 21:20:24 +00:00
Claude
96b2cb6547
refactor(frame): unify String() with View() via viewLocked()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:18:55 +00:00
Claude
1c6e910251
feat(frame): replace raw ANSI runLive with tea.Program
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:16:48 +00:00
Claude
331bcd564d
feat(frame): implement tea.Model (Init, Update, View) with lipgloss layout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:12:32 +00:00
Claude
acfbc2aaee
feat(frame): add focus management fields, Focused(), Focus(), WithKeyMap() 2026-02-22 21:07:32 +00:00
Claude
02e8343ee5
feat(frame): add KeyMap with default bindings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:03:50 +00:00
Claude
aa5cfc312d
feat(frame): add FrameModel interface and modelAdapter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 21:02:53 +00:00
Claude
6a8bd92189
feat: add pkg/cli with TUI components (#14, #15)
Some checks failed
Deploy / build (push) Failing after 4s
Security Scan / security (push) Successful in 15s
Move pkg/cli from core/go to core/cli. Includes Frame AppShell,
Stream, TaskTracker, Tree, Rich Table. Update imports to v0.0.1
tagged deps and fix openpgp import path for go-crypt split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 20:42:00 +00:00
abe74a1a3d refactor: split CLI from monorepo, import core/go as library (#1)
- Change module from forge.lthn.ai/core/go to forge.lthn.ai/core/cli
- Remove pkg/ directory (now served from core/go)
- Add require + replace for forge.lthn.ai/core/go => ../go
- Update go.work to include ../go workspace module
- Fix all internal/cmd/* imports: pkg/ refs → forge.lthn.ai/core/go/pkg/
- Rename internal/cmd/sdk package to sdkcmd (avoids conflict with pkg/sdk)
- Remove SDK library files from internal/cmd/sdk/ (now in core/go/pkg/sdk/)
- Remove duplicate RAG helper functions from internal/cmd/rag/
- Remove stale cmd/core-ide/ (now in core/ide repo)
- Update IDE variant to remove core-ide import
- Fix test assertion for new module name
- Run go mod tidy to sync dependencies

core/cli is now a pure CLI application importing core/go for packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-authored-by: Claude <developers@lethean.io>
Reviewed-on: #1
2026-02-16 14:24:37 +00:00
Snider
4e47485efe refactor: update import paths from cli to go package structure 2026-02-16 13:47:52 +00:00
a1306bc321 feat/ml-integration (#2)
Co-authored-by: Charon (snider-linux) <charon@lethean.io>
Co-authored-by: Snider <snider@host.uk.com>
Co-authored-by: Virgil <virgil@lethean.io>
Co-authored-by: Claude <developers@lethean.io>
Reviewed-on: #2
Co-authored-by: Snider <snider@lethean.io>
Co-committed-by: Snider <snider@lethean.io>
2026-02-16 06:19:09 +00:00
Claude
50da0adcb7 feat: integrate lab dashboard as core lab serve
Port the standalone lab dashboard (lab.lthn.io) into the core CLI as
pkg/lab/ with collectors, handlers, and HTML templates. The dashboard
monitors machines, Docker containers, Forgejo, HuggingFace models,
training runs, and InfluxDB metrics with SSE live updates.

New command: core lab serve --bind :8080

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
c5bc97de19 feat: port 11 LEM data management commands into core ml
Ports all remaining LEM pipeline commands from pkg/lem into core ml,
eliminating the standalone LEM CLI dependency. Each command is split
into reusable business logic (pkg/ml/) and a thin cobra wrapper
(internal/cmd/ml/).

New commands: query, inventory, metrics, ingest, normalize, seed-influx,
consolidate, import-all, approve, publish, coverage.

Adds Path(), Exec(), QueryRowScan() convenience methods to DB type.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
045f8fc110 feat: add Metal memory budget monitoring after each request
Tracks model size at load time and checks Metal active memory after
each generation. If usage exceeds 3× model size, forces double GC
and cache clear as a safety net.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
c5689c3e83 fix: remove Go-side array ref tracking, rely on MLX-C refcounting
The Go wrapper was tracking inter-array references via desc.inputs,
creating chains that kept all intermediate arrays alive across requests.
After 3-4 requests, Metal memory grew to 170GB+ and macOS killed the
process.

Fix: remove desc.inputs/numRefs entirely. MLX-C has its own internal
reference counting — when Go GC finalizes an Array wrapper, it calls
mlx_array_free which decrements the C-side refcount. If the C-side
count reaches 0, Metal memory is freed. Go GC + MLX-C refcounting
together handle all lifecycle management correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
1d4ec55d05 fix: add GC-based memory management for MLX array handles
Go GC cannot see Metal/C memory pressure, so intermediate arrays from
each forward pass accumulated without bound, causing OOM kills after
3-4 requests. Fix: runtime.SetFinalizer on every Array releases C
handles when GC collects them, and runtime.GC() is forced every 4
tokens during generation. Also adds SetMemoryLimit(24GB) as a hard
Metal ceiling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
e6ada25bd8 fix: add Metal cache management to prevent memory growth
- Add ClearCache() wrapping mlx_clear_cache
- Clear Metal allocator cache every 8 tokens during generation
- Set 16GB cache limit on backend init
- Prevents GPU memory from growing unbounded during inference

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
f76bf0f0c0 fix: correct SDPA mask mode and slice logits to last position 2026-02-16 05:53:52 +00:00
Claude
b1f76ce7db fix: use affine quantization mode and infer head_dim from weights 2026-02-16 05:53:52 +00:00
Claude
f416f7e529 debug: add shape logging and stderr error handler for inference debugging 2026-02-16 05:53:52 +00:00
Claude
d92d097a7f feat: support quantized inference (4-bit) for Gemma 3
- Add QuantizedLinear with QuantizedMatmul for packed uint32 weights
- Add quantized Embedding with Dequantize before lookup
- Parse quantization config (group_size, bits) from config.json
- Detect .scales/.biases weight tensors and auto-select quantized path
- Add Dequantize op wrapping mlx_dequantize
- Add safety guard to KVCache.Update for malformed shapes
- Handle tied embeddings with quantization (AsLinear helper)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
7fc1571f93 fix: handle both string and array merge formats in tokenizer
Gemma 3 tokenizer.json uses [["a","b"],...] format for merges
instead of the ["a b",...] format. Support both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
f4303fada2 feat: handle nested text_config and language_model weight prefix
Supports both multimodal (Gemma3ForConditionalGeneration) and
text-only configs. Resolves weights with language_model. prefix
fallback. Computes head_dim from hidden_size when missing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
97ffe91cde chore: target macOS 26.0, fix duplicate -lstdc++ linker warning
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
e4467dd977 fix: remove unused vars in TopP sampler placeholder
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
828a5c0853 fix: resolve CGo type conflict in error handler
Use pure C callback instead of //export to avoid const char* vs
GoString type mismatch in cgo-generated headers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
a0435a84ea fix: correct 20 mlx-c API mismatches for v0.4.1
- Use _axis/_axes variants for softmax, argmax, topk, sum, mean, squeeze,
  concatenate, argpartition
- Fix size_t vs int for count parameters throughout
- Fix int64_t strides in as_strided
- Add mlx_optional_int + mode param to quantized_matmul
- Use mlx_array_new() for null arrays (freqs, key, mask, sinks)
- Fix expand_dims to single-axis signature
- Fix compile callback signature (size_t index)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
c25e1a633c fix: correct mlx_closure_new_func_payload signature for mlx-c v0.4.1
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
8ee0c4bc4e feat: add native MLX backend for Apple Silicon inference (pkg/mlx)
CGo wrapper for mlx-c providing zero-Python Metal GPU inference.
Includes Gemma 3 model architecture, BPE tokenizer, KV cache,
composable sampling, and OpenAI-compatible serve command.

Build-tagged (darwin && arm64 && mlx) with stubs for cross-platform.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
5ff4b8a2eb feat: add ML inference, scoring, and training pipeline (pkg/ml)
Port LEM scoring/training pipeline into CoreGo as pkg/ml with:
- Inference abstraction with HTTP, llama-server, and Ollama backends
- 3-tier scoring engine (heuristic, exact, LLM judge)
- Capability and content probes for model evaluation
- GGUF/safetensors format converters, MLX to PEFT adapter conversion
- DuckDB integration for training data pipeline
- InfluxDB metrics for lab dashboard
- Training data export (JSONL + Parquet)
- Expansion generation pipeline with distributed workers
- 10 CLI commands under 'core ml' (score, probe, export, expand, status, gguf, convert, agent, worker)
- 5 MCP tools (ml_generate, ml_score, ml_probe, ml_status, ml_backends)

All 37 ML tests passing. Binary builds at 138MB with all commands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00
Claude
23b82482f2 refactor: rename module from github.com/host-uk/core to forge.lthn.ai/core/cli
Move module identity to our own Forgejo instance. All import paths
updated across 434 Go files, sub-module go.mod files, and go.work.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 05:53:52 +00:00