Clotho/LEM
1
0
Fork 0
forked from lthn/LEM
Code Pull requests Activity
LEM/go.mod

70 lines
3 KiB
Modula-2
Raw Normal View History

feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
module forge.lthn.ai/lthn/lem
chore: bump go directive to 1.26.0 Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-22 20:33:49 +00:00
go 1.26.0
feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
require (
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
forge.lthn.ai/core/cli v0.0.1
chore: pin forge deps to v0.0.1 tags for Go 1.26 compat Go 1.26 rejects non-semver version strings (like 'main') in go.mod. Tags v0.0.1 now exist on all forge repos — workspace still overrides for local development. Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-22 20:15:06 +00:00
forge.lthn.ai/core/go-i18n v0.0.1
forge.lthn.ai/core/go-ml v0.0.1
forge.lthn.ai/core/go-mlx v0.0.1
feat(lem): integrate Poindexter for spatial score indexing and analytics - Add feature vector extraction (6D grammar, 8D heuristic, 14D combined) - Add KDTree ScoreIndex with cosine distance for probe clustering - Add score distribution analytics (percentiles, variance, skewness) - Add grammar-profile dedup filtering to distill pipeline - Add spatial gap detection (FindGaps) for coverage analysis - Wire analytics into coverage CLI (PrintScoreAnalytics) New files: features.go, cluster.go, analytics.go + tests Modified: distill.go (dedup filter), coverage.go (analytics output) Dep: github.com/Snider/Poindexter Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-22 21:26:06 +00:00
github.com/Snider/Poindexter v0.0.0-20260104200422-91146b212a1f
feat: add parquet, publish, metrics, convert commands - `lem parquet` — export JSONL training splits to Parquet (parquet-go) - `lem publish` — push Parquet files to HuggingFace dataset repo - `lem metrics` — push DuckDB golden set stats to InfluxDB - `lem convert` — MLX LoRA adapter → HuggingFace PEFT format (pure Go safetensors read/write/transpose, no PyTorch needed) Dependencies added: parquet-go, go-huggingface, go-rocm, go-pytorch, gotch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:05:08 +00:00
github.com/marcboeker/go-duckdb v1.8.5
github.com/parquet-go/parquet-go v0.27.0
feat: native Metal distillation command + .core/ai config Add `lem distill` — full Go pipeline for self-distillation using go-mlx (native Metal inference) and go-i18n/reversal (v3 grammar scoring). Replaces the Python distill.py bridge entirely. New files: - .core/ai/ai.yaml: global defaults (scorer, generation, distill) - .core/ai/models/gemma3/{27b,1b}.yaml: model configs with paths, kernel, lessons, baselines - .core/ai/probes.yaml: probe sets grouped by training phase - pkg/lem/config.go: YAML config loaders for .core/ai/ - pkg/lem/grammar.go: in-process grammar scoring (ComputeGrammarScore, ComputeDelta, ScoreResponse) extracted from cmd/scorer - pkg/lem/distill.go: RunDistill command — best-of-N generation, grammar quality gate, training JSONL output - pkg/lem/backend_metal.go: blank import for go-mlx Metal registration Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-21 23:42:55 +00:00
gopkg.in/yaml.v3 v3.0.1
)
feat: add parquet, publish, metrics, convert commands - `lem parquet` — export JSONL training splits to Parquet (parquet-go) - `lem publish` — push Parquet files to HuggingFace dataset repo - `lem metrics` — push DuckDB golden set stats to InfluxDB - `lem convert` — MLX LoRA adapter → HuggingFace PEFT format (pure Go safetensors read/write/transpose, no PyTorch needed) Dependencies added: parquet-go, go-huggingface, go-rocm, go-pytorch, gotch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:05:08 +00:00
require (
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
forge.lthn.ai/core/go v0.0.1 // indirect
forge.lthn.ai/core/go-crypt v0.0.1 // indirect
chore: pin forge deps to v0.0.1 tags for Go 1.26 compat Go 1.26 rejects non-semver version strings (like 'main') in go.mod. Tags v0.0.1 now exist on all forge repos — workspace still overrides for local development. Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-22 20:15:06 +00:00
forge.lthn.ai/core/go-inference v0.0.1 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
github.com/ProtonMail/go-crypto v1.3.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/apache/arrow-go/v18 v18.5.1 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/charmbracelet/bubbletea v1.3.10 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
github.com/charmbracelet/colorprofile v0.4.1 // indirect
github.com/charmbracelet/lipgloss v1.1.1-0.20250404203927-76690c660834 // indirect
github.com/charmbracelet/x/ansi v0.11.4 // indirect
github.com/charmbracelet/x/cellbuf v0.0.14 // indirect
github.com/charmbracelet/x/term v0.2.2 // indirect
github.com/clipperhouse/displaywidth v0.7.0 // indirect
github.com/clipperhouse/stringish v0.1.1 // indirect
github.com/clipperhouse/uax29/v2 v2.4.0 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
github.com/goccy/go-json v0.10.5 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
github.com/google/flatbuffers v25.12.19+incompatible // indirect
feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
github.com/google/uuid v1.6.0 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/inconshreveable/mousetrap v1.1.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
github.com/klauspost/compress v1.18.4 // indirect
chore: sync indirect deps from workspace Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-19 13:13:08 +00:00
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
github.com/mattn/go-runewidth v0.0.19 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.16.0 // indirect
feat: add parquet, publish, metrics, convert commands - `lem parquet` — export JSONL training splits to Parquet (parquet-go) - `lem publish` — push Parquet files to HuggingFace dataset repo - `lem metrics` — push DuckDB golden set stats to InfluxDB - `lem convert` — MLX LoRA adapter → HuggingFace PEFT format (pure Go safetensors read/write/transpose, no PyTorch needed) Dependencies added: parquet-go, go-huggingface, go-rocm, go-pytorch, gotch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:05:08 +00:00
github.com/parquet-go/bitpack v1.0.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
github.com/parquet-go/jsonlite v1.4.0 // indirect
github.com/pierrec/lz4/v4 v4.1.25 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/rivo/uniseg v0.4.7 // indirect
github.com/spf13/cobra v1.10.2 // indirect
github.com/spf13/pflag v1.0.10 // indirect
feat: add parquet, publish, metrics, convert commands - `lem parquet` — export JSONL training splits to Parquet (parquet-go) - `lem publish` — push Parquet files to HuggingFace dataset repo - `lem metrics` — push DuckDB golden set stats to InfluxDB - `lem convert` — MLX LoRA adapter → HuggingFace PEFT format (pure Go safetensors read/write/transpose, no PyTorch needed) Dependencies added: parquet-go, go-huggingface, go-rocm, go-pytorch, gotch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 17:05:08 +00:00
github.com/twpayne/go-geom v1.6.1 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
chore: sync indirect deps from workspace Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-19 13:13:08 +00:00
github.com/zeebo/xxh3 v1.1.0 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
golang.org/x/crypto v0.48.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
golang.org/x/exp v0.0.0-20260212183809-81e46e3db34a // indirect
golang.org/x/mod v0.33.0 // indirect
chore: sync indirect deps from workspace Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-19 13:13:08 +00:00
golang.org/x/sync v0.19.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
golang.org/x/sys v0.41.0 // indirect
refactor: migrate CLI imports from core/go to core/cli All imports updated from forge.lthn.ai/core/go/pkg/cli to forge.lthn.ai/core/cli/pkg/cli. core/cli is now a direct dependency; core/go becomes indirect. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 23:01:41 +00:00
golang.org/x/telemetry v0.0.0-20260213145524-e0ab670178e1 // indirect
fix(cli): disable cobra flag parsing on passthrough commands Adds passthrough() helper with DisableFlagParsing=true so commands that do their own flag.FlagSet parsing receive flags directly. Without this, cobra rejects unknown flags like --model. Also runs go mod tidy — core/go transitively pulls in cobra and charmbracelet dependencies. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 19:00:58 +00:00
golang.org/x/term v0.40.0 // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
golang.org/x/text v0.34.0 // indirect
golang.org/x/tools v0.42.0 // indirect
feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
chore: go mod tidy after distill migration go-inference moves to indirect (pulled transitively via go-ml). go-ml is now a direct dependency. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 18:02:41 +00:00
google.golang.org/protobuf v1.36.11 // indirect
feat: add Go lem CLI and scoring-agent scripts Go lem CLI (stdlib + DuckDB) replaces scattered Python scripts: - score: heuristic regex + LLM-as-judge scoring - probe: generate responses then score - compare: diff two score files - status: InfluxDB training/generation progress - export: golden set to training JSONL splits - expand: distributed expansion via API + InfluxDB coordination New scripts from Feb 14 creative session: - scoring_agent.py: ROCm daemon that auto-scores checkpoints - probes.py: 23 binary pass/fail capability probes - convert_adapter.py: MLX to PEFT adapter conversion - score_r1_capability.py: DeepSeek R1 checkpoint scoring - lek_content_scorer.py: 6-dimension ethics content scorer - lem_train_15k.py: InfluxDB-coordinated training script - pipeline.py: DuckDB pipeline (seeds, golden set, expansion) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 16:22:13 +00:00
)
Copy permalink