go/pkg/ml
Claude fcd1758b7d
Some checks are pending
Security Scan / Go Vulnerability Check (push) Waiting to run
Security Scan / Secret Detection (push) Waiting to run
Security Scan / Dependency & Config Scan (push) Waiting to run
feat(ml): add format converters, data pipeline, and scoring agent
Port remaining lem-repo components into pkg/ml/:
- convert.go: safetensors reader/writer, MLX→PEFT converter
- gguf.go: GGUF v3 writer, MLX→GGUF LoRA converter
- export.go: training data JSONL export with split/filter
- parquet.go: Parquet export with snappy compression
- db.go: DuckDB wrapper for golden set and expansion prompts
- influx.go: InfluxDB v3 client for metrics/status
- ollama.go: Ollama model management (create/delete with adapters)
- status.go: training and generation status display
- expand.go: expansion generation pipeline (Backend interface)
- agent.go: scoring agent with probe running and InfluxDB push
- worker.go: distributed worker for LEM API task processing

Adds parquet-go and go-duckdb dependencies.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 23:46:24 +00:00
..
agent.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
backend_http.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
backend_http_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
backend_llama.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
compare.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
convert.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
db.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
exact.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
exact_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
expand.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
export.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
gguf.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
heuristic.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
heuristic_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
inference.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
influx.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
io.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
judge.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
judge_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
ollama.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
parquet.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
probes.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
probes_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
prompts.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
score.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
score_test.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
status.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00
types.go feat(ml): add ML inference and scoring engine from lem-repo 2026-02-15 23:34:54 +00:00
worker.go feat(ml): add format converters, data pipeline, and scoring agent 2026-02-15 23:46:24 +00:00