Native Apple Metal GPU inference via mlx-c bindings
fb95cde30c
1. RepeatPenalty: implemented applyRepeatPenalty() — tracks generated token IDs, deduplicates, divides positive logits by penalty and multiplies negative logits by penalty. 2 new tests. 2. DefaultGPUStream/DefaultCPUStream: now cached with sync.Once, no more C stream allocation on every call. 3. CompileShapeless: removed dead C closure, callback, sync.Map, and nextID infrastructure. CompiledFunc is now a plain function wrapper with mutex. API unchanged. 4. Tokenizer BPE: implemented bpeMerge() — standard BPE algorithm using merge rank lookup. Both SentencePiece and GPT-2 Encode paths now apply merges instead of falling back to character-level lookup. 3 new tests. 5. KV cache lifecycle: documented in Generate() godoc — fresh caches per call, ClearCache() between turns for prompt Metal reclaim. Co-Authored-By: Virgil <virgil@lethean.io> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| cpp | ||
| docs/plans | ||
| internal/metal | ||
| .gitignore | ||
| CLAUDE.md | ||
| CMakeLists.txt | ||
| FINDINGS.md | ||
| go.mod | ||
| mlx.go | ||
| mlx_stub.go | ||
| mlx_test.go | ||
| register_metal.go | ||
| TODO.md | ||