core/go-mlx
8
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
Native Apple Metal GPU inference via mlx-c bindings
33 commits 1 branch 4 tags 1.2 MiB
Find a file
Snider 18e8dca9f8 feat(metal): validate Gemma3-1B inference end-to-end (Phase 2) 
- Fix model_type "gemma3_text" not matched in architecture dispatch
- Fix GPT-2 BPE false detection on large SentencePiece vocabs (Gemma3
  262K vocab contains Ġ but uses ▁ for spaces — check "Ġthe" not bare "Ġ")
- Add TestGemma3_1B_Inference: greedy decode, 46 tok/s, coherent output
- Add TestGemma3_1B_Chat: validates chat template formatting
- Add TestGemma3_1B_ContextCancel: validates ctx.Done() stops generation

4-bit quantised Gemma3-1B loads in ~700ms, generates at 46 tok/s on M3 Ultra.

Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 21:44:28 +00:00
cpp fix(metal): address 4 minor code review items 2026-02-19 21:36:40 +00:00
docs/plans fix(metal): address 4 minor code review items 2026-02-19 21:36:40 +00:00
internal/metal feat(metal): validate Gemma3-1B inference end-to-end (Phase 2) 2026-02-19 21:44:28 +00:00
.gitignore chore: gitignore dist/ (CMake install output) 2026-02-19 19:30:23 +00:00
CLAUDE.md feat(api): migrate to go-inference shared interfaces 2026-02-19 20:15:42 +00:00
CMakeLists.txt feat: extract go-mlx from go-ai as standalone Metal inference package 2026-02-19 17:57:37 +00:00
FINDINGS.md fix(metal): address 4 minor code review items 2026-02-19 21:36:40 +00:00
go.mod feat(api): migrate to go-inference shared interfaces 2026-02-19 20:15:42 +00:00
mlx.go feat(api): migrate to go-inference shared interfaces 2026-02-19 20:15:42 +00:00
mlx_stub.go feat: extract go-mlx from go-ai as standalone Metal inference package 2026-02-19 17:57:37 +00:00
mlx_test.go feat(metal): validate Gemma3-1B inference end-to-end (Phase 2) 2026-02-19 21:44:28 +00:00
register_metal.go fix(metal): address 3 critical code review items 2026-02-19 21:24:10 +00:00
TODO.md feat(metal): validate Gemma3-1B inference end-to-end (Phase 2) 2026-02-19 21:44:28 +00:00