Native Apple Metal GPU inference via mlx-c bindings
535b04d5d6
Qwen2 and Qwen3 share the same architecture — Qwen3 adds Q/K RMS normalization which Qwen2 lacks. The loader auto-detects the variant from weight presence and reports the correct ModelType(). - Add "qwen2" to architecture dispatch in model.go - Make Q/K norm optional in attention forward (nil-safe check) - Store detected model type on Qwen3Model struct - Add "qwen2" to chat template routing - DeepSeek R1 7B (4-bit): 27 tok/s on M3 Ultra - 2 new tests: inference + chat Co-Authored-By: Virgil <virgil@lethean.io> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| cpp | ||
| docs/plans | ||
| internal/metal | ||
| .gitignore | ||
| CLAUDE.md | ||
| CMakeLists.txt | ||
| FINDINGS.md | ||
| go.mod | ||
| mlx.go | ||
| mlx_stub.go | ||
| mlx_test.go | ||
| register_metal.go | ||
| TODO.md | ||