Native Apple Metal GPU inference via mlx-c bindings
5644857034
- Add ForwardMasked to InternalModel, Gemma3 and Qwen3 architectures - Thread attention mask through decoder layers and SDPA calls - Use ScaledDotProductAttentionWithMask when explicit mask provided - Create batch.go with padded batching, mask construction, Classify (prefill-only) and BatchGenerate (autoregressive) implementations - Wire Classify/BatchGenerate through metalAdapter to go-inference - Tests: mask unit tests (shape, values, multi-batch), Classify with 4 prompts (152 prompts/s), WithLogits, BatchGenerate with 2 prompts Co-Authored-By: Virgil <virgil@lethean.io> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| cpp | ||
| docs/plans | ||
| internal/metal | ||
| .gitignore | ||
| CLAUDE.md | ||
| CMakeLists.txt | ||
| FINDINGS.md | ||
| go.mod | ||
| mlx.go | ||
| mlx_stub.go | ||
| mlx_test.go | ||
| register_metal.go | ||
| TODO.md | ||