core/go-inference
8
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Shared inference interfaces (TextModel, Backend, Token). Zero deps, compiles everywhere.
5 commits 1 branch 4 tags 146 KiB
Find a file
Snider df176765e7 feat: add GenerateMetrics type and Metrics() to TextModel 
Expose prefill/decode timing, token counts, throughput, and GPU memory
stats from the last inference operation. Same retrieval pattern as Err().

Co-Authored-By: Virgil <virgil@lethean.io>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-19 23:34:31 +00:00
CLAUDE.md feat: define shared TextModel, Backend, Token, Message interfaces 2026-02-19 19:37:27 +00:00
FINDINGS.md feat: define shared TextModel, Backend, Token, Message interfaces 2026-02-19 19:37:27 +00:00
go.mod feat: define shared TextModel, Backend, Token, Message interfaces 2026-02-19 19:37:27 +00:00
inference.go feat: add GenerateMetrics type and Metrics() to TextModel 2026-02-19 23:34:31 +00:00
options.go feat: add batch inference API (Classify, BatchGenerate) 2026-02-19 23:29:28 +00:00
README.md Initial commit 2026-02-19 19:35:54 +00:00
TODO.md feat: define shared TextModel, Backend, Token, Message interfaces 2026-02-19 19:37:27 +00:00

README.md

go-inference

Shared inference interfaces (TextModel, Backend, Token). Zero deps, compiles everywhere.