Commit graph

4 commits

Author SHA1 Message Date
Snider
45e9fed280 feat: add InspectAttention pass-through on InferenceAdapter
Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-23 00:37:37 +00:00
Snider
d90bd7b779 feat(adapter): return Result with Metrics from TextModel
InferenceAdapter.Generate and Chat now return Result{Text, Metrics}
where Metrics is populated from the underlying TextModel.Metrics().

Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-22 17:36:53 +00:00
Snider
747e703c7b feat: Phase 2 backend consolidation — Message alias, GenOpts, deprecation
- Replace Message struct with type alias for inference.Message (backward compat)
- Remove convertMessages() — types are now identical via alias
- Extend GenOpts with TopK, TopP, RepeatPenalty (mapped in convertOpts)
- Deprecate StreamingBackend with doc comment (only 2 callers, both in cli/)
- Simplify HTTPTextModel.Chat() — pass messages directly
- Update CLAUDE.md with Backend Architecture section
- Add 2 new tests, remove 1 obsolete test

Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-20 02:05:59 +00:00
Snider
a4d7686147 feat(adapter): bridge go-inference TextModel to ml.Backend/StreamingBackend
InferenceAdapter wraps inference.TextModel (iter.Seq[Token]) to satisfy
ml.Backend (string returns) and ml.StreamingBackend (TokenCallback).

- adapter.go: InferenceAdapter with Generate/Chat/Stream/Close
- adapter_test.go: 13 test cases with mock TextModel (all pass)
- backend_mlx.go: rewritten from 253 LOC to ~35 LOC using go-inference
- go.mod: add forge.lthn.ai/core/go-inference dependency
- TODO.md: mark Phase 1 steps 1.1-1.3 complete

Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-20 00:52:34 +00:00