core/go-rocm
8
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions
AMD ROCm GPU inference for Linux. llama.cpp + HIP backend for RDNA 3.
39 commits 4 branches 2 tags 203 KiB
Find a file
Claude 61a95e4d4f
docs: Phase 4 complete — benchmarks, flash attention, parallel slots 
Gemma3-4B: 103 tok/s decode, 14ms TTFT, 239 tok/s concurrent (4 slots).
Flash attention shows no benefit at ctx=2048 (designed for larger contexts).

Co-Authored-By: Virgil <virgil@lethean.io>
2026-02-19 23:22:04 +00:00
docs/plans docs: Phase 4 performance implementation plan 2026-02-19 23:11:30 +00:00
internal feat: GGUF metadata parser for model discovery 2026-02-19 22:20:02 +00:00
backend.go feat: pass --parallel N to llama-server for concurrent inference slots 2026-02-19 23:13:19 +00:00
CLAUDE.md docs: Phase 0 complete — environment validated, llama-server built 2026-02-19 19:57:14 +00:00
discover.go feat: model discovery scanning directories for GGUF files 2026-02-19 22:21:48 +00:00
discover_test.go feat: model discovery scanning directories for GGUF files 2026-02-19 22:21:48 +00:00
FINDINGS.md docs: Phase 4 complete — benchmarks, flash attention, parallel slots 2026-02-19 23:22:04 +00:00
go.mod feat: llamacpp health check client 2026-02-19 20:50:36 +00:00
go.sum feat: llamacpp health check client 2026-02-19 20:50:36 +00:00
model.go test: graceful shutdown and concurrent request integration tests 2026-02-19 21:50:47 +00:00
README.md Initial commit 2026-02-19 19:35:55 +00:00
register_rocm.go feat: scaffold go-rocm AMD GPU inference package 2026-02-19 19:39:40 +00:00
rocm.go feat: model discovery scanning directories for GGUF files 2026-02-19 22:21:48 +00:00
rocm_benchmark_test.go feat: benchmark suite for decode speed, TTFT, and concurrent throughput 2026-02-19 23:16:40 +00:00
rocm_integration_test.go docs: Phase 3 complete — GGUF metadata, discovery, auto context 2026-02-19 22:24:52 +00:00
rocm_stub.go feat: VRAM monitoring via sysfs with dGPU auto-detection 2026-02-19 21:45:02 +00:00
server.go feat: pass --parallel N to llama-server for concurrent inference slots 2026-02-19 23:13:19 +00:00
server_test.go feat: pass --parallel N to llama-server for concurrent inference slots 2026-02-19 23:13:19 +00:00
TODO.md docs: Phase 4 complete — benchmarks, flash attention, parallel slots 2026-02-19 23:22:04 +00:00
vram.go fix: clamp VRAM Free to prevent uint64 underflow 2026-02-19 21:48:19 +00:00
vram_test.go feat: VRAM monitoring via sysfs with dGPU auto-detection 2026-02-19 21:45:02 +00:00

README.md

go-rocm

AMD ROCm GPU inference for Linux. llama.cpp + HIP backend for RDNA 3.