Token counting, model quotas, and sliding window rate limiter
d74811f2d0
- Use slices.DeleteFunc in prune() for cleaner time-window filtering - Add Models() iter.Seq[string] and Iter() iter.Seq2[string, ModelStats] - Use range over int in benchmarks and tests - Update docs example to modern range syntax Co-Authored-By: Gemini <noreply@google.com> Co-Authored-By: Virgil <virgil@lethean.io> |
||
|---|---|---|
| .forgejo/workflows | ||
| docs | ||
| CLAUDE.md | ||
| go.mod | ||
| go.sum | ||
| ratelimit.go | ||
| ratelimit_test.go | ||
| README.md | ||
| sqlite.go | ||
| sqlite_test.go | ||
go-ratelimit
Provider-agnostic sliding window rate limiter for LLM API calls. Enforces requests per minute (RPM), tokens per minute (TPM), and requests per day (RPD) quotas per model using an in-memory sliding window. Ships with default quota profiles for Gemini, OpenAI, Anthropic, and a local inference provider. State persists across process restarts via YAML (single-process) or SQLite (multi-process, WAL mode). Includes a Gemini-specific token counting helper and a YAML-to-SQLite migration path.
Module: forge.lthn.ai/core/go-ratelimit
Licence: EUPL-1.2
Language: Go 1.25
Quick Start
import "forge.lthn.ai/core/go-ratelimit"
// YAML backend (default, single-process)
rl, err := ratelimit.New()
// SQLite backend (multi-process)
rl, err := ratelimit.NewWithSQLite("~/.core/ratelimits.db")
defer rl.Close()
ok, reason := rl.CanSend("gemini-2.0-flash", 1500)
if ok {
rl.RecordUsage("gemini-2.0-flash", 1500)
}
Documentation
- Architecture — sliding window algorithm, provider quotas, YAML and SQLite backends
- Development Guide — prerequisites, test patterns, coding standards
- Project History — completed phases with commit hashes, known limitations
Build & Test
go test ./...
go test -race ./...
go vet ./...
go build ./...
Licence
European Union Public Licence 1.2 — see LICENCE for details.