Token counting, model quotas, and sliding window rate limiter

Find a file

Snider 0afcf0fb64 chore: bump go directive to 1.26.0 Co-Authored-By: Virgil <virgil@lethean.io>		2026-02-22 20:33:48 +00:00
docs	docs: graduate TODO/FINDINGS into production documentation	2026-02-20 15:01:55 +00:00
CLAUDE.md	docs: graduate TODO/FINDINGS into production documentation	2026-02-20 15:01:55 +00:00
go.mod	chore: bump go directive to 1.26.0	2026-02-22 20:33:48 +00:00
go.sum	feat(persist): Phase 2 — SQLite backend with WAL mode	2026-02-20 07:50:48 +00:00
ratelimit.go	feat(persist): Phase 2 — SQLite backend with WAL mode	2026-02-20 07:50:48 +00:00
ratelimit_test.go	test: expand race coverage and benchmarks	2026-02-20 04:30:54 +00:00
README.md	docs: add README with quick start and docs links	2026-02-20 15:11:19 +00:00
sqlite.go	feat(persist): Phase 2 — SQLite backend with WAL mode	2026-02-20 07:50:48 +00:00
sqlite_test.go	feat(persist): Phase 2 — SQLite backend with WAL mode	2026-02-20 07:50:48 +00:00

README.md

go-ratelimit

Provider-agnostic sliding window rate limiter for LLM API calls. Enforces requests per minute (RPM), tokens per minute (TPM), and requests per day (RPD) quotas per model using an in-memory sliding window. Ships with default quota profiles for Gemini, OpenAI, Anthropic, and a local inference provider. State persists across process restarts via YAML (single-process) or SQLite (multi-process, WAL mode). Includes a Gemini-specific token counting helper and a YAML-to-SQLite migration path.

Module: forge.lthn.ai/core/go-ratelimit Licence: EUPL-1.2 Language: Go 1.25

Quick Start

import "forge.lthn.ai/core/go-ratelimit"

// YAML backend (default, single-process)
rl, err := ratelimit.New()

// SQLite backend (multi-process)
rl, err := ratelimit.NewWithSQLite("~/.core/ratelimits.db")
defer rl.Close()

ok, reason := rl.CanSend("gemini-2.0-flash", 1500)
if ok {
    rl.RecordUsage("gemini-2.0-flash", 1500)
}

Documentation

Architecture — sliding window algorithm, provider quotas, YAML and SQLite backends
Development Guide — prerequisites, test patterns, coding standards
Project History — completed phases with commit hashes, known limitations

Build & Test

go test ./...
go test -race ./...
go vet ./...
go build ./...

Licence

European Union Public Licence 1.2 — see LICENCE for details.