Merge pull request '[agent/codex] Full AX v0.8.0 compliance review. Read CODEX.md and .core/re...' (#16) from agent/upgrade-this-package-to-dappco-re-go-cor into dev
This commit is contained in:
commit
0e8d02a528
16 changed files with 109 additions and 67 deletions
11
CLAUDE.md
11
CLAUDE.md
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
|
@ -6,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|||
|
||||
Provider-agnostic sliding window rate limiter for LLM API calls. Single Go package (no sub-packages) with two persistence backends: YAML (single-process, default) and SQLite (multi-process, WAL mode). Enforces RPM, TPM, and RPD quotas per model. Ships default profiles for Gemini, OpenAI, Anthropic, and Local providers.
|
||||
|
||||
Module: `forge.lthn.ai/core/go-ratelimit` — Go 1.26, no CGO required.
|
||||
Module: `dappco.re/go/core/go-ratelimit` — Go 1.26, no CGO required.
|
||||
|
||||
## Commands
|
||||
|
||||
|
|
@ -28,7 +30,7 @@ Pre-commit gate: `go test -race ./...` and `go vet ./...` must both pass.
|
|||
- **Conventional commits**: `type(scope): description` — scopes: `ratelimit`, `sqlite`, `persist`, `config`
|
||||
- **Co-Author line** on every commit: `Co-Authored-By: Virgil <virgil@lethean.io>`
|
||||
- **Coverage** must not drop below 95%
|
||||
- **Error format**: `coreerr.E("ratelimit.FunctionName", "what", err)` via `go-log` — lowercase, no trailing punctuation
|
||||
- **Error format**: `core.E("ratelimit.FunctionName", "what", err)` via `dappco.re/go/core` — lowercase, no trailing punctuation
|
||||
- **No `init()` functions**, no global mutable state
|
||||
- **Mutex discipline**: lock at the top of public methods, never inside helpers. Helpers that need the lock document "Caller must hold the lock". `prune()` mutates state, so even "read-only" methods that call it take the write lock. Never call a public method from another public method while holding the lock.
|
||||
|
||||
|
|
@ -60,10 +62,9 @@ SQLite tests use `_Good`/`_Bad`/`_Ugly` suffixes (happy path / expected errors /
|
|||
|
||||
## Dependencies
|
||||
|
||||
Five direct dependencies — do not add more without justification:
|
||||
Four direct dependencies — do not add more without justification:
|
||||
|
||||
- `forge.lthn.ai/core/go-io` — file I/O abstraction
|
||||
- `forge.lthn.ai/core/go-log` — structured error handling (`coreerr.E`)
|
||||
- `dappco.re/go/core` — file I/O helpers, structured errors, JSON helpers, path/environment utilities
|
||||
- `gopkg.in/yaml.v3` — YAML backend
|
||||
- `modernc.org/sqlite` — pure Go SQLite (no CGO)
|
||||
- `github.com/stretchr/testify` — test-only
|
||||
|
|
|
|||
|
|
@ -1,15 +1,21 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
# Contributing
|
||||
|
||||
Thank you for your interest in contributing!
|
||||
|
||||
## Requirements
|
||||
- **Go Version**: 1.26 or higher is required.
|
||||
- **Tools**: `golangci-lint` and `task` (Taskfile.dev) are recommended.
|
||||
- **Tools**: `golangci-lint` is recommended.
|
||||
|
||||
## Development Workflow
|
||||
1. **Testing**: Ensure all tests pass before submitting changes.
|
||||
```bash
|
||||
go build ./...
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go test -cover ./...
|
||||
go mod tidy
|
||||
```
|
||||
2. **Code Style**: All code must follow standard Go formatting.
|
||||
```bash
|
||||
|
|
@ -22,14 +28,22 @@ Thank you for your interest in contributing!
|
|||
```
|
||||
|
||||
## Commit Message Format
|
||||
We follow the [Conventional Commits](https://www.conventionalcommits.org/) specification:
|
||||
We follow the [Conventional Commits](https://www.conventionalcommits.org/) specification using the repository format `type(scope): description`:
|
||||
- `feat`: A new feature
|
||||
- `fix`: A bug fix
|
||||
- `docs`: Documentation changes
|
||||
- `refactor`: A code change that neither fixes a bug nor adds a feature
|
||||
- `chore`: Changes to the build process or auxiliary tools and libraries
|
||||
|
||||
Example: `feat: add new endpoint for health check`
|
||||
Common scopes: `ratelimit`, `sqlite`, `persist`, `config`
|
||||
|
||||
## License
|
||||
Example:
|
||||
|
||||
```text
|
||||
fix(ratelimit): align module metadata with dappco.re
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
```
|
||||
|
||||
## Licence
|
||||
By contributing to this project, you agree that your contributions will be licensed under the **European Union Public Licence (EUPL-1.2)**.
|
||||
|
|
|
|||
35
README.md
35
README.md
|
|
@ -1,30 +1,41 @@
|
|||
[](https://pkg.go.dev/forge.lthn.ai/core/go-ratelimit)
|
||||
[](LICENSE.md)
|
||||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
[](https://pkg.go.dev/dappco.re/go/core/go-ratelimit)
|
||||
|
||||
[](go.mod)
|
||||
|
||||
# go-ratelimit
|
||||
|
||||
Provider-agnostic sliding window rate limiter for LLM API calls. Enforces requests per minute (RPM), tokens per minute (TPM), and requests per day (RPD) quotas per model using an in-memory sliding window. Ships with default quota profiles for Gemini, OpenAI, Anthropic, and a local inference provider. State persists across process restarts via YAML (single-process) or SQLite (multi-process, WAL mode). Includes a Gemini-specific token counting helper and a YAML-to-SQLite migration path.
|
||||
|
||||
**Module**: `forge.lthn.ai/core/go-ratelimit`
|
||||
**Module**: `dappco.re/go/core/go-ratelimit`
|
||||
**Licence**: EUPL-1.2
|
||||
**Language**: Go 1.25
|
||||
**Language**: Go 1.26
|
||||
|
||||
## Quick Start
|
||||
|
||||
```go
|
||||
import "forge.lthn.ai/core/go-ratelimit"
|
||||
import "dappco.re/go/core/go-ratelimit"
|
||||
|
||||
// YAML backend (default, single-process)
|
||||
rl, err := ratelimit.New()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// SQLite backend (multi-process)
|
||||
rl, err := ratelimit.NewWithSQLite("~/.core/ratelimits.db")
|
||||
rl, err = ratelimit.NewWithSQLite("/tmp/ratelimits.db")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer rl.Close()
|
||||
|
||||
ok, reason := rl.CanSend("gemini-2.0-flash", 1500)
|
||||
if ok {
|
||||
rl.RecordUsage("gemini-2.0-flash", 1500)
|
||||
if rl.CanSend("gemini-2.0-flash", 1500) {
|
||||
rl.RecordUsage("gemini-2.0-flash", 1000, 500)
|
||||
}
|
||||
|
||||
if err := rl.Persist(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -37,12 +48,14 @@ if ok {
|
|||
## Build & Test
|
||||
|
||||
```bash
|
||||
go build ./...
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
go test -cover ./...
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
## Licence
|
||||
|
||||
European Union Public Licence 1.2 — see [LICENCE](LICENCE) for details.
|
||||
European Union Public Licence 1.2.
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
# API Contract
|
||||
|
||||
Test coverage is marked `yes` when the symbol is exercised by the existing test suite in `ratelimit_test.go`, `sqlite_test.go`, `error_test.go`, or `iter_test.go`.
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
---
|
||||
title: Architecture
|
||||
description: Internals of go-ratelimit -- sliding window algorithm, provider quota system, persistence backends, and concurrency model.
|
||||
|
|
@ -10,7 +12,7 @@ three independent quota dimensions per model -- requests per minute (RPM), token
|
|||
per minute (TPM), and requests per day (RPD) -- using an in-memory sliding window
|
||||
that can be persisted across process restarts via YAML or SQLite.
|
||||
|
||||
Module path: `forge.lthn.ai/core/go-ratelimit`
|
||||
Module path: `dappco.re/go/core/go-ratelimit`
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -252,7 +254,7 @@ state:
|
|||
day_count: 42
|
||||
```
|
||||
|
||||
`Persist()` creates parent directories with `os.MkdirAll` before writing.
|
||||
`Persist()` creates parent directories with the `core.Fs` helper before writing.
|
||||
`Load()` treats a missing file as an empty state (no error). Corrupt or
|
||||
unreadable files return an error.
|
||||
|
||||
|
|
@ -317,8 +319,8 @@ precision and allows efficient range queries using the composite indices.
|
|||
|
||||
### Save Strategy
|
||||
|
||||
- **Quotas**: `INSERT ... ON CONFLICT(model) DO UPDATE` (upsert). Existing quota
|
||||
rows are updated in place without deleting unrelated models.
|
||||
- **Quotas**: full snapshot replace inside a single transaction. `saveQuotas()`
|
||||
clears the table and reinserts the current quota map.
|
||||
- **State**: Delete-then-insert inside a single transaction. All three state
|
||||
tables (`requests`, `tokens`, `daily`) are truncated and rewritten atomically.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
---
|
||||
title: Development Guide
|
||||
description: How to build, test, and contribute to go-ratelimit -- prerequisites, test patterns, coding standards, and commit conventions.
|
||||
|
|
@ -18,6 +20,9 @@ No C toolchain, no system SQLite library, no external build tools. A plain
|
|||
## Build and Test
|
||||
|
||||
```bash
|
||||
# Compile all packages
|
||||
go build ./...
|
||||
|
||||
# Run all tests
|
||||
go test ./...
|
||||
|
||||
|
|
@ -42,12 +47,16 @@ go vet ./...
|
|||
# Lint (requires golangci-lint)
|
||||
golangci-lint run ./...
|
||||
|
||||
# Coverage check
|
||||
go test -cover ./...
|
||||
|
||||
# Tidy dependencies
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
All three commands (`go test -race ./...`, `go vet ./...`, and `go mod tidy`)
|
||||
must produce no errors or warnings before a commit is pushed.
|
||||
Before a commit is pushed, `go build ./...`, `go test -race ./...`,
|
||||
`go vet ./...`, `go test -cover ./...`, and `go mod tidy` must all pass
|
||||
without errors, and coverage must remain at or above 95%.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -147,17 +156,8 @@ The following benchmarks are included:
|
|||
|
||||
### Coverage
|
||||
|
||||
Current coverage: 95.1%. The remaining paths cannot be covered in unit tests
|
||||
without modifying production code:
|
||||
|
||||
1. `CountTokens` success path -- the Google API URL is hardcoded; unit tests
|
||||
cannot intercept the HTTP call without URL injection support.
|
||||
2. `yaml.Marshal` error path in `Persist()` -- `yaml.Marshal` does not fail on
|
||||
valid Go structs.
|
||||
3. `os.UserHomeDir()` error path in `NewWithConfig()` -- triggered only when
|
||||
`$HOME` is unset, which test infrastructure prevents.
|
||||
|
||||
Do not lower coverage below 95% without a documented reason.
|
||||
Maintain at least 95% statement coverage. Verify it with `go test -cover ./...`
|
||||
and document any justified exception in the commit or PR that introduces it.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -172,8 +172,8 @@ Do not use American spellings in identifiers, comments, or documentation.
|
|||
|
||||
- All exported types, functions, and fields must have doc comments.
|
||||
- Error strings must be lowercase and not end with punctuation (Go convention).
|
||||
- Contextual errors use `fmt.Errorf("ratelimit.Function: what: %w", err)` so
|
||||
errors identify their origin clearly.
|
||||
- Contextual errors use `core.E("ratelimit.Function", "what", err)` so errors
|
||||
identify their origin clearly.
|
||||
- No `init()` functions.
|
||||
- No global mutable state. `DefaultProfiles()` returns a fresh map on each call.
|
||||
|
||||
|
|
@ -196,6 +196,7 @@ Direct dependencies are intentionally minimal:
|
|||
|
||||
| Dependency | Purpose |
|
||||
|------------|---------|
|
||||
| `dappco.re/go/core` | File I/O helpers, structured errors, JSON helpers, path/environment utilities |
|
||||
| `gopkg.in/yaml.v3` | YAML serialisation for legacy backend |
|
||||
| `modernc.org/sqlite` | Pure Go SQLite for persistent backend |
|
||||
| `github.com/stretchr/testify` | Test assertions (test-only) |
|
||||
|
|
|
|||
|
|
@ -1,10 +1,13 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
# Project History
|
||||
|
||||
## Origin
|
||||
|
||||
go-ratelimit was extracted from the `pkg/ratelimit` package inside
|
||||
`forge.lthn.ai/core/go` on 19 February 2026. The extraction gave the package
|
||||
its own module path, repository, and independent development cadence.
|
||||
`forge.lthn.ai/core/go` on 19 February 2026. The package now lives at
|
||||
`dappco.re/go/core/go-ratelimit`, with its own repository and independent
|
||||
development cadence.
|
||||
|
||||
Initial commit: `fa1a6fc` — `feat: extract go-ratelimit from core/go pkg/ratelimit`
|
||||
|
||||
|
|
@ -25,7 +28,7 @@ Commit: `3c63b10` — `feat(ratelimit): generalise beyond Gemini with provider p
|
|||
|
||||
Supplementary commit: `db958f2` — `test: expand race coverage and benchmarks`
|
||||
|
||||
Coverage increased from 77.1% to 95.1%. The test suite was rewritten using
|
||||
Coverage increased from 77.1% to above the 95% floor. The test suite was rewritten using
|
||||
testify with table-driven subtests throughout.
|
||||
|
||||
### Tests added
|
||||
|
|
@ -58,18 +61,6 @@ testify with table-driven subtests throughout.
|
|||
- `BenchmarkAllStats` — 5 models x 200 entries
|
||||
- `BenchmarkPersist` — YAML I/O
|
||||
|
||||
### Remaining uncovered paths (5%)
|
||||
|
||||
These three paths are structurally impossible to cover in unit tests without
|
||||
modifying production code:
|
||||
|
||||
1. `CountTokens` success path — the Google API URL is hardcoded; unit tests
|
||||
cannot intercept the HTTP call without URL injection support
|
||||
2. `yaml.Marshal` error path in `Persist()` — `yaml.Marshal` does not fail on
|
||||
valid Go structs; the error branch exists for correctness only
|
||||
3. `os.UserHomeDir()` error path in `NewWithConfig()` — triggered only when
|
||||
`$HOME` is unset, which test infrastructure prevents
|
||||
|
||||
`go test -race ./...` passed clean. `go vet ./...` produced no warnings.
|
||||
|
||||
---
|
||||
|
|
@ -139,7 +130,7 @@ established elsewhere in the ecosystem.
|
|||
|
||||
- `TestNewSQLiteStore_Good / _Bad` — creation and invalid path handling
|
||||
- `TestSQLiteQuotasRoundTrip_Good` — save/load round-trip
|
||||
- `TestSQLiteQuotasUpsert_Good` — upsert replaces existing rows
|
||||
- `TestSQLite_QuotasOverwrite_Good` — the latest quota snapshot replaces previous rows
|
||||
- `TestSQLiteStateRoundTrip_Good` — multi-model state with nanosecond precision
|
||||
- `TestSQLiteStateOverwrite_Good` — delete-then-insert atomicity
|
||||
- `TestSQLiteEmptyState_Good` — fresh database returns empty maps
|
||||
|
|
@ -168,11 +159,10 @@ Not yet implemented. Intended downstream integrations:
|
|||
|
||||
## Known Limitations
|
||||
|
||||
**CountTokens URL is hardcoded.** The `CountTokens` helper calls
|
||||
`generativelanguage.googleapis.com` directly. There is no way to override the
|
||||
base URL, which prevents testing the success path in unit tests and prevents
|
||||
use with Gemini-compatible proxies. A future refactor would accept a base URL
|
||||
parameter or an `http.Client`.
|
||||
**CountTokens URL is hardcoded.** The exported `CountTokens` helper calls
|
||||
`generativelanguage.googleapis.com` directly. Callers cannot redirect it to
|
||||
Gemini-compatible proxies or alternate endpoints without going through an
|
||||
internal helper or refactoring the API to accept a base URL or `http.Client`.
|
||||
|
||||
**saveState is a full table replace.** On every `Persist()` call, the `requests`,
|
||||
`tokens`, and `daily` tables are truncated and rewritten. For a limiter tracking
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
---
|
||||
title: go-ratelimit
|
||||
description: Provider-agnostic sliding window rate limiter for LLM API calls, with YAML and SQLite persistence backends.
|
||||
|
|
@ -5,7 +7,7 @@ description: Provider-agnostic sliding window rate limiter for LLM API calls, wi
|
|||
|
||||
# go-ratelimit
|
||||
|
||||
**Module**: `forge.lthn.ai/core/go-ratelimit`
|
||||
**Module**: `dappco.re/go/core/go-ratelimit`
|
||||
**Licence**: EUPL-1.2
|
||||
**Go version**: 1.26+
|
||||
|
||||
|
|
@ -19,7 +21,7 @@ migration helper is included.
|
|||
## Quick Start
|
||||
|
||||
```go
|
||||
import "forge.lthn.ai/core/go-ratelimit"
|
||||
import "dappco.re/go/core/go-ratelimit"
|
||||
|
||||
// Create a limiter with Gemini defaults (YAML backend).
|
||||
rl, err := ratelimit.New()
|
||||
|
|
@ -103,6 +105,7 @@ The module is a single package with no sub-packages.
|
|||
|
||||
| Dependency | Purpose | Category |
|
||||
|------------|---------|----------|
|
||||
| `dappco.re/go/core` | File I/O helpers, structured errors, JSON helpers, path/environment utilities | Direct |
|
||||
| `gopkg.in/yaml.v3` | YAML serialisation for the legacy persistence backend | Direct |
|
||||
| `modernc.org/sqlite` | Pure Go SQLite driver (no CGO required) | Direct |
|
||||
| `github.com/stretchr/testify` | Test assertions (`assert`, `require`) | Test only |
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
||||
|
||||
# Security Attack Vector Mapping
|
||||
|
||||
Scope: external inputs that cross into this package from callers, persisted storage, or the network. This is a mapping only; it does not propose or apply fixes.
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
@ -647,7 +649,7 @@ func TestError_MigrateYAMLToSQLiteNilQuotasAndState_Good(t *testing.T) {
|
|||
})
|
||||
}
|
||||
|
||||
func TestError_NewWithConfigUserHomeDir_Bad(t *testing.T) {
|
||||
func TestError_NewWithConfigHomeUnavailable_Bad(t *testing.T) {
|
||||
// Clear all supported home env vars so defaultStatePath cannot resolve a home directory.
|
||||
t.Setenv("CORE_HOME", "")
|
||||
t.Setenv("HOME", "")
|
||||
|
|
|
|||
4
go.mod
4
go.mod
|
|
@ -1,4 +1,6 @@
|
|||
module forge.lthn.ai/core/go-ratelimit
|
||||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
module dappco.re/go/core/go-ratelimit
|
||||
|
||||
go 1.26.0
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
// SPDX-License-Identifier: EUPL-1.2
|
||||
|
||||
package ratelimit
|
||||
|
||||
import (
|
||||
|
|
@ -57,8 +59,8 @@ func TestSQLite_QuotasRoundTrip_Good(t *testing.T) {
|
|||
}
|
||||
}
|
||||
|
||||
func TestSQLite_QuotasUpsert_Good(t *testing.T) {
|
||||
dbPath := testPath(t.TempDir(), "upsert.db")
|
||||
func TestSQLite_QuotasOverwrite_Good(t *testing.T) {
|
||||
dbPath := testPath(t.TempDir(), "overwrite.db")
|
||||
store, err := newSQLiteStore(dbPath)
|
||||
require.NoError(t, err)
|
||||
defer store.close()
|
||||
|
|
@ -68,7 +70,7 @@ func TestSQLite_QuotasUpsert_Good(t *testing.T) {
|
|||
"model-a": {MaxRPM: 100, MaxTPM: 50000, MaxRPD: 1000},
|
||||
}))
|
||||
|
||||
// Upsert with updated values.
|
||||
// Save a second snapshot with updated values.
|
||||
require.NoError(t, store.saveQuotas(map[string]ModelQuota{
|
||||
"model-a": {MaxRPM: 999, MaxTPM: 888, MaxRPD: 777},
|
||||
}))
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue