c570f08eba
Replace internal task tracking (TODO.md, FINDINGS.md) with structured documentation in docs/. Trim CLAUDE.md to agent instructions only. Co-Authored-By: Virgil <virgil@lethean.io>
218 lines
10 KiB
Markdown
218 lines
10 KiB
Markdown
# Architecture — go-store
|
|
|
|
Module: `forge.lthn.ai/core/go-store`
|
|
|
|
## Overview
|
|
|
|
go-store is a group-namespaced key-value store backed by SQLite. It provides persistent or in-memory storage with optional TTL expiry, namespace isolation for multi-tenant use, and a reactive event system for observing mutations.
|
|
|
|
The package has no external runtime dependencies beyond a pure-Go SQLite driver (`modernc.org/sqlite`). It requires no CGO and compiles on all platforms.
|
|
|
|
## Storage Layer
|
|
|
|
### SQLite with WAL Mode
|
|
|
|
Every `Store` instance opens a single SQLite database and immediately applies two pragmas:
|
|
|
|
```sql
|
|
PRAGMA journal_mode=WAL;
|
|
PRAGMA busy_timeout=5000;
|
|
```
|
|
|
|
WAL (Write-Ahead Logging) mode allows concurrent readers to proceed without blocking writers. The `busy_timeout` of 5000 milliseconds causes the driver to wait and retry rather than immediately returning `SQLITE_BUSY` under write contention.
|
|
|
|
**Single connection constraint.** The `database/sql` package maintains a connection pool by default. SQLite pragmas are per-connection: if the pool hands out a second connection, that connection inherits none of the WAL or busy-timeout settings, causing `SQLITE_BUSY` errors under concurrent load. go-store calls `db.SetMaxOpenConns(1)` to pin all access to a single connection. Since SQLite serialises writes at the file level regardless, this introduces no additional throughput penalty.
|
|
|
|
### Schema
|
|
|
|
```sql
|
|
CREATE TABLE IF NOT EXISTS kv (
|
|
grp TEXT NOT NULL,
|
|
key TEXT NOT NULL,
|
|
value TEXT NOT NULL,
|
|
expires_at INTEGER,
|
|
PRIMARY KEY (grp, key)
|
|
)
|
|
```
|
|
|
|
The compound primary key `(grp, key)` enforces uniqueness per group-key pair and provides efficient indexed lookups. The `expires_at` column stores a Unix millisecond timestamp (nullable); a `NULL` value means the key never expires.
|
|
|
|
**Schema migration.** Databases created before TTL support lacked the `expires_at` column. On `New()`, go-store runs `ALTER TABLE kv ADD COLUMN expires_at INTEGER`. If the column already exists, SQLite returns a "duplicate column" error which is silently ignored. This allows seamless upgrades of existing databases.
|
|
|
|
## Group/Key Model
|
|
|
|
Keys are addressed by a two-level path: `(group, key)`. Groups act as logical namespaces within a single database. Groups are implicit — they exist as a consequence of the keys they contain and are destroyed automatically when all their keys are deleted.
|
|
|
|
This model maps naturally to domain concepts:
|
|
|
|
```
|
|
group: "user:42:config" key: "theme"
|
|
group: "user:42:config" key: "language"
|
|
group: "session:abc" key: "token"
|
|
```
|
|
|
|
All read operations (`Get`, `GetAll`, `Count`, `Render`) are scoped to a single group. `DeleteGroup` atomically removes all keys in a group. `CountAll` and `Groups` operate across groups by prefix match.
|
|
|
|
## UPSERT Semantics
|
|
|
|
All writes use `INSERT ... ON CONFLICT(grp, key) DO UPDATE`. This means:
|
|
|
|
- Inserting a new key creates it.
|
|
- Inserting an existing key overwrites its value and (for `Set`) clears any TTL.
|
|
- UPSERT never duplicates a key.
|
|
- The operation is idempotent with respect to row count.
|
|
|
|
`Set` clears `expires_at` on upsert by setting it to `NULL`. `SetWithTTL` refreshes the expiry timestamp on upsert.
|
|
|
|
## TTL Expiry
|
|
|
|
Keys may be created with a time-to-live via `SetWithTTL`. Expiry is stored as a Unix millisecond timestamp in `expires_at`.
|
|
|
|
**Expiry is enforced in three ways:**
|
|
|
|
1. **Lazy deletion on `Get`.** If a key is found but its `expires_at` is in the past, it is deleted synchronously before returning `ErrNotFound`. This prevents stale values from being returned even if the background purge has not run yet.
|
|
|
|
2. **Query-time filtering.** All bulk operations (`GetAll`, `Count`, `Render`, `CountAll`, `Groups`) include `(expires_at IS NULL OR expires_at > ?)` in their `WHERE` clause. Expired keys are excluded from results without being deleted.
|
|
|
|
3. **Background purge goroutine.** `New()` launches a goroutine that calls `PurgeExpired()` every 60 seconds (configurable via `s.purgeInterval`). This recovers disk space by physically removing expired rows. The goroutine is stopped cleanly by `Close()` via `context.WithCancel`.
|
|
|
|
`PurgeExpired()` is also a public method for applications that want manual control over purge timing.
|
|
|
|
## Template Rendering
|
|
|
|
`Render(tmplStr, group string)` is a convenience method that fetches all non-expired key-value pairs from a group and renders a Go `text/template` against them. The template data is a `map[string]string` keyed by the field name.
|
|
|
|
Example:
|
|
|
|
```go
|
|
st.Set("miner", "pool", "pool.lthn.io:3333")
|
|
st.Set("miner", "wallet", "iz...")
|
|
out, _ := st.Render(`{"pool":"{{ .pool }}","wallet":"{{ .wallet }}"}`, "miner")
|
|
// out: {"pool":"pool.lthn.io:3333","wallet":"iz..."}
|
|
```
|
|
|
|
Template parse errors and execution errors are both returned as wrapped errors with context (e.g., `store.Render: parse: ...` and `store.Render: exec: ...`).
|
|
|
|
Missing template variables do not return an error by default — Go's `text/template` renders them as `<no value>`. Applications requiring strict variable presence should set `Option("missingkey=error")` on their own template instances before calling `Render`, or validate data beforehand.
|
|
|
|
## Watch/Unwatch Pattern
|
|
|
|
`Watch(group, key string) *Watcher` registers a subscription that receives events as they occur. Each `Watcher` holds a buffered channel (`Ch <-chan Event`) with capacity 16.
|
|
|
|
**Filter semantics:**
|
|
|
|
| group argument | key argument | Receives |
|
|
|---|---|---|
|
|
| `"mygroup"` | `"mykey"` | Only mutations to that exact key |
|
|
| `"mygroup"` | `"*"` | All mutations within the group, including `DeleteGroup` |
|
|
| `"*"` | `"*"` | Every mutation across the entire store |
|
|
|
|
`Unwatch(w *Watcher)` removes the watcher from the registry and closes its channel. It is safe to call multiple times — subsequent calls are no-ops.
|
|
|
|
**Backpressure.** Event dispatch to a watcher channel is non-blocking: if the channel buffer is full, the event is dropped silently. This prevents a slow consumer from blocking a writer. Applications that cannot afford dropped events should drain the channel promptly or use `OnChange` callbacks instead.
|
|
|
|
**Idiomatic usage:**
|
|
|
|
```go
|
|
w := st.Watch("config", "*")
|
|
defer st.Unwatch(w)
|
|
|
|
for e := range w.Ch {
|
|
fmt.Println(e.Type, e.Group, e.Key, e.Value)
|
|
}
|
|
```
|
|
|
|
## OnChange Callbacks
|
|
|
|
`OnChange(fn func(Event)) func()` registers a synchronous callback that fires on every mutation. The callback runs in the goroutine that performed the write, holding the watcher/callback read-lock. Callers must not call store methods from within a callback (deadlock risk) and should offload any significant work to a goroutine.
|
|
|
|
`OnChange` returns an unregister function. Calling it removes the callback from the registry. The unregister function is idempotent.
|
|
|
|
This is the designed integration point for go-ws:
|
|
|
|
```go
|
|
unreg := st.OnChange(func(e store.Event) {
|
|
hub.SendToChannel("store-events", e)
|
|
})
|
|
defer unreg()
|
|
```
|
|
|
|
go-store does not import go-ws. The dependency flows in one direction only: go-ws (or any consumer) imports go-store.
|
|
|
|
## Event Model
|
|
|
|
Events are defined in `events.go`:
|
|
|
|
```go
|
|
type Event struct {
|
|
Type EventType
|
|
Group string
|
|
Key string
|
|
Value string
|
|
Timestamp time.Time
|
|
}
|
|
```
|
|
|
|
| EventType | String() | Key populated | Value populated |
|
|
|---|---|---|---|
|
|
| `EventSet` | `"set"` | Yes | Yes |
|
|
| `EventDelete` | `"delete"` | Yes | No |
|
|
| `EventDeleteGroup` | `"delete_group"` | No (empty) | No |
|
|
|
|
Events are emitted synchronously after each successful database write inside `notify()`. `notify()` acquires a read-lock on `s.mu`, iterates watchers with non-blocking channel sends, then calls each registered callback. The read-lock allows multiple concurrent `notify()` calls to proceed simultaneously; `Watch`/`Unwatch`/`OnChange` take a write-lock when modifying the registry.
|
|
|
|
## Namespace Isolation (ScopedStore)
|
|
|
|
`ScopedStore` wraps a `*Store` and automatically prefixes all group names with `namespace + ":"`. This prevents key collisions when multiple tenants share a single underlying database.
|
|
|
|
```go
|
|
sc, _ := store.NewScoped(st, "tenant-42")
|
|
sc.Set("config", "theme", "dark")
|
|
// Stored in underlying store as group="tenant-42:config", key="theme"
|
|
```
|
|
|
|
Namespace strings must match `^[a-zA-Z0-9-]+$`. Invalid namespaces are rejected at construction time.
|
|
|
|
`ScopedStore` delegates all operations to the underlying `Store` after prefixing. Events emitted by scoped operations carry the full prefixed group name in `Event.Group`, enabling watchers on the underlying store to observe scoped mutations.
|
|
|
|
### Quota Enforcement
|
|
|
|
`NewScopedWithQuota(store, namespace, QuotaConfig)` adds per-namespace limits:
|
|
|
|
```go
|
|
type QuotaConfig struct {
|
|
MaxKeys int // maximum total keys across all groups in the namespace
|
|
MaxGroups int // maximum distinct groups in the namespace
|
|
}
|
|
```
|
|
|
|
Zero values mean unlimited. Before each `Set` or `SetWithTTL`, the scoped store:
|
|
|
|
1. Checks whether the key already exists (upserts never consume quota).
|
|
2. If the key is new, queries `CountAll(namespace + ":")` and compares against `MaxKeys`.
|
|
3. If the group is new (current count for that group is zero), queries `Groups(namespace + ":")` and compares against `MaxGroups`.
|
|
|
|
Exceeding a limit returns `ErrQuotaExceeded`.
|
|
|
|
## Concurrency Model
|
|
|
|
All SQLite access is serialised through a single connection (`SetMaxOpenConns(1)`). The store's watcher/callback registry is protected by a separate `sync.RWMutex` (`s.mu`). These two locks do not interact:
|
|
|
|
- DB writes acquire no application-level lock.
|
|
- `notify()` acquires `s.mu` (read) after the DB write completes.
|
|
- `Watch`/`Unwatch`/`OnChange` acquire `s.mu` (write) to modify the registry.
|
|
|
|
All operations are safe to call from multiple goroutines concurrently. The race detector is clean under the project's standard test suite (`go test -race ./...`).
|
|
|
|
## File Layout
|
|
|
|
```
|
|
store.go Core Store type, CRUD operations, TTL, background purge
|
|
events.go EventType, Event, Watcher, OnChange, notify
|
|
scope.go ScopedStore, QuotaConfig
|
|
store_test.go Tests: CRUD, TTL, concurrency, edge cases, benchmarks
|
|
events_test.go Tests: Watch, Unwatch, OnChange, event dispatch
|
|
scope_test.go Tests: namespace isolation, quota enforcement
|
|
coverage_test.go Tests: error paths for defensive code (scan errors, corruption)
|
|
bench_test.go Additional benchmarks
|
|
```
|