# Architecture — go-store

Module: `forge.lthn.ai/core/go-store`

## Overview

go-store is a group-namespaced key-value store backed by SQLite. It provides persistent or in-memory storage with optional TTL expiry, namespace isolation for multi-tenant use, and a reactive event system for observing mutations.

The package has no external runtime dependencies beyond a pure-Go SQLite driver (`modernc.org/sqlite`). It requires no CGO and compiles on all platforms.

## Storage Layer

### SQLite with WAL Mode

Every `Store` instance opens a single SQLite database and immediately applies two pragmas:

```sql
PRAGMA journal_mode=WAL;
PRAGMA busy_timeout=5000;
```

WAL (Write-Ahead Logging) mode allows concurrent readers to proceed without blocking writers. The `busy_timeout` of 5000 milliseconds causes the driver to wait and retry rather than immediately returning `SQLITE_BUSY` under write contention.

**Single connection constraint.** The `database/sql` package maintains a connection pool by default. SQLite pragmas are per-connection: if the pool hands out a second connection, that connection inherits none of the WAL or busy-timeout settings, causing `SQLITE_BUSY` errors under concurrent load. go-store calls `db.SetMaxOpenConns(1)` to pin all access to a single connection. Since SQLite serialises writes at the file level regardless, this introduces no additional throughput penalty.

### Schema

```sql
CREATE TABLE IF NOT EXISTS kv (
    grp        TEXT NOT NULL,
    key        TEXT NOT NULL,
    value      TEXT NOT NULL,
    expires_at INTEGER,
    PRIMARY KEY (grp, key)
)
```

The compound primary key `(grp, key)` enforces uniqueness per group-key pair and provides efficient indexed lookups. The `expires_at` column stores a Unix millisecond timestamp (nullable); a `NULL` value means the key never expires.

**Schema migration.** Databases created before TTL support lacked the `expires_at` column. On `New()`, go-store runs `ALTER TABLE kv ADD COLUMN expires_at INTEGER`. If the column already exists, SQLite returns a "duplicate column" error which is silently ignored. This allows seamless upgrades of existing databases.

## Group/Key Model

Keys are addressed by a two-level path: `(group, key)`. Groups act as logical namespaces within a single database. Groups are implicit — they exist as a consequence of the keys they contain and are destroyed automatically when all their keys are deleted.

This model maps naturally to domain concepts:

```
group: "user:42:config"     key: "theme"
group: "user:42:config"     key: "language"
group: "session:abc"        key: "token"
```

All read operations (`Get`, `GetAll`, `Count`, `Render`) are scoped to a single group. `DeleteGroup` atomically removes all keys in a group. `CountAll` and `Groups` operate across groups by prefix match.

## UPSERT Semantics

All writes use `INSERT ... ON CONFLICT(grp, key) DO UPDATE`. This means:

- Inserting a new key creates it.
- Inserting an existing key overwrites its value and (for `Set`) clears any TTL.
- UPSERT never duplicates a key.
- The operation is idempotent with respect to row count.

`Set` clears `expires_at` on upsert by setting it to `NULL`. `SetWithTTL` refreshes the expiry timestamp on upsert.

## TTL Expiry

Keys may be created with a time-to-live via `SetWithTTL`. Expiry is stored as a Unix millisecond timestamp in `expires_at`.

**Expiry is enforced in three ways:**

1. **Lazy deletion on `Get`.** If a key is found but its `expires_at` is in the past, it is deleted synchronously before returning `ErrNotFound`. This prevents stale values from being returned even if the background purge has not run yet.

2. **Query-time filtering.** All bulk operations (`GetAll`, `Count`, `Render`, `CountAll`, `Groups`) include `(expires_at IS NULL OR expires_at > ?)` in their `WHERE` clause. Expired keys are excluded from results without being deleted.

3. **Background purge goroutine.** `New()` launches a goroutine that calls `PurgeExpired()` every 60 seconds (configurable via `s.purgeInterval`). This recovers disk space by physically removing expired rows. The goroutine is stopped cleanly by `Close()` via `context.WithCancel`.

`PurgeExpired()` is also a public method for applications that want manual control over purge timing.

## Template Rendering

`Render(tmplStr, group string)` is a convenience method that fetches all non-expired key-value pairs from a group and renders a Go `text/template` against them. The template data is a `map[string]string` keyed by the field name.

Example:

```go
st.Set("miner", "pool", "pool.lthn.io:3333")
st.Set("miner", "wallet", "iz...")
out, _ := st.Render(`{"pool":"{{ .pool }}","wallet":"{{ .wallet }}"}`, "miner")
// out: {"pool":"pool.lthn.io:3333","wallet":"iz..."}
```

Template parse errors and execution errors are both returned as wrapped errors with context (e.g., `store.Render: parse: ...` and `store.Render: exec: ...`).

Missing template variables do not return an error by default — Go's `text/template` renders them as `<no value>`. Applications requiring strict variable presence should set `Option("missingkey=error")` on their own template instances before calling `Render`, or validate data beforehand.

## Watch/Unwatch Pattern

`Watch(group, key string) *Watcher` registers a subscription that receives events as they occur. Each `Watcher` holds a buffered channel (`Ch <-chan Event`) with capacity 16.

**Filter semantics:**

| group argument | key argument | Receives |
|---|---|---|
| `"mygroup"` | `"mykey"` | Only mutations to that exact key |
| `"mygroup"` | `"*"` | All mutations within the group, including `DeleteGroup` |
| `"*"` | `"*"` | Every mutation across the entire store |

`Unwatch(w *Watcher)` removes the watcher from the registry and closes its channel. It is safe to call multiple times — subsequent calls are no-ops.

**Backpressure.** Event dispatch to a watcher channel is non-blocking: if the channel buffer is full, the event is dropped silently. This prevents a slow consumer from blocking a writer. Applications that cannot afford dropped events should drain the channel promptly or use `OnChange` callbacks instead.

**Idiomatic usage:**

```go
w := st.Watch("config", "*")
defer st.Unwatch(w)

for e := range w.Ch {
    fmt.Println(e.Type, e.Group, e.Key, e.Value)
}
```

## OnChange Callbacks

`OnChange(fn func(Event)) func()` registers a synchronous callback that fires on every mutation. The callback runs in the goroutine that performed the write, holding the watcher/callback read-lock. Callers must not call store methods from within a callback (deadlock risk) and should offload any significant work to a goroutine.

`OnChange` returns an unregister function. Calling it removes the callback from the registry. The unregister function is idempotent.

This is the designed integration point for go-ws:

```go
unreg := st.OnChange(func(e store.Event) {
    hub.SendToChannel("store-events", e)
})
defer unreg()
```

go-store does not import go-ws. The dependency flows in one direction only: go-ws (or any consumer) imports go-store.

## Event Model

Events are defined in `events.go`:

```go
type Event struct {
    Type      EventType
    Group     string
    Key       string
    Value     string
    Timestamp time.Time
}
```

| EventType | String() | Key populated | Value populated |
|---|---|---|---|
| `EventSet` | `"set"` | Yes | Yes |
| `EventDelete` | `"delete"` | Yes | No |
| `EventDeleteGroup` | `"delete_group"` | No (empty) | No |

Events are emitted synchronously after each successful database write inside `notify()`. `notify()` acquires a read-lock on `s.mu`, iterates watchers with non-blocking channel sends, then calls each registered callback. The read-lock allows multiple concurrent `notify()` calls to proceed simultaneously; `Watch`/`Unwatch`/`OnChange` take a write-lock when modifying the registry.

## Namespace Isolation (ScopedStore)

`ScopedStore` wraps a `*Store` and automatically prefixes all group names with `namespace + ":"`. This prevents key collisions when multiple tenants share a single underlying database.

```go
sc, _ := store.NewScoped(st, "tenant-42")
sc.Set("config", "theme", "dark")
// Stored in underlying store as group="tenant-42:config", key="theme"
```

Namespace strings must match `^[a-zA-Z0-9-]+$`. Invalid namespaces are rejected at construction time.

`ScopedStore` delegates all operations to the underlying `Store` after prefixing. Events emitted by scoped operations carry the full prefixed group name in `Event.Group`, enabling watchers on the underlying store to observe scoped mutations.

### Quota Enforcement

`NewScopedWithQuota(store, namespace, QuotaConfig)` adds per-namespace limits:

```go
type QuotaConfig struct {
    MaxKeys   int // maximum total keys across all groups in the namespace
    MaxGroups int // maximum distinct groups in the namespace
}
```

Zero values mean unlimited. Before each `Set` or `SetWithTTL`, the scoped store:

1. Checks whether the key already exists (upserts never consume quota).
2. If the key is new, queries `CountAll(namespace + ":")` and compares against `MaxKeys`.
3. If the group is new (current count for that group is zero), queries `Groups(namespace + ":")` and compares against `MaxGroups`.

Exceeding a limit returns `ErrQuotaExceeded`.

## Concurrency Model

All SQLite access is serialised through a single connection (`SetMaxOpenConns(1)`). The store's watcher/callback registry is protected by a separate `sync.RWMutex` (`s.mu`). These two locks do not interact:

- DB writes acquire no application-level lock.
- `notify()` acquires `s.mu` (read) after the DB write completes.
- `Watch`/`Unwatch`/`OnChange` acquire `s.mu` (write) to modify the registry.

All operations are safe to call from multiple goroutines concurrently. The race detector is clean under the project's standard test suite (`go test -race ./...`).

## File Layout

```
store.go        Core Store type, CRUD operations, TTL, background purge
events.go       EventType, Event, Watcher, OnChange, notify
scope.go        ScopedStore, QuotaConfig
store_test.go   Tests: CRUD, TTL, concurrency, edge cases, benchmarks
events_test.go  Tests: Watch, Unwatch, OnChange, event dispatch
scope_test.go   Tests: namespace isolation, quota enforcement
coverage_test.go  Tests: error paths for defensive code (scan errors, corruption)
bench_test.go   Additional benchmarks
```