From bbf232238919e950cc1b88165cfbfeb8a97ae4fe Mon Sep 17 00:00:00 2001 From: Snider Date: Fri, 20 Feb 2026 15:01:55 +0000 Subject: [PATCH] docs: graduate TODO/FINDINGS into production documentation Replace internal task tracking (TODO.md, FINDINGS.md) with structured documentation in docs/. Trim CLAUDE.md to agent instructions only. Co-Authored-By: Virgil --- CLAUDE.md | 143 +++---------- FINDINGS.md | 179 ---------------- TODO.md | 56 ----- docs/architecture.md | 487 +++++++++++++++++++++++++++++++++++++++++++ docs/development.md | 197 +++++++++++++++++ docs/history.md | 236 +++++++++++++++++++++ 6 files changed, 947 insertions(+), 351 deletions(-) delete mode 100644 FINDINGS.md delete mode 100644 TODO.md create mode 100644 docs/architecture.md create mode 100644 docs/development.md create mode 100644 docs/history.md diff --git a/CLAUDE.md b/CLAUDE.md index 8870b8b..493ad26 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,25 +1,31 @@ -# CLAUDE.md — go-crypt Domain Expert Guide +# CLAUDE.md — go-crypt -You are a dedicated domain expert for `forge.lthn.ai/core/go-crypt`. Virgil (in core/go) orchestrates your work via TODO.md. Pick up tasks in phase order, mark `[x]` when done, commit and push. +You are a dedicated domain expert for `forge.lthn.ai/core/go-crypt`. Virgil (in +core/go) orchestrates your work. Pick up tasks in phase order, mark `[x]` when +done, commit and push. ## What This Package Does -Cryptographic primitives, authentication, and trust policy engine. ~3.7K LOC across 28 Go files. Provides: +Cryptographic primitives, authentication, and trust policy engine for the +Lethean agent platform. Provides: -- **Symmetric encryption** — ChaCha20-Poly1305 and AES-256-GCM with Argon2id key derivation -- **OpenPGP authentication** — Challenge-response (online + air-gapped courier mode) -- **Password hashing** — Argon2id (primary) + Bcrypt (fallback) -- **Trust policy engine** — 3-tier agent access control with capability evaluation -- **RSA** — OAEP-SHA256 key generation and encryption (2048+ bit) -- **LTHN hash** — RFC-0004 quasi-salted deterministic hash (content IDs, NOT passwords) +- Symmetric encryption — ChaCha20-Poly1305 and AES-256-GCM with Argon2id KDF +- OpenPGP authentication — challenge-response (online + air-gapped courier mode) +- Password hashing — Argon2id (primary) + Bcrypt (fallback) +- Trust policy engine — 3-tier agent access control with capability evaluation +- RSA — OAEP-SHA256 key generation and encryption (2048+ bit) +- LTHN hash — RFC-0004 quasi-salted deterministic hash (content IDs, NOT passwords) + +For architecture details see `docs/architecture.md`. For history and findings +see `docs/history.md`. ## Commands ```bash go test ./... # Run all tests +go test -race ./... # Race detector (required before committing) go test -v -run TestName ./... # Single test -go test -race ./... # Race detector -go vet ./... # Static analysis +go vet ./... # Static analysis (must be clean) ``` ## Local Dependencies @@ -29,119 +35,24 @@ go vet ./... # Static analysis | `forge.lthn.ai/core/go` | `../go` | Framework (core.E, core.Crypt, io.Medium) | | `forge.lthn.ai/core/go-store` | `../go-store` | SQLite KV store (session persistence) | -**Do NOT change the replace directive path.** Use go.work for local resolution if needed. - -## Architecture - -### auth/ — OpenPGP Challenge-Response + Password Auth (455 LOC) - -`Authenticator` backed by `io.Medium` storage abstraction. - -**Registration flow**: Generate PGP keypair → store `.pub`, `.key`, `.rev`, `.json`, `.lthn` files under `users/{userID}/`. - -**Online challenge-response**: -1. `CreateChallenge(userID)` → 32-byte random nonce, encrypted with user's public key -2. Client decrypts nonce, signs it with private key -3. `ValidateResponse(userID, signedNonce)` → verifies signature, issues 24h session token - -**Air-gapped (courier) mode**: -1. `WriteChallengeFile(userID, path)` → JSON with encrypted nonce -2. Client signs offline -3. `ReadResponseFile(userID, path)` → verify, issue session - -**Session management**: Abstracted behind `SessionStore` interface. 32-byte hex tokens, 24h TTL. `ValidateSession`, `RefreshSession`, `RevokeSession`. Two implementations: -- `MemorySessionStore` — in-memory `sync.RWMutex`-protected map (default, sessions lost on restart) -- `SQLiteSessionStore` — persistent via go-store (SQLite KV), mutex-serialised for single-writer safety - -Configure via `WithSessionStore(store)` option. Background cleanup via `StartCleanup(ctx, interval)`. - -**Protected users**: `"server"` cannot be deleted. - -### crypt/ — Symmetric Encryption & Hashing (624 LOC) - -| File | LOC | Purpose | -|------|-----|---------| -| `crypt.go` | 90 | High-level `Encrypt`/`Decrypt` (ChaCha20 + Argon2id) and AES-256-GCM variant | -| `kdf.go` | 60 | `DeriveKey` (Argon2id: 64MB/3 iter/4 threads), `DeriveKeyScrypt`, `HKDF` | -| `symmetric.go` | 100 | Low-level `ChaCha20Encrypt`/`Decrypt`, `AESGCMEncrypt`/`Decrypt` | -| `hash.go` | 89 | `HashPassword`/`VerifyPassword` (Argon2id format string), Bcrypt | -| `hmac.go` | 30 | `HMACSHA256`/`512`, constant-time `VerifyHMAC` | -| `checksum.go` | 55 | `SHA256File`, `SHA512File`, `SHA256Sum`, `SHA512Sum` | - -#### crypt/chachapoly/ (50 LOC) -Standalone ChaCha20-Poly1305 AEAD wrapper. 24-byte nonce prepended to ciphertext. - -#### crypt/lthn/ (94 LOC) -RFC-0004 quasi-salted hash. Deterministic: reverse input → leet-speak substitution → SHA-256. For content IDs and deduplication. **NOT for passwords.** - -#### crypt/pgp/ (230 LOC) -OpenPGP primitives via ProtonMail `go-crypto`: -- `CreateKeyPair(name, email, password)` → armored DSA primary + RSA subkey -- `Encrypt`/`Decrypt` → armored PGP messages -- `Sign`/`Verify` → detached signatures - -#### crypt/rsa/ (91 LOC) -RSA key generation (2048+ bit), OAEP-SHA256 encrypt/decrypt, PEM encoding. - -#### crypt/openpgp/ (191 LOC) -Service wrapper implementing `core.Crypt` interface. RSA-4096, SHA-256, AES-256. Registers IPC handler for `"openpgp.create_key_pair"`. - -### trust/ — Agent Trust & Policy Engine (403 LOC) - -| File | LOC | Purpose | -|------|-----|---------| -| `trust.go` | 165 | `Registry` (thread-safe agent store), `Agent` struct, `Tier` enum | -| `policy.go` | 238 | `PolicyEngine`, 9 capabilities, `Evaluate` → Allow/Deny/NeedsApproval | - -**Trust tiers**: -| Tier | Name | Rate Limit | Example Agents | -|------|------|-----------|----------------| -| 3 | Full | Unlimited | Athena, Virgil, Charon | -| 2 | Verified | 60/min | Clotho, Hypnos (scoped repos) | -| 1 | Untrusted | 10/min | BugSETI instances | - -**9 Capabilities**: `repo.push`, `pr.merge`, `pr.create`, `issue.create`, `issue.comment`, `secrets.read`, `cmd.privileged`, `workspace.access`, `flows.modify` - -**Evaluation order**: Agent exists → policy exists → explicitly denied → requires approval → allowed (with repo scope check). - -## Algorithm Reference - -| Component | Algorithm | Parameters | -|-----------|-----------|-----------| -| KDF (primary) | Argon2id | Time=3, Memory=64MB, Parallelism=4 | -| KDF (alt) | scrypt | N=32768, r=8, p=1 | -| KDF (expand) | HKDF-SHA256 | Variable key length | -| Symmetric | ChaCha20-Poly1305 | 24-byte nonce, 32-byte key | -| Symmetric (alt) | AES-256-GCM | 12-byte nonce, 32-byte key | -| Password hash | Argon2id | Custom format string | -| Password hash (alt) | Bcrypt | Default cost | -| Deterministic hash | SHA-256 + quasi-salt | RFC-0004 | -| Asymmetric | RSA-OAEP-SHA256 | 2048+ bit | -| PGP | DSA + RSA subkey | ProtonMail go-crypto | -| HMAC | SHA-256 / SHA-512 | Constant-time verify | - -## Security Considerations - -1. **LTHN hash is NOT for passwords** — deterministic, no random salt. Use `HashPassword()` (Argon2id) instead. -2. **Sessions default to in-memory** — use `WithSessionStore(NewSQLiteSessionStore(path))` for persistence across restarts. -3. **PGP output is armored** — ~33% Base64 overhead. Consider compression for large payloads. -4. **Policy engine returns decisions but doesn't enforce approval workflow** — higher-level layer needed. -5. **Challenge nonces are 32 bytes** — 256-bit, cryptographically random. +Do not change the replace directive paths. Use a `go.work` for local resolution +if working outside the full monorepo. ## Coding Standards -- **UK English**: colour, organisation, centre +- **UK English**: colour, organisation, centre, artefact, licence, serialise - **Tests**: testify assert/require, `_Good`/`_Bad`/`_Ugly` naming convention +- **Concurrency tests**: 10 goroutines via WaitGroup; must pass `-race` +- **Imports**: stdlib → forge.lthn.ai → third-party, separated by blank lines +- **Errors**: use `core.E("package.Function", "lowercase message", err)`; never + include secrets in error strings +- **Randomness**: `crypto/rand` only; never `math/rand` - **Conventional commits**: `feat(auth):`, `fix(crypt):`, `refactor(trust):` - **Co-Author**: `Co-Authored-By: Virgil ` - **Licence**: EUPL-1.2 -- **Imports**: stdlib → forge.lthn.ai → third-party, each group separated by blank line ## Forge - **Repo**: `forge.lthn.ai/core/go-crypt` -- **Push via SSH**: `git push forge main` (remote: `ssh://git@forge.lthn.ai:2223/core/go-crypt.git`) - -## Task Queue - -See `TODO.md` for prioritised work. See `FINDINGS.md` for research notes. +- **Push via SSH**: `git push forge main` + (remote: `ssh://git@forge.lthn.ai:2223/core/go-crypt.git`) diff --git a/FINDINGS.md b/FINDINGS.md deleted file mode 100644 index 129c565..0000000 --- a/FINDINGS.md +++ /dev/null @@ -1,179 +0,0 @@ -# FINDINGS.md — go-crypt Research & Discovery - -## 2026-02-20: Initial Analysis (Virgil) - -### Origin - -Extracted from `core/go` on 16 Feb 2026 (commit `8498ecf`). Single extraction commit — fresh repo with no prior history. - -### Package Inventory - -| Package | Source LOC | Test LOC | Test Count | Notes | -|---------|-----------|----------|-----------|-------| -| `auth/` | 455 | 581 | 25+ | OpenPGP challenge-response + LTHN password | -| `crypt/` | 90 | 45 | 4 | High-level encrypt/decrypt convenience | -| `crypt/kdf.go` | 60 | 56 | — | Argon2id, scrypt, HKDF | -| `crypt/symmetric.go` | 100 | 55 | — | ChaCha20-Poly1305, AES-256-GCM | -| `crypt/hash.go` | 89 | 50 | — | Argon2id password hashing, Bcrypt | -| `crypt/hmac.go` | 30 | 40 | — | HMAC-SHA256/512 | -| `crypt/checksum.go` | 55 | 23 | — | SHA-256/512 file and data checksums | -| `crypt/chachapoly/` | 50 | 114 | 9 | Standalone ChaCha20-Poly1305 wrapper | -| `crypt/lthn/` | 94 | 66 | 6 | RFC-0004 quasi-salted hash | -| `crypt/pgp/` | 230 | 164 | 11 | OpenPGP via ProtonMail go-crypto | -| `crypt/rsa/` | 91 | 101 | 3 | RSA OAEP-SHA256 | -| `crypt/openpgp/` | 191 | 43 | — | Service wrapper, core.Crypt interface | -| `trust/` | 165 | 164 | — | Agent registry, tier management | -| `trust/policy.go` | 238 | 268 | 40+ | Policy engine, 9 capabilities | - -**Total**: ~1,938 source LOC, ~1,770 test LOC (47.7% test ratio) - -### Dependencies - -- `forge.lthn.ai/core/go` — core.E error handling, core.Crypt interface, io.Medium storage -- `github.com/ProtonMail/go-crypto` v1.3.0 — OpenPGP (replaces deprecated golang.org/x/crypto/openpgp) -- `golang.org/x/crypto` v0.48.0 — Argon2, ChaCha20-Poly1305, scrypt, HKDF, bcrypt -- `github.com/cloudflare/circl` v1.6.3 — indirect (elliptic curve via ProtonMail) - -### Key Observations - -1. **Dual ChaCha20 wrappers** — `crypt/symmetric.go` and `crypt/chachapoly/chachapoly.go` implement nearly identical ChaCha20-Poly1305. The chachapoly sub-package pre-allocates nonce+plaintext capacity (minor optimisation). Consider consolidating. - -2. **LTHN hash is NOT constant-time** — `lthn.Verify()` uses direct string comparison (`==`), not `subtle.ConstantTimeCompare`. This is acceptable since LTHN is for content IDs, not passwords — but should be documented clearly. - -3. **OpenPGP service has IPC handler** — `openpgp.Service.HandleIPCEvents()` dispatches `"openpgp.create_key_pair"`. This is the only IPC-aware component in go-crypt. - -4. **Trust policy decisions are advisory** — `PolicyEngine.Evaluate()` returns `NeedsApproval` but there's no approval queue or workflow. The enforcement layer is expected to live in a higher-level package (go-agentic or go-scm). - -5. **Session tokens are in-memory** — No persistence. Suitable for development and single-process deployments, but not distributed systems or crash recovery. - -6. **Test naming follows `_Good`/`_Bad`/`_Ugly` pattern** — Consistent with core/go conventions. - -### Integration Points - -- **go-p2p** → UEPS layer will need crypt/ for consent-gated encryption -- **go-scm** → AgentCI trusts agents via trust/ policy engine -- **go-agentic** → Agent session management via auth/ -- **core/go** → OpenPGP service registered via core.Crypt interface - -### Security Review Flags - -- Argon2id parameters (64MB/3/4) are within OWASP recommended range -- RSA minimum 2048-bit enforced at key generation -- ChaCha20 nonces are 24-byte (XChaCha20-Poly1305), not 12-byte — good, avoids nonce reuse risk -- PGP uses ProtonMail fork (actively maintained, post-quantum research) -- No detected use of `math/rand` — all randomness from `crypto/rand` - ---- - -## Security Audit (Phase 0) - -Conducted 2026-02-20. All source files reviewed for cryptographic hygiene. - -### 1. Constant-Time Comparisons - -| Location | Comparison | Verdict | -|----------|-----------|---------| -| `crypt/hash.go:66` | `subtle.ConstantTimeCompare(computedHash, expectedHash)` | PASS — Argon2id password verification uses constant-time compare | -| `crypt/hmac.go:29` | `hmac.Equal(mac, expected.Sum(nil))` | PASS — HMAC verification uses constant-time compare | -| `crypt/lthn/lthn.go:93` | `Hash(input) == hash` | ACCEPTABLE — LTHN is for content IDs, not passwords. Documented in CLAUDE.md. | -| `auth/auth.go:282` | `a.sessions[token]` | ACCEPTABLE — Map lookup by token as key. 64-hex-char token (256-bit entropy) makes brute-force timing attacks infeasible. | -| `auth/auth.go:387` | `lthn.Verify(password, storedHash)` | **FINDING** — Password verification uses LTHN hash with non-constant-time `==`. See Finding F1 below. | - -### 2. Nonce/Randomness Generation - -All nonce and random value generation uses `crypto/rand`: - -| Location | Purpose | Entropy | -|----------|---------|---------| -| `auth/auth.go:218` | Challenge nonce | 32 bytes (256-bit) via `crypto/rand.Read` | -| `auth/auth.go:439` | Session token | 32 bytes (256-bit) via `crypto/rand.Read` | -| `crypt/kdf.go:55` | Salt generation | 16 bytes (128-bit) via `crypto/rand.Read` | -| `crypt/symmetric.go:22` | ChaCha20 nonce | 24 bytes via `crypto/rand.Read` | -| `crypt/symmetric.go:67` | AES-GCM nonce | 12 bytes via `crypto/rand.Read` | -| `crypt/rsa/rsa.go:25` | RSA key generation | `crypto/rand.Reader` | - -**No usage of `math/rand` detected anywhere in the codebase.** PASS. - -### 3. PGP Private Key Handling - -**FINDING F2**: PGP private key material is NOT zeroed after use. In `pgp.Decrypt()` and `pgp.Sign()`, the private key is decrypted into memory (via `entity.PrivateKey.Decrypt()`) but the decrypted key material remains in memory until garbage collected. The ProtonMail go-crypto library does not provide a `Wipe()` or `Zero()` method on `packet.PrivateKey`, so this is currently a limitation of the upstream library rather than a code defect. Mitigating this would require forking or patching go-crypto. - -**Severity**: Low. The Go runtime does not guarantee memory zeroing, and GC-managed languages inherently have this limitation. In practice, an attacker who can read process memory already has full access. - -### 4. Error Message Review - -No secrets (passwords, tokens, private keys, nonces) leak in error messages. All error strings are generic: -- `"user not found"`, `"invalid password"`, `"session not found"`, `"session expired"` -- `"failed to decrypt"`, `"failed to encrypt"`, `"challenge expired"` -- `"ciphertext too short"`, `"failed to generate nonce"` - -The `trust.Register` error includes the agent name (`"invalid tier %d for agent %q"`) which is acceptable — agent names are not secrets. - -PASS. - -### 5. Session Token Security - -- **Entropy**: 32 bytes from `crypto/rand` → 256-bit. Well above the 128-bit minimum. -- **Format**: Hex-encoded → 64-character string. No structural information leaked. -- **Storage**: In-memory `map[string]*Session` behind `sync.RWMutex`. -- **Expiry**: Checked on every `ValidateSession` and `RefreshSession` call. Expired sessions are deleted on access. - -PASS. - -### Findings - -#### F1: LTHN Hash Used for Password Verification (Medium Severity) - -`auth.Login()` verifies passwords via `lthn.Verify()` which uses the LTHN quasi-salted hash (RFC-0004) with a non-constant-time string comparison (`==`). LTHN was designed for content identifiers, NOT passwords. - -**Impact**: The LTHN hash is deterministic (same input always produces same output) with no random salt. While the quasi-salt derivation adds entropy, it provides weaker protection than Argon2id (`crypt.HashPassword`/`crypt.VerifyPassword` which is available but unused here). - -**Timing risk**: The `==` comparison in `lthn.Verify` could theoretically leak information through timing side-channels, though the practical impact is limited because: -1. The comparison is on SHA-256 hex digests (fixed 64 chars) -2. An attacker would need to hash candidate passwords through the LTHN algorithm first - -**Recommendation**: Consider migrating password storage from LTHN to Argon2id (`crypt.HashPassword`/`crypt.VerifyPassword`) in a future phase. This would add random salting and constant-time comparison. - -#### F2: PGP Private Keys Not Zeroed After Use (Low Severity) - -See Section 3 above. Upstream limitation of ProtonMail go-crypto. - -#### F3: Trust Policy — Empty ScopedRepos Bypasses Scope Check (Medium Severity) - -In `policy.go:122`, the repo scope check is: `if isRepoScoped(cap) && len(agent.ScopedRepos) > 0`. This means a Tier 2 agent with empty `ScopedRepos` (either `nil` or `[]string{}`) is treated as "unrestricted" rather than "no access". - -**Impact**: If an admin creates a Tier 2 agent without explicitly setting `ScopedRepos`, the agent gets access to ALL repositories for repo-scoped capabilities (`repo.push`, `pr.create`, `pr.merge`, `secrets.read`). - -**Recommendation**: Consider treating empty `ScopedRepos` as "no access" for Tier 2 agents, or requiring explicit `ScopedRepos: []string{"*"}` for unrestricted access. This is a design decision for Phase 3. - -#### F4: `go vet` Clean - -`go vet ./...` produces no warnings. PASS. - ---- - -## Phase 2: Key Management Implementation (20 Feb 2026) - -### F1 Resolution — Argon2id Migration - -Finding F1 addressed in `301eac1`. New registrations now use `crypt.HashPassword()` (Argon2id) with random salt and constant-time verification. Hash stored in `.hash` file. Legacy `.lthn` files transparently migrated on successful login: LTHN hash verified → Argon2id re-hash → `.hash` file written. Both paths handled by shared `verifyPassword()` helper. - -### Password Verification Dual-Path Design - -The `verifyPassword()` helper was extracted after `TestRevokeKey_Bad` failed — new registrations don't write `.lthn` files, so the fallback returned "user not found" instead of "invalid password". The helper tries Argon2id (`.hash`) first, then LTHN (`.lthn`), returning appropriate error messages for each path. Used by both `Login()` and `RevokeKey()`. - -### Revocation Design Choice - -Chose Option B (JSON record) over Option A (OpenPGP revocation cert). The `Revocation` struct stores `{UserID, Reason, RevokedAt}` as JSON. `IsRevoked()` parses JSON and ignores legacy `"REVOCATION_PLACEHOLDER"` strings. Login and CreateChallenge both check revocation before proceeding. - -### Key Rotation Flow - -`RotateKeyPair()` implements full key rotation: load private key → decrypt metadata with old password → generate new PGP keypair → re-encrypt metadata → overwrite `.pub/.key/.json/.hash` → invalidate sessions via `store.DeleteByUser()`. The old key material is implicitly discarded (same F2 limitation as PGP — Go GC, not zeroed). - -### HardwareKey Interface - -Contract-only definition in `hardware.go`. Four methods: `Sign`, `Decrypt`, `GetPublicKey`, `IsAvailable`. Integration points documented but not wired up. The `Authenticator.hardwareKey` field is set via `WithHardwareKey()` option. - -### Test Coverage After Phase 2 - -55 test functions across auth package. Key new tests: Argon2id registration/login (5), key rotation (4), key revocation (6). All pass with `-race`. diff --git a/TODO.md b/TODO.md deleted file mode 100644 index 8212acb..0000000 --- a/TODO.md +++ /dev/null @@ -1,56 +0,0 @@ -# TODO.md — go-crypt - -Dispatched from core/go orchestration. Pick up tasks in order. - ---- - -## Phase 0: Test Coverage & Hardening - -- [x] **Expand auth/ tests** — Added 8 new tests: concurrent session creation (10 goroutines), session token uniqueness (1000 tokens), challenge expiry boundary, empty password registration, very long username (10K chars), Unicode username/password, air-gapped round-trip, refresh already-expired session. All pass with `-race`. -- [x] **Expand crypt/ tests** — Added 12 new tests: wrong passphrase decrypt (ChaCha20+AES), empty plaintext round-trip (ChaCha20+AES), 1MB payload round-trip (ChaCha20+AES), ciphertext-too-short rejection, key derivation determinism (Argon2id+scrypt), HKDF different info strings, HKDF nil salt, checksum of empty file (SHA-256+SHA-512), checksum of non-existent file, checksum consistency with SHA256Sum. Note: large payload test uses 1MB (not 10MB) to keep tests fast. -- [x] **Expand trust/ tests** — Added 9 new tests: concurrent Register/Get/Remove (10 goroutines, race-safe), Tier 0 rejection, negative tier rejection, token expiry boundary, zero-value token expiry, concurrent List during mutations, empty ScopedRepos behaviour (documented as finding F3), capability not in any list, concurrent Evaluate. -- [x] **Security audit** — Full audit documented in FINDINGS.md. 4 findings: F1 (LTHN used for passwords, medium), F2 (PGP keys not zeroed, low), F3 (empty ScopedRepos bypasses scope, medium), F4 (go vet clean). No `math/rand` usage. All nonces use `crypto/rand`. No secrets in error messages. -- [x] **`go vet ./...` clean** — No warnings. -- [x] **Benchmark suite** — Created `crypt/bench_test.go` (7 benchmarks: Argon2Derive, ChaCha20 1KB/1MB, AESGCM 1KB/1MB, HMACSHA256 1KB, VerifyHMACSHA256) and `trust/bench_test.go` (3 benchmarks: PolicyEvaluate 100 agents, RegistryGet, RegistryRegister). - -## Phase 1: Session Persistence - -- [x] **Session storage interface** — Extracted in-memory session map into `SessionStore` interface with `Get`, `Set`, `Delete`, `DeleteByUser`, `Cleanup` methods. `MemorySessionStore` wraps the original map+mutex pattern. `ErrSessionNotFound` sentinel error. -- [x] **SQLite session store** — `SQLiteSessionStore` backed by go-store (SQLite KV). Sessions stored as JSON in `"sessions"` group. Mutex-serialised for SQLite single-writer safety. -- [x] **Background cleanup** — `StartCleanup(ctx, interval)` goroutine purges expired sessions periodically. Stops on context cancellation. -- [x] **Session migration** — Backward-compatible: `MemorySessionStore` is default, `WithSessionStore(store)` option for persistent store. All existing tests updated and passing. Commit `1aeabfd`. - -## Phase 2: Key Management - -### Step 2.1: Password hash migration (addresses Finding F1) - -- [x] **Migrate Login() from LTHN to Argon2id** — Register uses `crypt.HashPassword()` (Argon2id), writes `.hash` file. Login detects format: tries `.hash` (Argon2id) first, falls back to `.lthn` (LTHN). Successful legacy login transparently re-hashes with Argon2id. Shared `verifyPassword()` helper handles dual-path logic. 5 tests: RegisterArgon2id_Good, LoginArgon2id_Good, LoginArgon2id_Bad, LegacyLTHNMigration_Good, LegacyLTHNLogin_Bad. - -### Step 2.2: Key rotation - -- [x] **RotateKeyPair** — Full flow: load private key → decrypt metadata with old password → generate new PGP keypair → re-encrypt metadata → update .pub/.key/.json/.hash → invalidate sessions. 4 tests: RotateKeyPair_Good, RotateKeyPair_Bad (wrong password), RotateKeyPair_Ugly (non-existent user), RotateKeyPair_OldKeyCannotDecrypt_Good. - -### Step 2.3: Key revocation - -- [x] **RevokeKey + IsRevoked** — Option B chosen: JSON `Revocation{UserID, Reason, RevokedAt}` record in `.rev` file. `IsRevoked()` parses JSON, ignores legacy `"REVOCATION_PLACEHOLDER"`. Login and CreateChallenge reject revoked users. 6 tests including legacy user revocation. - -### Step 2.4: Hardware key interface (contract only) - -- [x] **HardwareKey interface** — `hardware.go`: Sign, Decrypt, GetPublicKey, IsAvailable methods. `WithHardwareKey()` option on Authenticator. Contract-only, no concrete implementations yet. Integration points documented in auth.go. - -All Phase 2: commit `301eac1`. 55 tests total, all pass with `-race`. - -## Phase 3: Trust Policy Extensions - -- [x] **Approval workflow** — `ApprovalQueue` with `Submit`, `Approve`, `Deny`, `Get`, `Pending` methods. Thread-safe queue with unique IDs, status tracking, reviewer attribution. 22 tests including concurrent and end-to-end integration with PolicyEngine. -- [x] **Audit log** — `AuditLog` with append-only `Record`, `Entries`, `EntriesFor` methods. Optional `io.Writer` for JSON-line persistence. Custom `Decision` JSON marshalling. 18 tests including writer errors and concurrent logging. -- [x] **Dynamic policies** — `LoadPolicies`/`LoadPoliciesFromFile` parse JSON config. `ApplyPolicies`/`ApplyPoliciesFromFile` replace engine policies. `ExportPolicies` for round-trip serialisation. `DisallowUnknownFields` for strict parsing. 18 tests including round-trip. -- [x] **Scope wildcards** — `matchScope` supports exact match, single-level wildcard (`core/*`), and recursive wildcard (`core/**`). Updated `repoAllowed` to use pattern matching. 18 tests covering all edge cases including integration with PolicyEngine. - ---- - -## Workflow - -1. Virgil in core/go writes tasks here after research -2. This repo's dedicated session picks up tasks in phase order -3. Mark `[x]` when done, note commit hash diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..7cffe3f --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,487 @@ +# Architecture — go-crypt + +`forge.lthn.ai/core/go-crypt` provides cryptographic primitives, authentication, +and a trust policy engine for the Lethean agent platform. The module is ~1,938 +source LOC across three top-level packages (`auth`, `crypt`, `trust`) and five +sub-packages (`crypt/chachapoly`, `crypt/lthn`, `crypt/pgp`, `crypt/rsa`, +`crypt/openpgp`). + +--- + +## Package Map + +``` +go-crypt/ +├── auth/ OpenPGP challenge-response authentication, sessions, key management +│ ├── auth.go Authenticator struct, registration, login, key rotation/revocation +│ ├── session_store.go SessionStore interface + MemorySessionStore +│ ├── session_store_sqlite.go SQLiteSessionStore (persistent via go-store) +│ └── hardware.go HardwareKey interface (contract only, no implementations) +├── crypt/ Symmetric encryption, key derivation, hashing +│ ├── crypt.go High-level Encrypt/Decrypt (ChaCha20) and EncryptAES/DecryptAES +│ ├── kdf.go DeriveKey (Argon2id), DeriveKeyScrypt, HKDF +│ ├── symmetric.go ChaCha20Encrypt/Decrypt, AESGCMEncrypt/Decrypt +│ ├── hash.go HashPassword/VerifyPassword (Argon2id), HashBcrypt/VerifyBcrypt +│ ├── hmac.go HMACSHA256, HMACSHA512, VerifyHMAC +│ ├── checksum.go SHA256File, SHA512File, SHA256Sum, SHA512Sum +│ ├── chachapoly/ Standalone ChaCha20-Poly1305 AEAD wrapper +│ ├── lthn/ RFC-0004 quasi-salted deterministic hash +│ ├── pgp/ OpenPGP primitives (ProtonMail go-crypto) +│ ├── rsa/ RSA OAEP-SHA256 key generation and encryption +│ └── openpgp/ Service wrapper implementing core.Crypt interface +└── trust/ Agent trust model and policy engine + ├── trust.go Registry, Agent struct, Tier enum + ├── policy.go PolicyEngine, 9 capabilities, Evaluate + ├── approval.go ApprovalQueue for NeedsApproval workflow + ├── audit.go AuditLog — append-only policy evaluation log + ├── config.go LoadPolicies/ExportPolicies — JSON config round-trip + └── scope.go matchScope — wildcard pattern matching for repo scopes +``` + +--- + +## crypt/ — Symmetric Encryption and Hashing + +### High-Level API (`crypt.go`) + +The entry point for most callers. `Encrypt`/`Decrypt` chain Argon2id key +derivation with ChaCha20-Poly1305 AEAD: + +``` +Encrypt(plaintext, passphrase): + 1. Generate 16-byte random salt (crypto/rand) + 2. DeriveKey(passphrase, salt) → 32-byte key via Argon2id + 3. ChaCha20Encrypt(plaintext, key) → 24-byte nonce || ciphertext + 4. Output: salt || nonce || ciphertext +``` + +`EncryptAES`/`DecryptAES` follow the same structure but use AES-256-GCM +with a 12-byte nonce instead of the 24-byte XChaCha20 nonce. + +### Key Derivation (`kdf.go`) + +Three KDF functions are provided: + +| Function | Algorithm | Parameters | +|----------|-----------|------------| +| `DeriveKey` | Argon2id | Memory=64MB, Time=3, Parallelism=4, KeyLen=32 | +| `DeriveKeyScrypt` | scrypt | N=32768, r=8, p=1 | +| `HKDF` | HKDF-SHA256 | Variable key length, optional salt and info | + +Argon2id parameters are within the OWASP recommended range for interactive +logins. `HKDF` is used for key expansion when a high-entropy secret is already +available (e.g. deriving sub-keys from a master key). + +### Low-Level Symmetric (`symmetric.go`) + +`ChaCha20Encrypt` prepends the 24-byte nonce to the ciphertext and returns a +single byte slice. `AESGCMEncrypt` prepends the 12-byte nonce. Both use +`crypto/rand` for nonce generation. The ciphertext format self-describes the +nonce position; callers must not alter the layout between encrypt and decrypt. + +### Password Hashing (`hash.go`) + +`HashPassword` produces an Argon2id format string: + +``` +$argon2id$v=19$m=65536,t=3,p=4$$ +``` + +`VerifyPassword` re-derives the hash from the stored parameters and uses +`crypto/subtle.ConstantTimeCompare` for the final comparison. This avoids +timing side-channels during password verification. + +`HashBcrypt`/`VerifyBcrypt` wrap `golang.org/x/crypto/bcrypt` as a fallback +for systems where bcrypt is required by policy. + +### HMAC (`hmac.go`) + +`HMACSHA256`/`HMACSHA512` return raw MAC bytes. `VerifyHMAC` uses +`crypto/hmac.Equal` (constant-time) to compare a computed MAC against an +expected value. + +### Checksums (`checksum.go`) + +`SHA256File`/`SHA512File` compute checksums of files via streaming reads. +`SHA256Sum`/`SHA512Sum` operate on byte slices. All return lowercase hex strings. + +### crypt/chachapoly/ + +A standalone AEAD wrapper with slightly different capacity pre-allocation. The +nonce (24 bytes) is prepended to the ciphertext on encrypt and stripped on +decrypt. This package exists separately from `crypt/symmetric.go` for callers +that import only ChaCha20-Poly1305 without the full `crypt` package. + +Note: the two implementations are nearly identical. The main difference is that +`chachapoly` pre-allocates `cap(nonce) + len(plaintext) + overhead` before +appending, which can reduce allocations for small payloads. + +### crypt/lthn/ + +RFC-0004 quasi-salted deterministic hash. The algorithm: + +1. Reverse the input string. +2. Apply leet-speak character substitutions (`o`→`0`, `l`→`1`, `e`→`3`, + `a`→`4`, `s`→`z`, `t`→`7`, and inverses). +3. Concatenate original input with the derived quasi-salt. +4. Return SHA-256 of the concatenation, hex-encoded. + +This is deterministic — the same input always produces the same output. It is +designed for content identifiers, cache keys, and deduplication. It is **not** +suitable for password hashing because there is no random salt and the +comparison in `Verify` is not constant-time. + +### crypt/pgp/ + +OpenPGP primitives via `github.com/ProtonMail/go-crypto`: + +- `CreateKeyPair(name, email, password)` — generates a DSA primary key with an + RSA encryption subkey; returns armored public and private keys. +- `Encrypt(plaintext, publicKey)` — produces an armored PGP message. +- `Decrypt(ciphertext, privateKey, password)` — decrypts an armored message. +- `Sign(data, privateKey, password)` — creates a detached armored signature. +- `Verify(data, signature, publicKey)` — verifies a detached signature. + +PGP output is Base64-armored, which adds approximately 33% overhead relative +to raw binary. For large payloads consider compression before encryption. + +### crypt/rsa/ + +RSA OAEP-SHA256. `GenerateKeyPair(bits)` generates an RSA keypair (minimum +2048 bit is enforced at the call site). `Encrypt`/`Decrypt` use +`crypto/rsa.EncryptOAEP` with SHA-256. Keys are serialised as PEM blocks. + +### crypt/openpgp/ + +Service wrapper that implements the `core.Crypt` interface from `forge.lthn.ai/core/go`. +Uses RSA-4096 with SHA-256 and AES-256. This is the only IPC-aware component +in go-crypt: `HandleIPCEvents` dispatches the `"openpgp.create_key_pair"` action +when registered with a Core instance. + +--- + +## auth/ — OpenPGP Authentication + +### Authenticator + +The `Authenticator` struct manages all user identity operations. It takes an +`io.Medium` (from `forge.lthn.ai/core/go`) for storage and an optional +`SessionStore` for session persistence. + +```go +a := auth.New(medium, + auth.WithSessionStore(auth.NewSQLiteSessionStore("/var/lib/app/sessions.db")), + auth.WithSessionTTL(8*time.Hour), + auth.WithChallengeTTL(2*time.Minute), +) +``` + +### Storage Layout + +All user artefacts are stored under `users/` on the Medium, keyed by a userID +derived from `lthn.Hash(username)`: + +| File | Content | +|------|---------| +| `users/{userID}.pub` | Armored PGP public key | +| `users/{userID}.key` | Armored PGP private key (password-encrypted) | +| `users/{userID}.rev` | JSON revocation record, or legacy placeholder string | +| `users/{userID}.json` | User metadata, PGP-encrypted with the user's public key | +| `users/{userID}.hash` | Argon2id password hash (new registrations and migrated accounts) | +| `users/{userID}.lthn` | Legacy LTHN hash (pre-Phase-2 registrations only) | + +### Registration + +`Register(username, password)`: + +1. Derive `userID = lthn.Hash(username)`. +2. Check `users/{userID}.pub` does not exist. +3. `pgp.CreateKeyPair(userID, ...)` → armored keypair. +4. Write `.pub`, `.key`, `.rev` (placeholder). +5. `crypt.HashPassword(password)` → Argon2id hash string → write `.hash`. +6. JSON-marshal `User` metadata, PGP-encrypt with public key, write `.json`. + +### Online Challenge-Response + +``` +Client Server + | | + |-- CreateChallenge(userID) -------> | + | | 1. Generate 32-byte nonce (crypto/rand) + | | 2. PGP-encrypt nonce with user's public key + | | 3. Store pending challenge (TTL: 5 min) + | <-- Challenge{Encrypted} --------- | + | | + | (client decrypts nonce, signs it) | + | | + |-- ValidateResponse(signedNonce) -> | + | | 4. Verify detached PGP signature + | | 5. Create session (32-byte token, 24h TTL) + | <-- Session{Token} --------------- | +``` + +### Air-Gapped (Courier) Mode + +`WriteChallengeFile(userID, path)` writes the encrypted challenge as JSON to +the Medium. The client signs the nonce offline. `ReadResponseFile(userID, path)` +reads the armored signature and calls `ValidateResponse` to complete authentication. +This mode supports agents or users who cannot receive live HTTP responses. + +### Password-Based Login + +`Login(userID, password)` bypasses the PGP challenge-response flow and verifies +the password directly. It supports both hash formats via a dual-path strategy: + +1. If `users/{userID}.hash` exists and starts with `$argon2id$`: verify with + `crypt.VerifyPassword` (constant-time Argon2id comparison). +2. Otherwise fall back to `users/{userID}.lthn`: verify with `lthn.Verify`. + On success, transparently re-hash the password with Argon2id and write a + `.hash` file (best-effort, does not fail the login if the write fails). + +### Key Management + +**Rotation** (`RotateKeyPair(userID, oldPassword, newPassword)`): +- Load and decrypt current metadata using the old private key and password. +- Generate a new PGP keypair. +- Re-encrypt metadata with the new public key. +- Overwrite `.pub`, `.key`, `.json`, `.hash`. +- Invalidate all active sessions for the user via `store.DeleteByUser`. + +**Revocation** (`RevokeKey(userID, password, reason)`): +- Verify password (dual-path, same as Login). +- Write a `Revocation{UserID, Reason, RevokedAt}` JSON record to `.rev`. +- Invalidate all sessions. +- `IsRevoked` returns true only when the `.rev` file contains valid JSON with a + non-zero `RevokedAt`. The legacy `"REVOCATION_PLACEHOLDER"` string is treated + as non-revoked for backward compatibility. +- Both `Login` and `CreateChallenge` reject revoked users immediately. + +**Protected users**: The `"server"` userID cannot be deleted. It holds the +server keypair; deletion would permanently destroy the server's joining data. + +### Session Management + +Sessions are managed through the `SessionStore` interface: + +```go +type SessionStore interface { + Get(token string) (*Session, error) + Set(session *Session) error + Delete(token string) error + DeleteByUser(userID string) error + Cleanup() (int, error) +} +``` + +Two implementations are provided: + +| Implementation | Persistence | Concurrency | +|----------------|-------------|-------------| +| `MemorySessionStore` | None (lost on restart) | `sync.RWMutex` | +| `SQLiteSessionStore` | SQLite via go-store | Single mutex (SQLite single-writer) | + +Session tokens are 32 bytes from `crypto/rand`, hex-encoded to 64 characters +(256-bit entropy). Expiry is checked on every `ValidateSession` and +`RefreshSession` call; expired sessions are deleted on access. Background +cleanup runs via `StartCleanup(ctx, interval)`. + +### Hardware Key Interface + +`hardware.go` defines a `HardwareKey` interface for future PKCS#11, YubiKey, +or TPM integration: + +```go +type HardwareKey interface { + Sign(data []byte) ([]byte, error) + Decrypt(ciphertext []byte) ([]byte, error) + GetPublicKey() (string, error) + IsAvailable() bool +} +``` + +Configured via `WithHardwareKey(hk)`. Integration points are documented in +`auth.go` but not yet wired — there are no concrete implementations in this +module. + +--- + +## trust/ — Agent Trust and Policy Engine + +### Registry + +`Registry` is a thread-safe map of agent names to `Agent` structs, protected by +`sync.RWMutex`. An `Agent` carries: + +- `Name` — unique identifier (e.g. `"Athena"`, `"BugSETI-42"`). +- `Tier` — trust level (1, 2, or 3). +- `ScopedRepos` — repository patterns constraining Tier 2 repo access. +- `RateLimit` — requests per minute (0 = unlimited for Tier 3). +- `TokenExpiresAt` — optional token expiry. + +Default rate limits by tier: Tier 1 = 10/min, Tier 2 = 60/min, Tier 3 = unlimited. + +### Trust Tiers + +| Tier | Name | Default Rate Limit | Typical Agents | +|------|------|-------------------|----------------| +| 3 | Full | Unlimited | Athena, Virgil, Charon | +| 2 | Verified | 60/min | Clotho, Hypnos (scoped repos) | +| 1 | Untrusted | 10/min | BugSETI community instances | + +### Capabilities + +Nine capabilities are defined: + +| Capability | Description | +|------------|-------------| +| `repo.push` | Push commits to a repository | +| `pr.create` | Open a pull request | +| `pr.merge` | Merge a pull request | +| `issue.create` | Create an issue | +| `issue.comment` | Comment on an issue | +| `secrets.read` | Read repository secrets | +| `cmd.privileged` | Run privileged shell commands | +| `workspace.access` | Access another agent's workspace | +| `flows.modify` | Modify CI/CD flow definitions | + +### Policy Engine + +`NewPolicyEngine(registry)` loads default policies. Evaluation order in +`Evaluate(agentName, cap, repo)`: + +1. Agent not in registry → Deny. +2. No policy for agent's tier → Deny. +3. Capability in `Denied` list → Deny. +4. Capability in `RequiresApproval` list → NeedsApproval. +5. Capability in `Allowed` list: + - If repo-scoped capability and `len(agent.ScopedRepos) > 0`: check repo + against scope patterns → Deny if no match. + - Otherwise → Allow. +6. Capability not in any list → Deny. + +Default policies by tier: + +| Tier | Allowed | RequiresApproval | Denied | +|------|---------|-----------------|--------| +| Full (3) | All 9 capabilities | — | — | +| Verified (2) | repo.push, pr.create, issue.create, issue.comment, secrets.read | pr.merge | workspace.access, flows.modify, cmd.privileged | +| Untrusted (1) | pr.create, issue.comment | — | repo.push, pr.merge, issue.create, secrets.read, cmd.privileged, workspace.access, flows.modify | + +### Repo Scope Matching + +`matchScope(pattern, repo)` supports three forms: + +| Pattern | Matches | Does Not Match | +|---------|---------|----------------| +| `core/go-crypt` | `core/go-crypt` | `core/go-crypt/sub` | +| `core/*` | `core/go-crypt` | `core/go-crypt/sub` | +| `core/**` | `core/go-crypt`, `core/go-crypt/sub` | `other/repo` | + +Empty `ScopedRepos` on a Tier 2 agent is treated as unrestricted (no scope +check is applied). See known limitations in `docs/history.md` (Finding F3). + +### Approval Queue + +`ApprovalQueue` is a thread-safe queue for `NeedsApproval` decisions. It is +separate from the `PolicyEngine` — the engine returns `NeedsApproval` as a +decision, and the caller is responsible for submitting to the queue and polling +for resolution. The queue tracks: submitting agent, capability, repo context, +status (pending/approved/denied), reviewer identity, and timestamps. + +### Audit Log + +`AuditLog` records every policy evaluation as an `AuditEntry`. Entries are +stored in-memory and optionally streamed as JSON lines to an `io.Writer` for +persistence. `Decision` marshals to/from string (`"allow"`, `"deny"`, +`"needs_approval"`). `EntriesFor(agent)` filters by agent name. + +### Dynamic Policy Configuration + +Policies can be loaded from JSON and applied at runtime: + +```go +engine.ApplyPoliciesFromFile("/etc/agent/policies.json") + +// Export current state +engine.ExportPolicies(os.Stdout) +``` + +JSON format: + +```json +{ + "policies": [ + { + "tier": 1, + "allowed": ["pr.create", "issue.comment"], + "denied": ["repo.push", "pr.merge"] + } + ] +} +``` + +`json.Decoder.DisallowUnknownFields()` is set during load to catch +configuration errors early. + +--- + +## Algorithm Reference + +| Component | Algorithm | Parameters | +|-----------|-----------|------------| +| KDF (primary) | Argon2id | Memory=64MB, Time=3, Parallelism=4, KeyLen=32 | +| KDF (alternative) | scrypt | N=32768, r=8, p=1 | +| KDF (expansion) | HKDF-SHA256 | Variable key length | +| Symmetric (primary) | ChaCha20-Poly1305 | 24-byte nonce (XChaCha20), 32-byte key | +| Symmetric (alternative) | AES-256-GCM | 12-byte nonce, 32-byte key | +| Password hash | Argon2id | Custom `$argon2id$` format string with random salt | +| Password hash (legacy) | LTHN quasi-salted SHA-256 | RFC-0004 (deterministic, no random salt) | +| Password hash (fallback) | Bcrypt | Configurable cost | +| Content ID | LTHN quasi-salted SHA-256 | RFC-0004 | +| Asymmetric | RSA-OAEP-SHA256 | 2048+ bit | +| PGP keypair | DSA primary + RSA subkey | ProtonMail go-crypto | +| PGP service | RSA-4096 + AES-256 + SHA-256 | core.Crypt interface | +| HMAC | HMAC-SHA256 / HMAC-SHA512 | Constant-time verify | +| Challenge nonce | crypto/rand | 32 bytes (256-bit) | +| Session token | crypto/rand | 32 bytes, hex-encoded (64 chars) | + +--- + +## Dependencies + +| Module | Version | Role | +|--------|---------|------| +| `forge.lthn.ai/core/go` | local | `core.E` error helper, `core.Crypt` interface, `io.Medium` storage | +| `forge.lthn.ai/core/go-store` | local | SQLite KV store for session persistence | +| `github.com/ProtonMail/go-crypto` | v1.3.0 | OpenPGP (actively maintained fork, post-quantum research) | +| `golang.org/x/crypto` | v0.48.0 | Argon2, ChaCha20-Poly1305, scrypt, HKDF, bcrypt | +| `github.com/cloudflare/circl` | v1.6.3 | Indirect; elliptic curves via ProtonMail | + +--- + +## Integration Points + +| Consumer | Package Used | Purpose | +|----------|-------------|---------| +| go-p2p | `crypt/` | UEPS consent-gated encryption | +| go-scm / AgentCI | `trust/` | Agent capability evaluation before CI operations | +| go-agentic | `auth/` | Agent session management | +| core/go | `crypt/openpgp/` | Service registered via `core.Crypt` interface | + +--- + +## Security Notes + +1. The LTHN hash (`crypt/lthn`) is **not** suitable for password hashing. It + is deterministic with no random salt. Use `crypt.HashPassword` (Argon2id). +2. PGP private keys are not zeroed after use. The ProtonMail `go-crypto` + library does not expose a `Wipe` method. This is a known upstream + limitation; mitigating it would require forking the library. +3. Empty `ScopedRepos` on a Tier 2 agent currently bypasses the repo scope + check (treated as unrestricted). Explicit `["*"]` or `["org/**"]` should be + required for unrestricted Tier 2 access if this design is revisited. +4. The `PolicyEngine` returns decisions but does not enforce the approval + workflow. A higher-level layer (go-agentic, go-scm) must handle the + `NeedsApproval` case by routing through the `ApprovalQueue`. +5. The `MemorySessionStore` is the default. Use `WithSessionStore(NewSQLiteSessionStore(path))` + for persistence across restarts. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..4357d91 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,197 @@ +# Development Guide — go-crypt + +## Prerequisites + +- Go 1.25 or later (the module declares `go 1.25.5`). +- A Go workspace (`go.work`) that resolves the local replace directives for + `forge.lthn.ai/core/go` (at `../go`) and `forge.lthn.ai/core/go-store` + (at `../go-store`). If you are working outside the full monorepo, edit + `go.mod` replace directives to point to your local checkouts. +- No C toolchain, CGo, or system libraries are required. + +## Build and Test Commands + +```bash +# Run all tests +go test ./... + +# Run with race detector (always use before committing) +go test -race ./... + +# Run a single test by name +go test -v -run TestName ./... + +# Run tests in a specific package +go test ./auth/... +go test ./crypt/... +go test ./trust/... + +# Static analysis +go vet ./... + +# Run benchmarks +go test -bench=. -benchmem ./crypt/... +go test -bench=. -benchmem ./trust/... +``` + +There is no build step — this is a library module with no binaries. The +`go vet ./...` check must pass cleanly before any commit. + +## Repository Layout + +``` +go-crypt/ +├── auth/ Authentication package +├── crypt/ Cryptographic utilities +│ ├── chachapoly/ Standalone ChaCha20-Poly1305 sub-package +│ ├── lthn/ RFC-0004 quasi-salted hash +│ ├── openpgp/ Service wrapper (core.Crypt interface) +│ ├── pgp/ OpenPGP primitives +│ └── rsa/ RSA OAEP-SHA256 +├── docs/ Architecture, development, and history docs +├── trust/ Agent trust model and policy engine +├── go.mod +└── go.sum +``` + +## Test Patterns + +Tests use the `github.com/stretchr/testify` library (`assert` and `require`). +The naming convention follows three suffixes: + +| Suffix | Purpose | +|--------|---------| +| `_Good` | Happy path — expected success | +| `_Bad` | Expected failure — invalid input, wrong credentials, not-found errors | +| `_Ugly` | Edge cases — panics, zero values, empty inputs, extreme lengths | + +Example: + +```go +func TestLogin_Good(t *testing.T) { ... } +func TestLogin_Bad(t *testing.T) { ... } +func TestLogin_Ugly(t *testing.T) { ... } +``` + +Concurrency tests use `t.Parallel()` and typically spawn 10 goroutines via a +`sync.WaitGroup`. The race detector (`-race`) must pass for all concurrent tests. + +## Benchmark Structure + +Benchmarks live in `bench_test.go` files alongside the packages they cover. +Benchmark names follow the `BenchmarkFuncName_Context` pattern: + +```go +func BenchmarkArgon2Derive(b *testing.B) { ... } +func BenchmarkChaCha20_1KB(b *testing.B) { ... } +func BenchmarkChaCha20_1MB(b *testing.B) { ... } +``` + +Run benchmarks with: + +```bash +go test -bench=. -benchmem -benchtime=3s ./crypt/... +``` + +Do not optimise without measuring first. The Argon2id KDF is intentionally slow +(~200ms on typical hardware) — this is a security property, not a defect. + +## Adding a New Cryptographic Primitive + +1. Add the implementation in the appropriate sub-package. +2. Write tests covering `_Good`, `_Bad`, and `_Ugly` cases. +3. Add a benchmark if the function is called on hot paths. +4. Update `docs/architecture.md` with the algorithm reference entry. +5. Run `go vet ./...` and `go test -race ./...` before committing. + +## Adding a New Trust Capability + +1. Add the `Capability` constant in `trust/trust.go`. +2. Update `isRepoScoped()` in `trust/policy.go` if the capability is + repository-scoped. +3. Update the default policies in `loadDefaults()` in `trust/policy.go`. +4. Add tests covering all three tiers. +5. Update the capability table in `docs/architecture.md`. + +## Coding Standards + +### Language + +UK English throughout: _colour_, _organisation_, _centre_, _artefact_, +_licence_ (noun), _license_ (verb), _behaviour_, _initialise_, _serialise_. + +### Go Style + +- `declare(strict_types=1)` is a PHP convention; Go has no equivalent. Use + explicit type assertions and avoid `any` except at interface boundaries. +- Every exported function and type must have a doc comment. +- Error strings are lowercase and do not end with a full stop, per Go convention. +- Use the `core.E(op, msg, err)` helper from `forge.lthn.ai/core/go` for + contextual error wrapping: `op` is `"package.Function"`, `msg` is a brief + lowercase description. +- Import groups: stdlib → `forge.lthn.ai/core` → third-party. Separate each + group with a blank line. + +### Cryptography + +- All randomness from `crypto/rand`. Never use `math/rand` for cryptographic + purposes. +- Use `crypto/subtle.ConstantTimeCompare` for any comparison of secret material + (MACs, hashes). The one exception is `lthn.Verify`, which compares content + identifiers (not secrets) and documents this explicitly. +- Never log or return secrets in error messages. Error strings should be generic: + `"invalid password"`, `"session not found"`, `"failed to decrypt"`. + +### Licence + +All files are licenced under EUPL-1.2. Do not add files under a different licence. + +## Commit Convention + +Commits follow the Conventional Commits specification: + +``` +type(scope): short imperative description + +Optional body explaining motivation and context. + +Co-Authored-By: Virgil +``` + +Types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`. + +Scopes match package names: `auth`, `crypt`, `trust`, `pgp`, `lthn`, `rsa`, +`openpgp`, `chachapoly`. + +Examples: + +``` +feat(auth): add SQLite session store for crash recovery +fix(trust): reject empty ScopedRepos as no-access for Tier 2 +test(crypt): add benchmark suite for Argon2 and ChaCha20 +``` + +## Forge Push + +The canonical remote is `forge.lthn.ai`. Push via SSH only; HTTPS authentication +is not configured: + +```bash +git push forge main +# remote: ssh://git@forge.lthn.ai:2223/core/go-crypt.git +``` + +## Local Replace Directives + +The `go.mod` contains: + +``` +replace ( + forge.lthn.ai/core/go => ../go + forge.lthn.ai/core/go-store => ../go-store +) +``` + +Do not modify these paths. If you need to work with a different local checkout, +use a Go workspace (`go.work`) at the parent directory level rather than editing +the replace directives directly. diff --git a/docs/history.md b/docs/history.md new file mode 100644 index 0000000..976ef36 --- /dev/null +++ b/docs/history.md @@ -0,0 +1,236 @@ +# Project History — go-crypt + +## Origin + +go-crypt was extracted from `forge.lthn.ai/core/go` on 16 February 2026 +(extraction commit `8498ecf`). The repository started with a single extraction +commit — no prior per-file history. The original implementation was ported from +`dAppServer`'s `mod-auth/lethean.service.ts` (TypeScript). + +At extraction the module contained ~1,938 source LOC across 14 files and ~1,770 +test LOC (47.7% test ratio). The `auth/` and `trust/` packages each had strong +test suites; `crypt/` sub-packages varied from well-tested (`chachapoly/`, `pgp/`) +to lightly covered (the top-level `crypt.go`). + +--- + +## Phase 0: Test Coverage and Hardening + +**Status**: Complete. + +**auth/ additions**: 8 new tests covering concurrent session creation (10 +goroutines), session token uniqueness (1,000 tokens), challenge expiry boundary, +empty password registration, very long username (10K characters), Unicode +username and password, air-gapped round-trip, and refresh of an already-expired +session. All pass under `-race`. + +**crypt/ additions**: 12 new tests covering wrong passphrase decryption +(ChaCha20 and AES), empty plaintext round-trip, 1MB payload round-trip (not 10MB +— kept fast), ciphertext-too-short rejection, key derivation determinism +(Argon2id and scrypt), HKDF with different info strings, HKDF with nil salt, +checksum of empty file (SHA-256 and SHA-512), checksum of non-existent file, and +checksum consistency with `SHA256Sum`. + +**trust/ additions**: 9 new tests covering concurrent Register/Get/Remove (10 +goroutines), Tier 0 and negative tier rejection, token expiry boundary, zero-value +token expiry, concurrent List during mutations, empty ScopedRepos behaviour +(documented as Finding F3), capability not in any list, and concurrent Evaluate. + +**Security audit**: Full review of all source files for cryptographic hygiene. +Findings documented below. `go vet ./...` produces no warnings. + +**Benchmark suite**: `crypt/bench_test.go` (7 benchmarks: Argon2Derive, +ChaCha20 1KB, ChaCha20 1MB, AESGCM 1KB, AESGCM 1MB, HMACSHA256 1KB, +VerifyHMACSHA256) and `trust/bench_test.go` (3 benchmarks: PolicyEvaluate 100 +agents, RegistryGet, RegistryRegister). + +--- + +## Phase 1: Session Persistence + +**Status**: Complete. Commit `1aeabfd`. + +Extracted the in-memory session map into a `SessionStore` interface with `Get`, +`Set`, `Delete`, `DeleteByUser`, and `Cleanup` methods. `ErrSessionNotFound` +sentinel error added. + +`MemorySessionStore` wraps the original map and mutex pattern. +`SQLiteSessionStore` is backed by go-store (SQLite KV). Sessions are stored as +JSON in a `"sessions"` group. A mutex serialises all operations for SQLite +single-writer safety. + +Background cleanup via `StartCleanup(ctx, interval)` goroutine. Stops on context +cancellation. All existing tests updated and passing. + +--- + +## Phase 2: Key Management + +**Status**: Complete. Commit `301eac1`. 55 tests total, all pass under `-race`. + +### Step 2.1: Password Hash Migration (resolves Finding F1) + +`Register` now uses `crypt.HashPassword()` (Argon2id) and writes a `.hash` file. +`Login` detects the hash format: tries `.hash` (Argon2id) first, falls back to +`.lthn` (LTHN). A successful legacy login transparently re-hashes with Argon2id +and writes the `.hash` file (best-effort). A shared `verifyPassword()` helper +handles the dual-path logic and is used by both `Login` and `RevokeKey`. + +The `verifyPassword` helper was extracted after `TestRevokeKey_Bad` failed: new +registrations do not write `.lthn` files, so the original fallback returned +`"user not found"` instead of `"invalid password"`. + +### Step 2.2: Key Rotation + +`RotateKeyPair(userID, oldPassword, newPassword)` implements full key rotation: +load private key → decrypt metadata with old password → generate new PGP keypair +→ re-encrypt metadata with new public key → overwrite `.pub`, `.key`, `.json`, +`.hash` → invalidate sessions via `store.DeleteByUser`. 4 tests. + +### Step 2.3: Key Revocation + +Chose JSON revocation record (Option B) over OpenPGP revocation certificate +(Option A). `Revocation{UserID, Reason, RevokedAt}` is written as JSON to `.rev`. +`IsRevoked()` parses JSON and ignores the legacy `"REVOCATION_PLACEHOLDER"` string. +Both `Login` and `CreateChallenge` reject revoked users. 6 tests including legacy +user revocation. + +### Step 2.4: Hardware Key Interface + +`hardware.go` defines the `HardwareKey` interface: `Sign`, `Decrypt`, +`GetPublicKey`, `IsAvailable`. Configured via `WithHardwareKey()` option. +Contract-only — no concrete implementations exist. Integration points documented +in `auth.go` comments but not wired. + +--- + +## Phase 3: Trust Policy Extensions + +**Status**: Complete. + +### Approval Workflow + +`ApprovalQueue` with `Submit`, `Approve`, `Deny`, `Get`, `Pending`, `Len` methods. +Thread-safe with unique monotonic IDs, status tracking, reviewer attribution, and +timestamps. 22 tests including concurrent and end-to-end integration with +`PolicyEngine`. + +### Audit Log + +`AuditLog` with append-only `Record`, `Entries`, `EntriesFor`, `Len` methods. +Optional `io.Writer` for JSON-line persistence. Custom `Decision` +`MarshalJSON`/`UnmarshalJSON`. 18 tests including writer errors and concurrent +logging. + +### Dynamic Policies + +`LoadPolicies`/`LoadPoliciesFromFile` parse JSON config. +`ApplyPolicies`/`ApplyPoliciesFromFile` replace engine policies. +`ExportPolicies` for round-trip serialisation. `DisallowUnknownFields` for strict +parsing. 18 tests including round-trip. + +### Scope Wildcards + +`matchScope` supports exact match, single-level wildcard (`core/*`), and +recursive wildcard (`core/**`). `repoAllowed` updated to use pattern matching. +18 tests covering all edge cases including integration with `PolicyEngine`. + +--- + +## Security Audit Findings + +Conducted 2026-02-20. Full audit reviewed all source files for cryptographic hygiene. + +### Finding F1: LTHN Hash Used for Password Verification (Medium) — RESOLVED + +`auth.Login()` originally verified passwords via `lthn.Verify()`, which uses the +LTHN quasi-salted hash (RFC-0004) with a non-constant-time string comparison. +LTHN was designed for content identifiers, not passwords, and provides no random +salt. Resolved in Phase 2 (commit `301eac1`) by migrating to Argon2id with +transparent legacy path migration. + +### Finding F2: PGP Private Keys Not Zeroed After Use (Low) — Open + +In `pgp.Decrypt()` and `pgp.Sign()`, the private key is decrypted into memory +via `entity.PrivateKey.Decrypt()` but the decrypted key material is not zeroed +before garbage collection. The ProtonMail `go-crypto` library does not expose a +`Wipe()` or `Zero()` method on `packet.PrivateKey`. Resolving this would require +forking or patching the upstream library. + +Severity is low: an attacker with read access to process memory already has full +access to the process. The Go runtime does not guarantee memory zeroing and +GC-managed runtimes inherently have this limitation. + +### Finding F3: Empty ScopedRepos Bypasses Scope Check on Tier 2 (Medium) — Open + +In `policy.go`, the repo scope check is conditioned on `len(agent.ScopedRepos) > 0`. +A Tier 2 agent with empty `ScopedRepos` (nil or `[]string{}`) is treated as +unrestricted rather than as having no access. If an admin registers a Tier 2 +agent without explicitly setting `ScopedRepos`, it gets access to all repositories +for repo-scoped capabilities (`repo.push`, `pr.create`, `pr.merge`, `secrets.read`). + +Potential remediation: treat empty `ScopedRepos` as no access for Tier 2 agents, +requiring explicit `["*"]` or `["org/**"]` for unrestricted access. This is a +design decision with backward-compatibility implications. + +### Finding F4: `go vet` Clean — Passed + +`go vet ./...` produces no warnings. All nonces use `crypto/rand`. No usage of +`math/rand` detected. No secrets in error messages. + +--- + +## Known Limitations + +### Dual ChaCha20 Implementations + +`crypt/symmetric.go` and `crypt/chachapoly/chachapoly.go` implement nearly +identical ChaCha20-Poly1305 AEAD. The `chachapoly` sub-package pre-allocates +capacity before appending, which is a minor optimisation for small payloads. The +two packages have different import paths and test suites. Consolidation would +reduce duplication but would require updating all importers. + +### LTHN Hash Non-Constant-Time Comparison + +`lthn.Verify()` uses direct string comparison (`==`), not +`subtle.ConstantTimeCompare`. This is acceptable because LTHN is for content +identifiers where timing attacks are not a realistic threat model. However, the +package comment and doc string document this explicitly to prevent misuse. + +### Policy Engine Does Not Enforce Workflow + +`PolicyEngine.Evaluate()` returns `NeedsApproval` as a decision value but +provides no enforcement. The caller is responsible for submitting to the +`ApprovalQueue` and polling for resolution. A higher-level package (go-agentic +or go-scm) must implement the actual enforcement layer. + +### Hardware Key Interface Is Contract-Only + +The `HardwareKey` interface in `auth/hardware.go` has no concrete implementations. +PKCS#11, YubiKey, and TPM backends are planned but not implemented. The +`Authenticator.hardwareKey` field is never consulted in the current code. + +### Session Cleanup Prints to Stdout + +`StartCleanup` logs via `fmt.Printf` rather than a structured logger. This is +acceptable for a library that does not want to impose a logging dependency, but +callers that need structured logs should wrap or replace the cleanup goroutine. + +--- + +## Future Considerations + +- **Consolidate ChaCha20 wrappers**: merge `crypt/symmetric.go` and + `crypt/chachapoly` into a single implementation. +- **Hardware key backends**: implement `HardwareKey` for PKCS#11 (via + `miekg/pkcs11` or `ThalesIgnite/crypto11`) and YubiKey (via `go-piv`). +- **Resolve Finding F3**: require explicit wildcard for unrestricted Tier 2 + access; treat empty `ScopedRepos` as no-access. +- **Structured logging**: replace `fmt.Printf` in `StartCleanup` with an + `slog.Logger` option on `Authenticator`. +- **Rate limiting enforcement**: the `Agent.RateLimit` field is stored in the + registry but never enforced. An enforcement layer (middleware, interceptor) + is needed in the consuming service. +- **Policy persistence**: `PolicyEngine` policies are in-memory only. A storage + backend (similar to `SQLiteSessionStore`) would allow runtime policy changes + to survive restarts.