From cf2af53ed318fbbc083972934a780907a0da7182 Mon Sep 17 00:00:00 2001 From: snider Date: Tue, 13 Jan 2026 17:26:21 +0000 Subject: [PATCH] feat: add RFC specifications and documentation for Borg project --- rfc/README.md | 40 ++ RFC-001-OSS-DRM.md => rfc/RFC-001-OSS-DRM.md | 0 rfc/RFC-002-SMSG-FORMAT.md | 480 +++++++++++++++++++ rfc/RFC-003-DATANODE.md | 326 +++++++++++++ rfc/RFC-004-TIM.md | 330 +++++++++++++ rfc/RFC-005-STIM.md | 303 ++++++++++++ rfc/RFC-006-TRIX.md | 342 +++++++++++++ rfc/RFC-007-LTHN.md | 355 ++++++++++++++ rfc/RFC-008-BORGFILE.md | 255 ++++++++++ rfc/RFC-009-STMF.md | 365 ++++++++++++++ rfc/RFC-010-WASM-API.md | 458 ++++++++++++++++++ 11 files changed, 3254 insertions(+) create mode 100644 rfc/README.md rename RFC-001-OSS-DRM.md => rfc/RFC-001-OSS-DRM.md (100%) create mode 100644 rfc/RFC-002-SMSG-FORMAT.md create mode 100644 rfc/RFC-003-DATANODE.md create mode 100644 rfc/RFC-004-TIM.md create mode 100644 rfc/RFC-005-STIM.md create mode 100644 rfc/RFC-006-TRIX.md create mode 100644 rfc/RFC-007-LTHN.md create mode 100644 rfc/RFC-008-BORGFILE.md create mode 100644 rfc/RFC-009-STMF.md create mode 100644 rfc/RFC-010-WASM-API.md diff --git a/rfc/README.md b/rfc/README.md new file mode 100644 index 0000000..697564d --- /dev/null +++ b/rfc/README.md @@ -0,0 +1,40 @@ +# Borg RFC Specifications + +This directory contains technical specifications (RFCs) for the Borg project. + +## Index + +| RFC | Title | Status | Description | +|-----|-------|--------|-------------| +| [001](RFC-001-OSS-DRM.md) | Open Source DRM | Proposed | Core DRM system for independent artists | +| [002](RFC-002-SMSG-FORMAT.md) | SMSG Container Format | Draft | Encrypted container format (v1/v2/v3) | +| [003](RFC-003-DATANODE.md) | DataNode | Draft | In-memory filesystem abstraction | +| [004](RFC-004-TIM.md) | Terminal Isolation Matrix | Draft | OCI-compatible container bundle | +| [005](RFC-005-STIM.md) | Encrypted TIM | Draft | ChaCha20-Poly1305 encrypted containers | +| [006](RFC-006-TRIX.md) | TRIX PGP Format | Draft | PGP encryption for archives and accounts | +| [007](RFC-007-LTHN.md) | LTHN Key Derivation | Draft | Rainbow-table resistant rolling keys | +| [008](RFC-008-BORGFILE.md) | Borgfile | Draft | Container compilation syntax | +| [009](RFC-009-STMF.md) | Secure To-Me Form | Draft | Asymmetric form encryption | +| [010](RFC-010-WASM-API.md) | WASM Decryption API | Draft | Browser decryption interface | + +## Status Definitions + +| Status | Meaning | +|--------|---------| +| **Draft** | Initial specification, subject to change | +| **Proposed** | Ready for review, implementation may begin | +| **Accepted** | Approved, implementation complete | +| **Deprecated** | Superseded by newer specification | + +## Contributing + +1. Create a new RFC with the next available number +2. Use the template format (see existing RFCs) +3. Start with "Draft" status +4. Update this README index + +## Related Documentation + +- [CLAUDE.md](../CLAUDE.md) - Developer quick reference +- [docs/](../docs/) - User documentation +- [examples/formats/](../examples/formats/) - Format examples diff --git a/RFC-001-OSS-DRM.md b/rfc/RFC-001-OSS-DRM.md similarity index 100% rename from RFC-001-OSS-DRM.md rename to rfc/RFC-001-OSS-DRM.md diff --git a/rfc/RFC-002-SMSG-FORMAT.md b/rfc/RFC-002-SMSG-FORMAT.md new file mode 100644 index 0000000..e0f13bb --- /dev/null +++ b/rfc/RFC-002-SMSG-FORMAT.md @@ -0,0 +1,480 @@ +# RFC-002: SMSG Container Format + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-001, RFC-007 + +--- + +## Abstract + +SMSG (Secure Message) is an encrypted container format using ChaCha20-Poly1305 authenticated encryption. This RFC specifies the binary wire format, versioning, and encoding rules for SMSG files. + +## 1. Overview + +SMSG provides: +- Authenticated encryption (ChaCha20-Poly1305) +- Public metadata (manifest) readable without decryption +- Multiple format versions (v1 legacy, v2 binary, v3 streaming) +- Optional chunking for large files and seeking + +## 2. File Structure + +### 2.1 Binary Layout + +``` +Offset Size Field +------ ----- ------------------------------------ +0 4 Magic: "SMSG" (ASCII) +4 2 Version: uint16 little-endian +6 3 Header Length: 3-byte big-endian +9 N Header JSON (plaintext) +9+N M Encrypted Payload +``` + +### 2.2 Magic Number + +| Format | Value | +|--------|-------| +| Binary | `0x53 0x4D 0x53 0x47` | +| ASCII | `SMSG` | +| Base64 (first 6 chars) | `U01TRw` | + +### 2.3 Version Field + +Current version: `0x0001` (1) + +Decoders MUST reject versions they don't understand. + +### 2.4 Header Length + +3 bytes, big-endian unsigned integer. Supports headers up to 16 MB. + +## 3. Header Format (JSON) + +Header is always plaintext (never encrypted), enabling metadata inspection without decryption. + +### 3.1 Base Header + +```json +{ + "version": "1.0", + "algorithm": "chacha20poly1305", + "format": "v2", + "compression": "zstd", + "manifest": { ... } +} +``` + +### 3.2 V3 Header Extensions + +```json +{ + "version": "1.0", + "algorithm": "chacha20poly1305", + "format": "v3", + "compression": "zstd", + "keyMethod": "lthn-rolling", + "cadence": "daily", + "manifest": { ... }, + "wrappedKeys": [ + {"date": "2026-01-13", "wrapped": ""}, + {"date": "2026-01-14", "wrapped": ""} + ], + "chunked": { + "chunkSize": 1048576, + "totalChunks": 42, + "totalSize": 44040192, + "index": [ + {"offset": 0, "size": 1048600}, + {"offset": 1048600, "size": 1048600} + ] + } +} +``` + +### 3.3 Header Field Reference + +| Field | Type | Values | Description | +|-------|------|--------|-------------| +| version | string | "1.0" | Format version string | +| algorithm | string | "chacha20poly1305" | Always ChaCha20-Poly1305 | +| format | string | "", "v2", "v3" | Payload format version | +| compression | string | "", "gzip", "zstd" | Compression algorithm | +| keyMethod | string | "", "lthn-rolling" | Key derivation method | +| cadence | string | "daily", "12h", "6h", "1h" | Rolling key period (v3) | +| manifest | object | - | Content metadata | +| wrappedKeys | array | - | CEK wrapped for each period (v3) | +| chunked | object | - | Chunk index for seeking (v3) | + +## 4. Manifest Structure + +### 4.1 Complete Manifest + +```go +type Manifest struct { + Title string `json:"title,omitempty"` + Artist string `json:"artist,omitempty"` + Album string `json:"album,omitempty"` + Genre string `json:"genre,omitempty"` + Year int `json:"year,omitempty"` + ReleaseType string `json:"release_type,omitempty"` + Duration int `json:"duration,omitempty"` + Format string `json:"format,omitempty"` + ExpiresAt int64 `json:"expires_at,omitempty"` + IssuedAt int64 `json:"issued_at,omitempty"` + LicenseType string `json:"license_type,omitempty"` + Tracks []Track `json:"tracks,omitempty"` + Links map[string]string `json:"links,omitempty"` + Tags []string `json:"tags,omitempty"` + Extra map[string]string `json:"extra,omitempty"` +} + +type Track struct { + Title string `json:"title"` + Start float64 `json:"start"` + End float64 `json:"end,omitempty"` + Type string `json:"type,omitempty"` + TrackNum int `json:"track_num,omitempty"` +} +``` + +### 4.2 Manifest Field Reference + +| Field | Type | Range | Description | +|-------|------|-------|-------------| +| title | string | 0-255 chars | Display name (required for discovery) | +| artist | string | 0-255 chars | Creator name | +| album | string | 0-255 chars | Album/collection name | +| genre | string | 0-255 chars | Genre classification | +| year | int | 0-9999 | Release year (0 = unset) | +| releaseType | string | enum | "single", "album", "ep", "mix" | +| duration | int | 0+ | Total duration in seconds | +| format | string | any | Platform format string (e.g., "dapp.fm/v1") | +| expiresAt | int64 | 0+ | Unix timestamp (0 = never expires) | +| issuedAt | int64 | 0+ | Unix timestamp of license issue | +| licenseType | string | enum | "perpetual", "rental", "stream", "preview" | +| tracks | []Track | - | Track boundaries for multi-track releases | +| links | map | - | Platform name → URL (e.g., "bandcamp" → URL) | +| tags | []string | - | Arbitrary string tags | +| extra | map | - | Free-form key-value extension data | + +## 5. Format Versions + +### 5.1 Version Comparison + +| Aspect | v1 (Legacy) | v2 (Binary) | v3 (Streaming) | +|--------|-------------|-------------|----------------| +| Payload Structure | JSON only | Length-prefixed JSON + binary | Same as v2 | +| Attachment Encoding | Base64 in JSON | Size field + raw binary | Size field + raw binary | +| Compression | None | zstd (default) | zstd (default) | +| Key Derivation | SHA256(password) | SHA256(password) | LTHN rolling keys | +| Chunked Support | No | No | Yes (optional) | +| Size Overhead | ~33% | ~25% | ~15% | +| Use Case | Legacy | General purpose | Time-limited streaming | + +### 5.2 V1 Format (Legacy) + +**Payload (after decryption):** + +```json +{ + "body": "Message content", + "subject": "Optional subject", + "from": "sender@example.com", + "to": "recipient@example.com", + "timestamp": 1673644800, + "attachments": [ + { + "name": "file.bin", + "content": "base64encodeddata==", + "mime": "application/octet-stream", + "size": 1024 + } + ], + "reply_key": { + "public_key": "base64x25519key==", + "algorithm": "x25519" + }, + "meta": { + "custom_field": "custom_value" + } +} +``` + +- Attachments base64-encoded inline in JSON (~33% overhead) +- Simple but inefficient for large files + +### 5.3 V2 Format (Binary) + +**Payload structure (after decryption and decompression):** + +``` +Offset Size Field +------ ----- ------------------------------------ +0 4 Message JSON Length (big-endian uint32) +4 N Message JSON (attachments have size only, no content) +4+N B1 Attachment 1 raw binary +4+N+B1 B2 Attachment 2 raw binary +... +``` + +**Message JSON (within payload):** + +```json +{ + "body": "Message text", + "subject": "Subject", + "from": "sender", + "attachments": [ + {"name": "file1.bin", "mime": "application/octet-stream", "size": 4096}, + {"name": "file2.bin", "mime": "image/png", "size": 65536} + ], + "timestamp": 1673644800 +} +``` + +- Attachment `content` field omitted; binary data follows JSON +- Compressed before encryption +- 3-10x faster than v1, ~25% smaller + +### 5.4 V3 Format (Streaming) + +Same payload structure as v2, but with: +- LTHN-derived rolling keys instead of password +- CEK (Content Encryption Key) wrapped for each time period +- Optional chunking for seek support + +**CEK Wrapping:** + +``` +For each rolling period: + streamKey = SHA256(LTHN(period:license:fingerprint)) + wrappedKey = ChaCha20-Poly1305(CEK, streamKey) +``` + +**Rolling Periods (cadence):** + +| Cadence | Period Format | Example | +|---------|---------------|---------| +| daily | YYYY-MM-DD | "2026-01-13" | +| 12h | YYYY-MM-DD-AM/PM | "2026-01-13-AM" | +| 6h | YYYY-MM-DD-HH | "2026-01-13-00", "2026-01-13-06" | +| 1h | YYYY-MM-DD-HH | "2026-01-13-15" | + +### 5.5 V3 Chunked Format + +**Payload (independently decryptable chunks):** + +``` +Offset Size Content +------ ----- ---------------------------------- +0 1048600 Chunk 0: [24-byte nonce][ciphertext][16-byte tag] +1048600 1048600 Chunk 1: [24-byte nonce][ciphertext][16-byte tag] +... +``` + +- Each chunk encrypted separately with same CEK, unique nonce +- Enables seeking, HTTP Range requests +- Chunk size typically 1MB (configurable) + +## 6. Encryption + +### 6.1 Algorithm + +XChaCha20-Poly1305 (extended nonce variant) + +| Parameter | Value | +|-----------|-------| +| Key size | 32 bytes | +| Nonce size | 24 bytes (XChaCha) | +| Tag size | 16 bytes | + +### 6.2 Ciphertext Structure + +``` +[24-byte XChaCha20 nonce][encrypted data][16-byte Poly1305 tag] +``` + +**Critical**: Nonces are embedded IN the ciphertext by the Enchantrix library, NOT transmitted separately in headers. + +### 6.3 Key Derivation + +**V1/V2 (Password-based):** + +```go +key := sha256.Sum256([]byte(password)) // 32 bytes +``` + +**V3 (LTHN Rolling):** + +```go +// For each period in rolling window: +streamKey := sha256.Sum256([]byte( + crypt.NewService().Hash(crypt.LTHN, period + ":" + license + ":" + fingerprint) +)) +``` + +## 7. Compression + +| Value | Algorithm | Notes | +|-------|-----------|-------| +| "" (empty) | None | Raw bytes, default for v1 | +| "gzip" | RFC 1952 | Stdlib, WASM compatible | +| "zstd" | Zstandard | Default for v2/v3, better ratio | + +**Order**: Compress → Encrypt (on write), Decrypt → Decompress (on read) + +## 8. Message Structure + +### 8.1 Go Types + +```go +type Message struct { + From string `json:"from,omitempty"` + To string `json:"to,omitempty"` + Subject string `json:"subject,omitempty"` + Body string `json:"body"` + Timestamp int64 `json:"timestamp,omitempty"` + Attachments []Attachment `json:"attachments,omitempty"` + ReplyKey *KeyInfo `json:"reply_key,omitempty"` + Meta map[string]string `json:"meta,omitempty"` +} + +type Attachment struct { + Name string `json:"name"` + Mime string `json:"mime"` + Size int `json:"size"` + Content string `json:"content,omitempty"` // Base64, v1 only + Data []byte `json:"-"` // Binary, v2/v3 +} + +type KeyInfo struct { + PublicKey string `json:"public_key"` + Algorithm string `json:"algorithm"` +} +``` + +### 8.2 Stream Parameters (V3) + +```go +type StreamParams struct { + License string `json:"license"` // User's license identifier + Fingerprint string `json:"fingerprint"` // Device fingerprint (optional) + Cadence string `json:"cadence"` // Rolling period: daily, 12h, 6h, 1h + ChunkSize int `json:"chunk_size"` // Bytes per chunk (default 1MB) +} +``` + +## 9. Error Handling + +### 9.1 Error Types + +```go +var ( + ErrInvalidMagic = errors.New("invalid SMSG magic") + ErrInvalidPayload = errors.New("invalid SMSG payload") + ErrDecryptionFailed = errors.New("decryption failed (wrong password?)") + ErrPasswordRequired = errors.New("password is required") + ErrEmptyMessage = errors.New("message cannot be empty") + ErrStreamKeyExpired = errors.New("stream key expired (outside rolling window)") + ErrNoValidKey = errors.New("no valid wrapped key found for current date") + ErrLicenseRequired = errors.New("license is required for stream decryption") +) +``` + +### 9.2 Error Conditions + +| Error | Cause | Recovery | +|-------|-------|----------| +| ErrInvalidMagic | File magic is not "SMSG" | Verify file format | +| ErrInvalidPayload | Corrupted payload structure | Re-download or restore | +| ErrDecryptionFailed | Wrong password or corrupted | Try correct password | +| ErrPasswordRequired | Empty password provided | Provide password | +| ErrStreamKeyExpired | Time outside rolling window | Wait for valid period or update file | +| ErrNoValidKey | No wrapped key for current period | License/fingerprint mismatch | +| ErrLicenseRequired | Empty StreamParams.License | Provide license identifier | + +## 10. Constants + +```go +const Magic = "SMSG" // 4 ASCII bytes +const Version = "1.0" // String version identifier +const DefaultChunkSize = 1024 * 1024 // 1 MB + +const FormatV1 = "" // Legacy JSON format +const FormatV2 = "v2" // Binary format +const FormatV3 = "v3" // Streaming with rolling keys + +const KeyMethodDirect = "" // Password-direct (v1/v2) +const KeyMethodLTHNRolling = "lthn-rolling" // LTHN rolling (v3) + +const CompressionNone = "" +const CompressionGzip = "gzip" +const CompressionZstd = "zstd" + +const CadenceDaily = "daily" +const CadenceHalfDay = "12h" +const CadenceQuarter = "6h" +const CadenceHourly = "1h" +``` + +## 11. API Usage + +### 11.1 V1 (Legacy) + +```go +msg := NewMessage("Hello").WithSubject("Test") +encrypted, _ := Encrypt(msg, "password") +decrypted, _ := Decrypt(encrypted, "password") +``` + +### 11.2 V2 (Binary) + +```go +msg := NewMessage("Hello").AddBinaryAttachment("file.bin", data, "application/octet-stream") +manifest := NewManifest("My Content") +encrypted, _ := EncryptV2WithManifest(msg, "password", manifest) +decrypted, _ := Decrypt(encrypted, "password") +``` + +### 11.3 V3 (Streaming) + +```go +msg := NewMessage("Stream content") +params := &StreamParams{ + License: "user-license", + Fingerprint: "device-fingerprint", + Cadence: CadenceDaily, + ChunkSize: 1048576, +} +manifest := NewManifest("Stream Track") +manifest.LicenseType = "stream" +encrypted, _ := EncryptV3(msg, params, manifest) +decrypted, header, _ := DecryptV3(encrypted, params) +``` + +## 12. Implementation Reference + +- Types: `pkg/smsg/types.go` +- Encryption: `pkg/smsg/smsg.go` +- Streaming: `pkg/smsg/stream.go` +- WASM: `pkg/wasm/stmf/main.go` +- Tests: `pkg/smsg/*_test.go` + +## 13. Security Considerations + +1. **Nonce uniqueness**: Enchantrix generates random 24-byte nonces automatically +2. **Key entropy**: Passwords should have 64+ bits entropy (no key stretching) +3. **Manifest exposure**: Manifest is public; never include sensitive data +4. **Constant-time crypto**: Enchantrix uses constant-time comparison for auth tags +5. **Rolling window**: V3 keys valid for current + next period only + +## 14. Future Work + +- [ ] Key stretching (Argon2 option) +- [ ] Multi-recipient encryption +- [ ] Streaming API with ReadableStream +- [ ] Hardware key support (WebAuthn) diff --git a/rfc/RFC-003-DATANODE.md b/rfc/RFC-003-DATANODE.md new file mode 100644 index 0000000..41eae36 --- /dev/null +++ b/rfc/RFC-003-DATANODE.md @@ -0,0 +1,326 @@ +# RFC-003: DataNode In-Memory Filesystem + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 + +--- + +## Abstract + +DataNode is an in-memory filesystem abstraction implementing Go's `fs.FS` interface. It provides the foundation for collecting, manipulating, and serializing file trees without touching disk. + +## 1. Overview + +DataNode serves as the core data structure for: +- Collecting files from various sources (GitHub, websites, PWAs) +- Building container filesystems (TIM rootfs) +- Serializing to/from tar archives +- Encrypting as TRIX format + +## 2. Implementation + +### 2.1 Core Type + +```go +type DataNode struct { + files map[string]*dataFile +} + +type dataFile struct { + name string + content []byte + modTime time.Time +} +``` + +**Key insight**: DataNode uses a **flat key-value map**, not a nested tree structure. Paths are stored as keys directly, and directories are implicit (derived from path prefixes). + +### 2.2 fs.FS Implementation + +DataNode implements these interfaces: + +| Interface | Method | Description | +|-----------|--------|-------------| +| `fs.FS` | `Open(name string)` | Returns fs.File for path | +| `fs.StatFS` | `Stat(name string)` | Returns fs.FileInfo | +| `fs.ReadDirFS` | `ReadDir(name string)` | Lists directory contents | + +### 2.3 Internal Helper Types + +```go +// File metadata +type dataFileInfo struct { + name string + size int64 + modTime time.Time +} +func (fi *dataFileInfo) Mode() fs.FileMode { return 0444 } // Read-only + +// Directory metadata +type dirInfo struct { + name string +} +func (di *dirInfo) Mode() fs.FileMode { return fs.ModeDir | 0555 } + +// File reader (implements fs.File) +type dataFileReader struct { + info *dataFileInfo + reader *bytes.Reader +} + +// Directory reader (implements fs.File) +type dirFile struct { + info *dirInfo + entries []fs.DirEntry + offset int +} +``` + +## 3. Operations + +### 3.1 Construction + +```go +// Create empty DataNode +node := datanode.New() + +// Returns: &DataNode{files: make(map[string]*dataFile)} +``` + +### 3.2 Adding Files + +```go +// Add file with content +node.AddData("path/to/file.txt", []byte("content")) + +// Trailing slashes are ignored (treated as directory indicator) +node.AddData("path/to/dir/", []byte("")) // Stored as "path/to/dir" +``` + +**Note**: Parent directories are NOT explicitly created. They are implicit based on path prefixes. + +### 3.3 File Access + +```go +// Open file (fs.FS interface) +f, err := node.Open("path/to/file.txt") +if err != nil { + // fs.ErrNotExist if not found +} +defer f.Close() +content, _ := io.ReadAll(f) + +// Stat file +info, err := node.Stat("path/to/file.txt") +// info.Name(), info.Size(), info.ModTime(), info.Mode() + +// Read directory +entries, err := node.ReadDir("path/to") +for _, entry := range entries { + // entry.Name(), entry.IsDir(), entry.Type() +} +``` + +### 3.4 Walking + +```go +err := fs.WalkDir(node, ".", func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if !d.IsDir() { + // Process file + } + return nil +}) +``` + +## 4. Path Semantics + +### 4.1 Path Handling + +- **Leading slashes stripped**: `/path/file` → `path/file` +- **Trailing slashes ignored**: `path/dir/` → `path/dir` +- **Forward slashes only**: Uses `/` regardless of OS +- **Case-sensitive**: `File.txt` ≠ `file.txt` +- **Direct lookup**: Paths stored as flat keys + +### 4.2 Valid Paths + +``` +file.txt → stored as "file.txt" +dir/file.txt → stored as "dir/file.txt" +/absolute/path → stored as "absolute/path" (leading / stripped) +path/to/dir/ → stored as "path/to/dir" (trailing / stripped) +``` + +### 4.3 Directory Detection + +Directories are **implicit**. A directory exists if: +1. Any file path has it as a prefix +2. Example: Adding `a/b/c.txt` implicitly creates directories `a` and `a/b` + +```go +// ReadDir finds directories by scanning all paths +func (dn *DataNode) ReadDir(name string) ([]fs.DirEntry, error) { + // Scans all keys for matching prefix + // Returns unique immediate children +} +``` + +## 5. Tar Serialization + +### 5.1 ToTar + +```go +tarBytes, err := node.ToTar() +``` + +**Format**: +- All files written as `tar.TypeReg` (regular files) +- Header Mode: **0600** (fixed, not original mode) +- No explicit directory entries +- ModTime preserved from dataFile + +```go +// Serialization logic +for path, file := range dn.files { + header := &tar.Header{ + Name: path, + Mode: 0600, // Fixed mode + Size: int64(len(file.content)), + ModTime: file.modTime, + Typeflag: tar.TypeReg, + } + tw.WriteHeader(header) + tw.Write(file.content) +} +``` + +### 5.2 FromTar + +```go +node, err := datanode.FromTar(tarBytes) +``` + +**Parsing**: +- Only reads `tar.TypeReg` entries +- Ignores directory entries (`tar.TypeDir`) +- Stores path and content in flat map + +```go +// Deserialization logic +for { + header, err := tr.Next() + if header.Typeflag == tar.TypeReg { + content, _ := io.ReadAll(tr) + dn.files[header.Name] = &dataFile{ + name: filepath.Base(header.Name), + content: content, + modTime: header.ModTime, + } + } +} +``` + +### 5.3 Compressed Variants + +```go +// gzip compressed +tarGz, err := node.ToTarGz() +node, err := datanode.FromTarGz(tarGzBytes) + +// xz compressed +tarXz, err := node.ToTarXz() +node, err := datanode.FromTarXz(tarXzBytes) +``` + +## 6. File Modes + +| Context | Mode | Notes | +|---------|------|-------| +| File read (fs.FS) | 0444 | Read-only for all | +| Directory (fs.FS) | 0555 | Read+execute for all | +| Tar export | 0600 | Owner read/write only | + +**Note**: Original file modes are NOT preserved. All files get fixed modes. + +## 7. Memory Model + +- All content held in memory as `[]byte` +- No lazy loading +- No memory mapping +- Thread-safe for concurrent reads (map is not mutated after creation) + +### 7.1 Size Calculation + +```go +func (dn *DataNode) Size() int64 { + var total int64 + for _, f := range dn.files { + total += int64(len(f.content)) + } + return total +} +``` + +## 8. Integration Points + +### 8.1 TIM RootFS + +```go +tim := &tim.TIM{ + Config: configJSON, + RootFS: datanode, // DataNode as container filesystem +} +``` + +### 8.2 TRIX Encryption + +```go +// Encrypt DataNode to TRIX +encrypted, err := trix.Encrypt(datanode.ToTar(), password) + +// Decrypt TRIX to DataNode +tarBytes, err := trix.Decrypt(encrypted, password) +node, err := datanode.FromTar(tarBytes) +``` + +### 8.3 Collectors + +```go +// GitHub collector returns DataNode +node, err := github.CollectRepo(url) + +// Website collector returns DataNode +node, err := website.Collect(url, depth) +``` + +## 9. Implementation Reference + +- Source: `pkg/datanode/datanode.go` +- Tests: `pkg/datanode/datanode_test.go` + +## 10. Security Considerations + +1. **Path traversal**: Leading slashes stripped; no `..` handling needed (flat map) +2. **Memory exhaustion**: No built-in limits; caller must validate input size +3. **Tar bombs**: FromTar reads all entries into memory +4. **Symlinks**: Not supported (intentional - tar.TypeReg only) + +## 11. Limitations + +- No symlink support +- No extended attributes +- No sparse files +- Fixed file modes (0600 on export) +- No streaming (full content in memory) + +## 12. Future Work + +- [ ] Streaming tar generation for large files +- [ ] Optional mode preservation +- [ ] Size limits for untrusted input +- [ ] Lazy loading for large datasets diff --git a/rfc/RFC-004-TIM.md b/rfc/RFC-004-TIM.md new file mode 100644 index 0000000..2e70d89 --- /dev/null +++ b/rfc/RFC-004-TIM.md @@ -0,0 +1,330 @@ +# RFC-004: Terminal Isolation Matrix (TIM) + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-003 + +--- + +## Abstract + +TIM (Terminal Isolation Matrix) is an OCI-compatible container bundle format. It packages a runtime configuration with a root filesystem (DataNode) for execution via runc or compatible runtimes. + +## 1. Overview + +TIM provides: +- OCI runtime-spec compatible bundles +- Portable container packaging +- Integration with DataNode filesystem +- Encryption via STIM (RFC-005) + +## 2. Implementation + +### 2.1 Core Type + +```go +// pkg/tim/tim.go:28-32 +type TerminalIsolationMatrix struct { + Config []byte // Raw OCI runtime specification (JSON) + RootFS *datanode.DataNode // In-memory filesystem +} +``` + +### 2.2 Error Variables + +```go +var ( + ErrDataNodeRequired = errors.New("datanode is required") + ErrConfigIsNil = errors.New("config is nil") + ErrPasswordRequired = errors.New("password is required for encryption") + ErrInvalidStimPayload = errors.New("invalid stim payload") + ErrDecryptionFailed = errors.New("decryption failed (wrong password?)") +) +``` + +## 3. Public API + +### 3.1 Constructors + +```go +// Create empty TIM with default config +func New() (*TerminalIsolationMatrix, error) + +// Wrap existing DataNode into TIM +func FromDataNode(dn *DataNode) (*TerminalIsolationMatrix, error) + +// Deserialize from tar archive +func FromTar(data []byte) (*TerminalIsolationMatrix, error) +``` + +### 3.2 Serialization + +```go +// Serialize to tar archive +func (m *TerminalIsolationMatrix) ToTar() ([]byte, error) + +// Encrypt to STIM format (ChaCha20-Poly1305) +func (m *TerminalIsolationMatrix) ToSigil(password string) ([]byte, error) +``` + +### 3.3 Decryption + +```go +// Decrypt from STIM format +func FromSigil(data []byte, password string) (*TerminalIsolationMatrix, error) +``` + +### 3.4 Execution + +```go +// Run plain .tim file with runc +func Run(timPath string) error + +// Decrypt and run .stim file +func RunEncrypted(stimPath, password string) error +``` + +## 4. Tar Archive Structure + +### 4.1 Layout + +``` +config.json (root level, mode 0600) +rootfs/ (directory, mode 0755) +rootfs/bin/app (files within rootfs/) +rootfs/etc/config +... +``` + +### 4.2 Serialization (ToTar) + +```go +// pkg/tim/tim.go:111-195 +func (m *TerminalIsolationMatrix) ToTar() ([]byte, error) { + // 1. Write config.json header (size = len(m.Config), mode 0600) + // 2. Write config.json content + // 3. Write rootfs/ directory entry (TypeDir, mode 0755) + // 4. Walk m.RootFS depth-first + // 5. For each file: tar entry with name "rootfs/" + path, mode 0600 +} +``` + +### 4.3 Deserialization (FromTar) + +```go +func FromTar(data []byte) (*TerminalIsolationMatrix, error) { + // 1. Parse tar entries + // 2. "config.json" → stored as raw bytes in Config + // 3. "rootfs/*" prefix → stripped and added to DataNode + // 4. Error if config.json missing (ErrConfigIsNil) +} +``` + +## 5. OCI Config + +### 5.1 Default Config + +The `New()` function creates a TIM with a default config from `pkg/tim/config.go`: + +```go +func defaultConfig() (*trix.Trix, error) { + return &trix.Trix{Header: make(map[string]interface{})}, nil +} +``` + +**Note**: The default config is minimal. Applications should populate the Config field with a proper OCI runtime spec. + +### 5.2 OCI Runtime Spec Example + +```json +{ + "ociVersion": "1.0.2", + "process": { + "terminal": false, + "user": {"uid": 0, "gid": 0}, + "args": ["/bin/app"], + "env": ["PATH=/usr/bin:/bin"], + "cwd": "/" + }, + "root": { + "path": "rootfs", + "readonly": true + }, + "mounts": [], + "linux": { + "namespaces": [ + {"type": "pid"}, + {"type": "network"}, + {"type": "mount"} + ] + } +} +``` + +## 6. Execution Flow + +### 6.1 Plain TIM (Run) + +```go +// pkg/tim/run.go:18-74 +func Run(timPath string) error { + // 1. Create temporary directory (borg-run-*) + // 2. Extract tar entry-by-entry + // - Security: Path traversal check (prevents ../) + // - Validates: target = Clean(target) within tempDir + // 3. Create directories as needed (0755) + // 4. Write files with 0600 permissions + // 5. Execute: runc run -b borg-container + // 6. Stream stdout/stderr directly + // 7. Return exit code +} +``` + +### 6.2 Encrypted TIM (RunEncrypted) + +```go +// pkg/tim/run.go:79-134 +func RunEncrypted(stimPath, password string) error { + // 1. Read encrypted .stim file + // 2. Decrypt using FromSigil() with password + // 3. Create temporary directory (borg-run-*) + // 4. Write config.json to tempDir + // 5. Create rootfs/ subdirectory + // 6. Walk DataNode and extract all files to rootfs/ + // - Uses CopyFile() with 0600 permissions + // 7. Execute: runc run -b borg-container + // 8. Stream stdout/stderr + // 9. Clean up temp directory (defer os.RemoveAll) + // 10. Return exit code +} +``` + +### 6.3 Security Controls + +| Control | Implementation | +|---------|----------------| +| Path traversal | `filepath.Clean()` + prefix validation | +| Temp cleanup | `defer os.RemoveAll(tempDir)` | +| File permissions | Hardcoded 0600 (files), 0755 (dirs) | +| Test injection | `ExecCommand` variable for mocking runc | + +## 7. Cache API + +### 7.1 Cache Structure + +```go +// pkg/tim/cache.go +type Cache struct { + Dir string // Directory path for storage + Password string // Shared password for all TIMs +} +``` + +### 7.2 Cache Operations + +```go +// Create cache with master password +func NewCache(dir, password string) (*Cache, error) + +// Store TIM (encrypted automatically as .stim) +func (c *Cache) Store(name string, m *TerminalIsolationMatrix) error + +// Load TIM (decrypted automatically) +func (c *Cache) Load(name string) (*TerminalIsolationMatrix, error) + +// Delete cached TIM +func (c *Cache) Delete(name string) error + +// Check if TIM exists +func (c *Cache) Exists(name string) bool + +// List all cached TIM names +func (c *Cache) List() ([]string, error) + +// Load and execute cached TIM +func (c *Cache) Run(name string) error + +// Get file size of cached .stim +func (c *Cache) Size(name string) (int64, error) +``` + +### 7.3 Cache Directory Structure + +``` +cache/ +├── mycontainer.stim (encrypted) +├── another.stim (encrypted) +└── ... +``` + +- All TIMs stored as `.stim` files (encrypted) +- Single password protects entire cache +- Directory created with 0700 permissions +- Files stored with 0600 permissions + +## 8. CLI Usage + +```bash +# Compile Borgfile to TIM +borg compile -f Borgfile -o container.tim + +# Compile with encryption +borg compile -f Borgfile -e "password" -o container.stim + +# Run plain TIM +borg run container.tim + +# Run encrypted TIM +borg run container.stim -p "password" + +# Decode (extract) to tar +borg decode container.stim -p "password" --i-am-in-isolation -o container.tar + +# Inspect metadata without decrypting +borg inspect container.stim +``` + +## 9. Implementation Reference + +- TIM core: `pkg/tim/tim.go` +- Execution: `pkg/tim/run.go` +- Cache: `pkg/tim/cache.go` +- Config: `pkg/tim/config.go` +- Tests: `pkg/tim/tim_test.go`, `pkg/tim/run_test.go`, `pkg/tim/cache_test.go` + +## 10. Security Considerations + +1. **Path traversal prevention**: `filepath.Clean()` + prefix validation +2. **Permission hardcoding**: 0600 files, 0755 directories +3. **Secure cleanup**: `defer os.RemoveAll()` on temp directories +4. **Command injection prevention**: `ExecCommand` variable (no shell) +5. **Config validation**: Validate OCI spec before execution + +## 11. OCI Compatibility + +TIM bundles are compatible with: +- runc +- crun +- youki +- Any OCI runtime-spec 1.0.2 compliant runtime + +## 12. Test Coverage + +| Area | Tests | +|------|-------| +| TIM creation | DataNode wrapping, default config | +| Serialization | Tar round-trips, large files (1MB+) | +| Encryption | ToSigil/FromSigil, wrong password detection | +| Caching | Store/Load/Delete, List, Size | +| Execution | ZIP slip prevention, temp cleanup | +| Error handling | Nil DataNode, nil config, invalid tar | + +## 13. Future Work + +- [ ] Image layer support +- [ ] Registry push/pull +- [ ] Multi-platform bundles +- [ ] Signature verification +- [ ] Full OCI config generation diff --git a/rfc/RFC-005-STIM.md b/rfc/RFC-005-STIM.md new file mode 100644 index 0000000..6f8012a --- /dev/null +++ b/rfc/RFC-005-STIM.md @@ -0,0 +1,303 @@ +# RFC-005: STIM Encrypted Container Format + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-003, RFC-004 + +--- + +## Abstract + +STIM (Secure TIM) is an encrypted container format that wraps TIM bundles using ChaCha20-Poly1305 authenticated encryption. It enables secure distribution and execution of containers without exposing the contents. + +## 1. Overview + +STIM provides: +- Encrypted TIM containers +- ChaCha20-Poly1305 authenticated encryption +- Separate encryption of config and rootfs +- Direct execution without persistent decryption + +## 2. Format Name + +**ChaChaPolySigil** - The internal name for the STIM format, using: +- ChaCha20-Poly1305 algorithm (via Enchantrix library) +- Trix container wrapper with "STIM" magic + +## 3. File Structure + +### 3.1 Container Format + +STIM uses the **Trix container format** from Enchantrix library: + +``` +┌─────────────────────────────────────────┐ +│ Magic: "STIM" (4 bytes ASCII) │ +├─────────────────────────────────────────┤ +│ Trix Header (Gob-encoded JSON) │ +│ - encryption_algorithm: "chacha20poly1305" +│ - tim: true │ +│ - config_size: uint32 │ +│ - rootfs_size: uint32 │ +│ - version: "1.0" │ +├─────────────────────────────────────────┤ +│ Trix Payload: │ +│ [config_size: 4 bytes BE uint32] │ +│ [encrypted config] │ +│ [encrypted rootfs tar] │ +└─────────────────────────────────────────┘ +``` + +### 3.2 Payload Structure + +``` +Offset Size Field +------ ----- ------------------------------------ +0 4 Config size (big-endian uint32) +4 N Encrypted config (includes nonce + tag) +4+N M Encrypted rootfs tar (includes nonce + tag) +``` + +### 3.3 Encrypted Component Format + +Each encrypted component (config and rootfs) follows Enchantrix format: + +``` +[24-byte XChaCha20 nonce][ciphertext][16-byte Poly1305 tag] +``` + +**Critical**: Nonces are **embedded in the ciphertext**, not transmitted separately. + +## 4. Encryption + +### 4.1 Algorithm + +XChaCha20-Poly1305 (extended nonce variant) + +| Parameter | Value | +|-----------|-------| +| Key size | 32 bytes | +| Nonce size | 24 bytes (embedded) | +| Tag size | 16 bytes | + +### 4.2 Key Derivation + +```go +// pkg/trix/trix.go:64-67 +func DeriveKey(password string) []byte { + hash := sha256.Sum256([]byte(password)) + return hash[:] // 32 bytes +} +``` + +### 4.3 Dual Encryption + +Config and RootFS are encrypted **separately** with independent nonces: + +```go +// pkg/tim/tim.go:217-232 +func (m *TerminalIsolationMatrix) ToSigil(password string) ([]byte, error) { + // 1. Derive key + key := trix.DeriveKey(password) + + // 2. Create sigil + sigil, _ := enchantrix.NewChaChaPolySigil(key) + + // 3. Encrypt config (generates fresh nonce automatically) + encConfig, _ := sigil.In(m.Config) + + // 4. Serialize rootfs to tar + rootfsTar, _ := m.RootFS.ToTar() + + // 5. Encrypt rootfs (generates different fresh nonce) + encRootFS, _ := sigil.In(rootfsTar) + + // 6. Build payload + payload := make([]byte, 4+len(encConfig)+len(encRootFS)) + binary.BigEndian.PutUint32(payload[:4], uint32(len(encConfig))) + copy(payload[4:4+len(encConfig)], encConfig) + copy(payload[4+len(encConfig):], encRootFS) + + // 7. Create Trix container with STIM magic + // ... +} +``` + +**Rationale for dual encryption:** +- Config can be decrypted separately for inspection +- Allows streaming decryption of large rootfs +- Independent nonces prevent any nonce reuse + +## 5. Decryption Flow + +```go +// pkg/tim/tim.go:255-308 +func FromSigil(data []byte, password string) (*TerminalIsolationMatrix, error) { + // 1. Decode Trix container with magic "STIM" + t, _ := trix.Decode(data, "STIM", nil) + + // 2. Derive key from password + key := trix.DeriveKey(password) + + // 3. Create sigil + sigil, _ := enchantrix.NewChaChaPolySigil(key) + + // 4. Parse payload: extract configSize from first 4 bytes + configSize := binary.BigEndian.Uint32(t.Payload[:4]) + + // 5. Validate bounds + if int(configSize) > len(t.Payload)-4 { + return nil, ErrInvalidStimPayload + } + + // 6. Extract encrypted components + encConfig := t.Payload[4 : 4+configSize] + encRootFS := t.Payload[4+configSize:] + + // 7. Decrypt config (nonce auto-extracted by Enchantrix) + config, err := sigil.Out(encConfig) + if err != nil { + return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err) + } + + // 8. Decrypt rootfs + rootfsTar, err := sigil.Out(encRootFS) + if err != nil { + return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err) + } + + // 9. Reconstruct DataNode from tar + rootfs, _ := datanode.FromTar(rootfsTar) + + return &TerminalIsolationMatrix{Config: config, RootFS: rootfs}, nil +} +``` + +## 6. Trix Header + +```go +Header: map[string]interface{}{ + "encryption_algorithm": "chacha20poly1305", + "tim": true, + "config_size": len(encConfig), + "rootfs_size": len(encRootFS), + "version": "1.0", +} +``` + +## 7. CLI Usage + +```bash +# Create encrypted container +borg compile -f Borgfile -e "password" -o container.stim + +# Run encrypted container +borg run container.stim -p "password" + +# Decode (extract) encrypted container +borg decode container.stim -p "password" --i-am-in-isolation -o container.tar + +# Inspect without decrypting (shows header metadata only) +borg inspect container.stim +# Output: +# Format: STIM +# encryption_algorithm: chacha20poly1305 +# config_size: 1234 +# rootfs_size: 567890 +``` + +## 8. Cache API + +```go +// Create cache with master password +cache, err := tim.NewCache("/path/to/cache", masterPassword) + +// Store TIM (encrypted automatically as .stim) +err := cache.Store("name", tim) + +// Load TIM (decrypted automatically) +tim, err := cache.Load("name") + +// List cached containers +names, err := cache.List() +``` + +## 9. Execution Security + +```go +// Secure execution flow +func RunEncrypted(path, password string) error { + // 1. Create secure temp directory + tmpDir, _ := os.MkdirTemp("", "borg-run-*") + defer os.RemoveAll(tmpDir) // Secure cleanup + + // 2. Read and decrypt + data, _ := os.ReadFile(path) + tim, _ := FromSigil(data, password) + + // 3. Extract to temp + tim.ExtractTo(tmpDir) + + // 4. Execute with runc + return runRunc(tmpDir) +} +``` + +## 10. Security Properties + +### 10.1 Confidentiality + +- Contents encrypted with ChaCha20-Poly1305 +- Password-derived key never stored +- Nonces are random, never reused + +### 10.2 Integrity + +- Poly1305 MAC prevents tampering +- Decryption fails if modified +- Separate MACs for config and rootfs + +### 10.3 Error Detection + +| Error | Cause | +|-------|-------| +| `ErrPasswordRequired` | Empty password provided | +| `ErrInvalidStimPayload` | Payload < 4 bytes or invalid size | +| `ErrDecryptionFailed` | Wrong password or corrupted data | + +## 11. Comparison to TRIX + +| Feature | STIM | TRIX | +|---------|------|------| +| Algorithm | ChaCha20-Poly1305 | PGP/AES or ChaCha | +| Content | TIM bundles | DataNode (raw files) | +| Structure | Dual encryption | Single blob | +| Magic | "STIM" | "TRIX" | +| Use case | Container execution | General encryption, accounts | + +STIM is for containers. TRIX is for general file encryption and accounts. + +## 12. Implementation Reference + +- Encryption: `pkg/tim/tim.go` (ToSigil, FromSigil) +- Key derivation: `pkg/trix/trix.go` (DeriveKey) +- Cache: `pkg/tim/cache.go` +- CLI: `cmd/run.go`, `cmd/decode.go`, `cmd/compile.go` +- Enchantrix: `github.com/Snider/Enchantrix` + +## 13. Security Considerations + +1. **Password strength**: Recommend 64+ bits entropy (12+ chars) +2. **Key derivation**: SHA-256 only (no stretching) - use strong passwords +3. **Memory handling**: Keys should be wiped after use +4. **Temp files**: Use tmpfs when available, secure wipe after +5. **Side channels**: Enchantrix uses constant-time crypto operations + +## 14. Future Work + +- [ ] Hardware key support (YubiKey, TPM) +- [ ] Key stretching (Argon2) +- [ ] Multi-recipient encryption +- [ ] Streaming decryption for large rootfs diff --git a/rfc/RFC-006-TRIX.md b/rfc/RFC-006-TRIX.md new file mode 100644 index 0000000..3ca093d --- /dev/null +++ b/rfc/RFC-006-TRIX.md @@ -0,0 +1,342 @@ +# RFC-006: TRIX PGP Encryption Format + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-003 + +--- + +## Abstract + +TRIX is a PGP-based encryption format for DataNode archives and account credentials. It provides symmetric and asymmetric encryption using OpenPGP standards and ChaCha20-Poly1305, enabling secure data exchange and identity management. + +## 1. Overview + +TRIX provides: +- PGP symmetric encryption for DataNode archives +- ChaCha20-Poly1305 modern encryption +- PGP armored keys for account/identity management +- Integration with Enchantrix library + +## 2. Public API + +### 2.1 Key Derivation + +```go +// pkg/trix/trix.go:64-67 +func DeriveKey(password string) []byte { + hash := sha256.Sum256([]byte(password)) + return hash[:] // 32 bytes +} +``` + +- Input: password string (any length) +- Output: 32-byte key (256 bits) +- Algorithm: SHA-256 hash of UTF-8 bytes +- Deterministic: identical passwords → identical keys + +### 2.2 Legacy PGP Encryption + +```go +// Encrypt DataNode to TRIX (PGP symmetric) +func ToTrix(dn *datanode.DataNode, password string) ([]byte, error) + +// Decrypt TRIX to DataNode (DISABLED for encrypted payloads) +func FromTrix(data []byte, password string) (*datanode.DataNode, error) +``` + +**Note**: `FromTrix` with a non-empty password returns error `"decryption disabled: cannot accept encrypted payloads"`. This is intentional to prevent accidental password use. + +### 2.3 Modern ChaCha20-Poly1305 Encryption + +```go +// Encrypt with ChaCha20-Poly1305 +func ToTrixChaCha(dn *datanode.DataNode, password string) ([]byte, error) + +// Decrypt ChaCha20-Poly1305 +func FromTrixChaCha(data []byte, password string) (*datanode.DataNode, error) +``` + +### 2.4 Error Variables + +```go +var ( + ErrPasswordRequired = errors.New("password is required for encryption") + ErrDecryptionFailed = errors.New("decryption failed (wrong password?)") +) +``` + +## 3. File Format + +### 3.1 Container Structure + +``` +[4 bytes] Magic: "TRIX" (ASCII) +[Variable] Gob-encoded Header (map[string]interface{}) +[Variable] Payload (encrypted or unencrypted tarball) +``` + +### 3.2 Header Examples + +**Unencrypted:** +```go +Header: map[string]interface{}{} // Empty map +``` + +**ChaCha20-Poly1305:** +```go +Header: map[string]interface{}{ + "encryption_algorithm": "chacha20poly1305", +} +``` + +### 3.3 ChaCha20-Poly1305 Payload + +``` +[24 bytes] XChaCha20 Nonce (embedded) +[N bytes] Encrypted tar archive +[16 bytes] Poly1305 authentication tag +``` + +**Note**: Nonces are embedded in the ciphertext by Enchantrix, not stored separately. + +## 4. Encryption Workflows + +### 4.1 ChaCha20-Poly1305 (Recommended) + +```go +// Encryption +func ToTrixChaCha(dn *datanode.DataNode, password string) ([]byte, error) { + // 1. Validate password is non-empty + if password == "" { + return nil, ErrPasswordRequired + } + + // 2. Serialize DataNode to tar + tarball, _ := dn.ToTar() + + // 3. Derive 32-byte key + key := DeriveKey(password) + + // 4. Create sigil and encrypt + sigil, _ := enchantrix.NewChaChaPolySigil(key) + encrypted, _ := sigil.In(tarball) // Generates nonce automatically + + // 5. Create Trix container + t := &trix.Trix{ + Header: map[string]interface{}{"encryption_algorithm": "chacha20poly1305"}, + Payload: encrypted, + } + + // 6. Encode with TRIX magic + return trix.Encode(t, "TRIX", nil) +} +``` + +### 4.2 Decryption + +```go +func FromTrixChaCha(data []byte, password string) (*datanode.DataNode, error) { + // 1. Validate password + if password == "" { + return nil, ErrPasswordRequired + } + + // 2. Decode TRIX container + t, _ := trix.Decode(data, "TRIX", nil) + + // 3. Derive key and decrypt + key := DeriveKey(password) + sigil, _ := enchantrix.NewChaChaPolySigil(key) + tarball, err := sigil.Out(t.Payload) // Extracts nonce, verifies MAC + if err != nil { + return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err) + } + + // 4. Deserialize DataNode + return datanode.FromTar(tarball) +} +``` + +### 4.3 Legacy PGP (Disabled Decryption) + +```go +func ToTrix(dn *datanode.DataNode, password string) ([]byte, error) { + tarball, _ := dn.ToTar() + + var payload []byte + if password != "" { + // PGP symmetric encryption + cryptService := crypt.NewService() + payload, _ = cryptService.SymmetricallyEncryptPGP([]byte(password), tarball) + } else { + payload = tarball + } + + t := &trix.Trix{Header: map[string]interface{}{}, Payload: payload} + return trix.Encode(t, "TRIX", nil) +} + +func FromTrix(data []byte, password string) (*datanode.DataNode, error) { + // Security: Reject encrypted payloads + if password != "" { + return nil, errors.New("decryption disabled: cannot accept encrypted payloads") + } + + t, _ := trix.Decode(data, "TRIX", nil) + return datanode.FromTar(t.Payload) +} +``` + +## 5. Enchantrix Library + +### 5.1 Dependencies + +```go +import ( + "github.com/Snider/Enchantrix/pkg/trix" // Container format + "github.com/Snider/Enchantrix/pkg/crypt" // PGP operations + "github.com/Snider/Enchantrix/pkg/enchantrix" // AEAD sigils +) +``` + +### 5.2 Trix Container + +```go +type Trix struct { + Header map[string]interface{} + Payload []byte +} + +func Encode(t *Trix, magic string, extra interface{}) ([]byte, error) +func Decode(data []byte, magic string, extra interface{}) (*Trix, error) +``` + +### 5.3 ChaCha20-Poly1305 Sigil + +```go +// Create sigil with 32-byte key +sigil, err := enchantrix.NewChaChaPolySigil(key) + +// Encrypt (generates random 24-byte nonce) +ciphertext, err := sigil.In(plaintext) + +// Decrypt (extracts nonce, verifies MAC) +plaintext, err := sigil.Out(ciphertext) +``` + +## 6. Account System Integration + +### 6.1 PGP Armored Keys + +``` +-----BEGIN PGP PUBLIC KEY BLOCK----- + +mQENBGX...base64... +-----END PGP PUBLIC KEY BLOCK----- +``` + +### 6.2 Key Storage + +``` +~/.borg/ +├── identity.pub # PGP public key (armored) +├── identity.key # PGP private key (armored, encrypted) +└── keyring/ # Trusted public keys +``` + +## 7. CLI Usage + +```bash +# Encrypt with TRIX (PGP symmetric) +borg collect github repo https://github.com/user/repo \ + --format trix \ + --password "password" + +# Decrypt unencrypted TRIX +borg decode archive.trix -o decoded.tar + +# Inspect without decrypting +borg inspect archive.trix +# Output: +# Format: TRIX +# encryption_algorithm: chacha20poly1305 (if present) +# Payload Size: N bytes +``` + +## 8. Format Comparison + +| Format | Extension | Algorithm | Use Case | +|--------|-----------|-----------|----------| +| `datanode` | `.tar` | None | Uncompressed archive | +| `tim` | `.tim` | None | Container bundle | +| `trix` | `.trix` | PGP/AES or ChaCha | Encrypted archives, accounts | +| `stim` | `.stim` | ChaCha20-Poly1305 | Encrypted containers | +| `smsg` | `.smsg` | ChaCha20-Poly1305 | Encrypted media | + +## 9. Security Analysis + +### 9.1 Key Derivation Limitations + +**Current implementation: SHA-256 (single round)** + +| Metric | Value | +|--------|-------| +| Algorithm | SHA-256 | +| Iterations | 1 | +| Salt | None | +| Key stretching | None | + +**Implications:** +- GPU brute force: ~10 billion guesses/second +- 8-character password: ~10 seconds to break +- Recommendation: Use 15+ character passwords + +### 9.2 ChaCha20-Poly1305 Properties + +| Property | Status | +|----------|--------| +| Authentication | Poly1305 MAC (16 bytes) | +| Key size | 256 bits | +| Nonce size | 192 bits (XChaCha) | +| Standard | RFC 7539 compliant | + +## 10. Test Coverage + +| Test | Description | +|------|-------------| +| DeriveKey length | Output is exactly 32 bytes | +| DeriveKey determinism | Same password → same key | +| DeriveKey uniqueness | Different passwords → different keys | +| ToTrix without password | Valid TRIX with "TRIX" magic | +| ToTrix with password | PGP encryption applied | +| FromTrix unencrypted | Round-trip preserves files | +| FromTrix password rejection | Returns error | +| ToTrixChaCha success | Valid TRIX created | +| ToTrixChaCha empty password | Returns ErrPasswordRequired | +| FromTrixChaCha round-trip | Preserves nested directories | +| FromTrixChaCha wrong password | Returns ErrDecryptionFailed | +| FromTrixChaCha large data | 1MB file processed | + +## 11. Implementation Reference + +- Source: `pkg/trix/trix.go` +- Tests: `pkg/trix/trix_test.go` +- Enchantrix: `github.com/Snider/Enchantrix v0.0.2` + +## 12. Security Considerations + +1. **Use strong passwords**: 15+ characters due to no key stretching +2. **Prefer ChaCha**: Use `ToTrixChaCha` over legacy PGP +3. **Key backup**: Securely backup private keys +4. **Interoperability**: TRIX files with GPG require password + +## 13. Future Work + +- [ ] Key stretching (Argon2 option in DeriveKey) +- [ ] Public key encryption support +- [ ] Signature support +- [ ] Key expiration metadata +- [ ] Multi-recipient encryption diff --git a/rfc/RFC-007-LTHN.md b/rfc/RFC-007-LTHN.md new file mode 100644 index 0000000..d573bc0 --- /dev/null +++ b/rfc/RFC-007-LTHN.md @@ -0,0 +1,355 @@ +# RFC-007: LTHN Key Derivation + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-002 + +--- + +## Abstract + +LTHN (Leet-Hash-Nonce) is a rainbow-table resistant key derivation function used for streaming DRM with time-limited access. It generates rolling keys that automatically expire without requiring revocation infrastructure. + +## 1. Overview + +LTHN provides: +- Rainbow-table resistant hashing +- Time-based key rolling +- Zero-trust key derivation (no key server) +- Configurable cadence (daily to hourly) + +## 2. Motivation + +Traditional DRM requires: +- Central key server +- License validation +- Revocation lists +- Network connectivity + +LTHN eliminates these by: +- Deriving keys from public information + secret +- Time-bounding keys automatically +- Making rainbow tables impractical +- Working completely offline + +## 3. Algorithm + +### 3.1 Core Function + +The LTHN hash is implemented in the Enchantrix library: + +```go +import "github.com/Snider/Enchantrix/pkg/crypt" + +cryptService := crypt.NewService() +lthnHash := cryptService.Hash(crypt.LTHN, input) +``` + +**LTHN formula**: +``` +LTHN(input) = SHA256(input || reverse_leet(input)) +``` + +Where `reverse_leet` performs bidirectional character substitution. + +### 3.2 Reverse Leet Mapping + +| Original | Leet | Bidirectional | +|----------|------|---------------| +| o | 0 | o ↔ 0 | +| l | 1 | l ↔ 1 | +| e | 3 | e ↔ 3 | +| a | 4 | a ↔ 4 | +| s | z | s ↔ z | +| t | 7 | t ↔ 7 | + +### 3.3 Example + +``` +Input: "2026-01-13:license:fp" +reverse_leet: "pf:3zn3ci1:31-10-6202" +Combined: "2026-01-13:license:fppf:3zn3ci1:31-10-6202" +Result: SHA256(combined) → 32-byte hash +``` + +## 4. Stream Key Derivation + +### 4.1 Implementation + +```go +// pkg/smsg/stream.go:49-60 +func DeriveStreamKey(date, license, fingerprint string) []byte { + input := fmt.Sprintf("%s:%s:%s", date, license, fingerprint) + cryptService := crypt.NewService() + lthnHash := cryptService.Hash(crypt.LTHN, input) + key := sha256.Sum256([]byte(lthnHash)) + return key[:] +} +``` + +### 4.2 Input Format + +``` +period:license:fingerprint + +Where: +- period: Time period identifier (see Cadence) +- license: User's license key (password) +- fingerprint: Device/browser fingerprint +``` + +### 4.3 Output + +32-byte key suitable for ChaCha20-Poly1305. + +## 5. Cadence + +### 5.1 Options + +| Cadence | Constant | Period Format | Example | Duration | +|---------|----------|---------------|---------|----------| +| Daily | `CadenceDaily` | `2006-01-02` | `2026-01-13` | 24h | +| 12-hour | `CadenceHalfDay` | `2006-01-02-AM/PM` | `2026-01-13-PM` | 12h | +| 6-hour | `CadenceQuarter` | `2006-01-02-HH` | `2026-01-13-12` | 6h | +| Hourly | `CadenceHourly` | `2006-01-02-HH` | `2026-01-13-15` | 1h | + +### 5.2 Period Calculation + +```go +// pkg/smsg/stream.go:73-119 +func GetCurrentPeriod(cadence Cadence) string { + return GetPeriodAt(time.Now(), cadence) +} + +func GetPeriodAt(t time.Time, cadence Cadence) string { + switch cadence { + case CadenceDaily: + return t.Format("2006-01-02") + case CadenceHalfDay: + suffix := "AM" + if t.Hour() >= 12 { + suffix = "PM" + } + return t.Format("2006-01-02") + "-" + suffix + case CadenceQuarter: + bucket := (t.Hour() / 6) * 6 + return fmt.Sprintf("%s-%02d", t.Format("2006-01-02"), bucket) + case CadenceHourly: + return fmt.Sprintf("%s-%02d", t.Format("2006-01-02"), t.Hour()) + } + return t.Format("2006-01-02") +} + +func GetNextPeriod(cadence Cadence) string { + return GetPeriodAt(time.Now().Add(GetCadenceDuration(cadence)), cadence) +} +``` + +### 5.3 Duration Mapping + +```go +func GetCadenceDuration(cadence Cadence) time.Duration { + switch cadence { + case CadenceDaily: + return 24 * time.Hour + case CadenceHalfDay: + return 12 * time.Hour + case CadenceQuarter: + return 6 * time.Hour + case CadenceHourly: + return 1 * time.Hour + } + return 24 * time.Hour +} +``` + +## 6. Rolling Windows + +### 6.1 Dual-Key Strategy + +At encryption time, CEK is wrapped with **two** keys: +1. Current period key +2. Next period key + +This creates a rolling validity window: + +``` +Time: 2026-01-13 23:30 (daily cadence) + +Valid keys: +- "2026-01-13:license:fp" (current period) +- "2026-01-14:license:fp" (next period) + +Window: 24-48 hours of validity +``` + +### 6.2 Key Wrapping + +```go +// pkg/smsg/stream.go:135-155 +func WrapCEK(cek []byte, streamKey []byte) (string, error) { + sigil := enchantrix.NewChaChaPolySigil() + wrapped, err := sigil.Seal(cek, streamKey) + if err != nil { + return "", err + } + return base64.StdEncoding.EncodeToString(wrapped), nil +} +``` + +**Wrapped format**: +``` +[24-byte nonce][encrypted CEK][16-byte auth tag] +→ base64 encoded for header storage +``` + +### 6.3 Key Unwrapping + +```go +// pkg/smsg/stream.go:157-170 +func UnwrapCEK(wrapped string, streamKey []byte) ([]byte, error) { + data, err := base64.StdEncoding.DecodeString(wrapped) + if err != nil { + return nil, err + } + sigil := enchantrix.NewChaChaPolySigil() + return sigil.Open(data, streamKey) +} +``` + +### 6.4 Decryption Flow + +```go +// pkg/smsg/stream.go:606-633 +func UnwrapCEKFromHeader(header *V3Header, params *StreamParams) ([]byte, error) { + // Try current period first + currentPeriod := GetCurrentPeriod(params.Cadence) + currentKey := DeriveStreamKey(currentPeriod, params.License, params.Fingerprint) + + for _, wk := range header.WrappedKeys { + cek, err := UnwrapCEK(wk.Key, currentKey) + if err == nil { + return cek, nil + } + } + + // Try next period (for clock skew) + nextPeriod := GetNextPeriod(params.Cadence) + nextKey := DeriveStreamKey(nextPeriod, params.License, params.Fingerprint) + + for _, wk := range header.WrappedKeys { + cek, err := UnwrapCEK(wk.Key, nextKey) + if err == nil { + return cek, nil + } + } + + return nil, ErrKeyExpired +} +``` + +## 7. V3 Header Format + +```go +type V3Header struct { + Format string `json:"format"` // "v3" + Manifest *Manifest `json:"manifest"` + WrappedKeys []WrappedKey `json:"wrappedKeys"` + Chunked *ChunkInfo `json:"chunked,omitempty"` +} + +type WrappedKey struct { + Period string `json:"period"` // e.g., "2026-01-13" + Key string `json:"key"` // base64-encoded wrapped CEK +} +``` + +## 8. Rainbow Table Resistance + +### 8.1 Why It Works + +Standard hash: +``` +SHA256("2026-01-13:license:fp") → predictable, precomputable +``` + +LTHN hash: +``` +LTHN("2026-01-13:license:fp") += SHA256("2026-01-13:license:fp" + reverse_leet("2026-01-13:license:fp")) += SHA256("2026-01-13:license:fp" + "pf:3zn3ci1:31-10-6202") +``` + +The salt is **derived from the input itself**, making precomputation impractical: +- Each unique input has a unique salt +- Cannot build rainbow tables without knowing all possible inputs +- Input space includes license keys (high entropy) + +### 8.2 Security Analysis + +| Attack | Mitigation | +|--------|------------| +| Rainbow tables | Input-derived salt makes precomputation infeasible | +| Brute force | License key entropy (64+ bits recommended) | +| Time oracle | Rolling window prevents precise timing attacks | +| Key sharing | Keys expire within cadence window | + +## 9. Zero-Trust Properties + +| Property | Implementation | +|----------|----------------| +| No key server | Keys derived locally from LTHN | +| Auto-expiration | Rolling periods invalidate old keys | +| No revocation | Keys naturally expire within cadence window | +| Device binding | Fingerprint in derivation input | +| User binding | License key in derivation input | + +## 10. Test Vectors + +From `pkg/smsg/stream_test.go`: + +```go +// Stream key generation +date := "2026-01-12" +license := "test-license" +fingerprint := "test-fp" +key := DeriveStreamKey(date, license, fingerprint) +// key is 32 bytes, deterministic + +// Period calculation at 2026-01-12 15:30:00 UTC +t := time.Date(2026, 1, 12, 15, 30, 0, 0, time.UTC) + +GetPeriodAt(t, CadenceDaily) // "2026-01-12" +GetPeriodAt(t, CadenceHalfDay) // "2026-01-12-PM" +GetPeriodAt(t, CadenceQuarter) // "2026-01-12-12" +GetPeriodAt(t, CadenceHourly) // "2026-01-12-15" + +// Next periods +// Daily: "2026-01-12" → "2026-01-13" +// 12h: "2026-01-12-PM" → "2026-01-13-AM" +// 6h: "2026-01-12-12" → "2026-01-12-18" +// 1h: "2026-01-12-15" → "2026-01-12-16" +``` + +## 11. Implementation Reference + +- Stream key derivation: `pkg/smsg/stream.go` +- LTHN hash: `github.com/Snider/Enchantrix/pkg/crypt` +- WASM bindings: `pkg/wasm/stmf/main.go` (decryptV3, unwrapCEK) +- Tests: `pkg/smsg/stream_test.go` + +## 12. Security Considerations + +1. **License entropy**: Recommend 64+ bits (12+ alphanumeric chars) +2. **Fingerprint stability**: Should be stable but not user-controllable +3. **Clock skew**: Rolling windows handle ±1 period drift +4. **Key exposure**: Derived keys valid only for one period + +## 13. References + +- RFC-002: SMSG Format (v3 streaming) +- RFC-001: OSS DRM (Section 3.4) +- RFC 8439: ChaCha20-Poly1305 +- Enchantrix: github.com/Snider/Enchantrix diff --git a/rfc/RFC-008-BORGFILE.md b/rfc/RFC-008-BORGFILE.md new file mode 100644 index 0000000..be4c978 --- /dev/null +++ b/rfc/RFC-008-BORGFILE.md @@ -0,0 +1,255 @@ +# RFC-008: Borgfile Compilation + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-003, RFC-004 + +--- + +## Abstract + +Borgfile is a declarative syntax for defining TIM container contents. It specifies how local files are mapped into the container filesystem, enabling reproducible container builds. + +## 1. Overview + +Borgfile provides: +- Dockerfile-like syntax for familiarity +- File mapping into containers +- Simple ADD directive +- Integration with TIM encryption + +## 2. File Format + +### 2.1 Location + +- Default: `Borgfile` in current directory +- Override: `borg compile -f path/to/Borgfile` + +### 2.2 Encoding + +- UTF-8 text +- Unix line endings (LF) +- No BOM + +## 3. Syntax + +### 3.1 Parsing Implementation + +```go +// cmd/compile.go:33-54 +lines := strings.Split(content, "\n") +for _, line := range lines { + parts := strings.Fields(line) // Whitespace-separated tokens + if len(parts) == 0 { + continue // Skip empty lines + } + switch parts[0] { + case "ADD": + // Process ADD directive + default: + return fmt.Errorf("unknown instruction: %s", parts[0]) + } +} +``` + +### 3.2 ADD Directive + +``` +ADD +``` + +| Parameter | Description | +|-----------|-------------| +| source | Local path (relative to current working directory) | +| destination | Container path (leading slash stripped) | + +### 3.3 Examples + +```dockerfile +# Add single file +ADD ./app /usr/local/bin/app + +# Add configuration +ADD ./config.yaml /etc/myapp/config.yaml + +# Multiple files +ADD ./bin/server /app/server +ADD ./static /app/static +``` + +## 4. Path Resolution + +### 4.1 Source Paths + +- Resolved relative to **current working directory** (not Borgfile location) +- Must exist at compile time +- Read via `os.ReadFile(src)` + +### 4.2 Destination Paths + +- Leading slash stripped: `strings.TrimPrefix(dest, "/")` +- Added to DataNode as-is + +```go +// cmd/compile.go:46-50 +data, err := os.ReadFile(src) +if err != nil { + return fmt.Errorf("invalid ADD instruction: %s", line) +} +name := strings.TrimPrefix(dest, "/") +m.RootFS.AddData(name, data) +``` + +## 5. File Handling + +### 5.1 Permissions + +**Current implementation**: Permissions are NOT preserved. + +| Source | Container | +|--------|-----------| +| Any file | 0600 (hardcoded in DataNode.ToTar) | +| Any directory | 0755 (implicit) | + +### 5.2 Timestamps + +- Set to `time.Now()` when added to DataNode +- Original timestamps not preserved + +### 5.3 File Types + +- Regular files only +- No directory recursion (each file must be added explicitly) +- No symlink following + +## 6. Error Handling + +| Error | Cause | +|-------|-------| +| `invalid ADD instruction: {line}` | Wrong number of arguments | +| `os.ReadFile` error | Source file not found | +| `unknown instruction: {name}` | Unrecognized directive | +| `ErrPasswordRequired` | Encryption requested without password | + +## 7. CLI Flags + +```go +// cmd/compile.go:80-82 +-f, --file string Path to Borgfile (default: "Borgfile") +-o, --output string Output path (default: "a.tim") +-e, --encrypt string Password for .stim encryption (optional) +``` + +## 8. Output Formats + +### 8.1 Plain TIM + +```bash +borg compile -f Borgfile -o container.tim +``` + +Output: Standard TIM tar archive with `config.json` + `rootfs/` + +### 8.2 Encrypted STIM + +```bash +borg compile -f Borgfile -e "password" -o container.stim +``` + +Output: ChaCha20-Poly1305 encrypted STIM container + +**Auto-detection**: If `-e` flag provided, output automatically uses `.stim` format even if `-o` specifies `.tim`. + +## 9. Default OCI Config + +The current implementation creates a minimal config: + +```go +// pkg/tim/config.go:6-10 +func defaultConfig() (*trix.Trix, error) { + return &trix.Trix{Header: make(map[string]interface{})}, nil +} +``` + +**Note**: This is a placeholder. For full OCI runtime execution, you'll need to provide a proper `config.json` in the container or modify the TIM after compilation. + +## 10. Compilation Process + +``` +1. Read Borgfile content +2. Parse line-by-line +3. For each ADD directive: + a. Read source file from filesystem + b. Strip leading slash from destination + c. Add to DataNode +4. Create TIM with default config + populated RootFS +5. If password provided: + a. Encrypt to STIM via ToSigil() + b. Adjust output extension to .stim +6. Write output file +``` + +## 11. Implementation Reference + +- Parser/Compiler: `cmd/compile.go` +- TIM creation: `pkg/tim/tim.go` +- DataNode: `pkg/datanode/datanode.go` +- Tests: `cmd/compile_test.go` + +## 12. Current Limitations + +| Feature | Status | +|---------|--------| +| Comment support (`#`) | Not implemented | +| Quoted paths | Not implemented | +| Directory recursion | Not implemented | +| Permission preservation | Not implemented | +| Path resolution relative to Borgfile | Not implemented (uses CWD) | +| Full OCI config generation | Not implemented (empty header) | +| Symlink following | Not implemented | + +## 13. Examples + +### 13.1 Simple Application + +```dockerfile +ADD ./myapp /usr/local/bin/myapp +ADD ./config.yaml /etc/myapp/config.yaml +``` + +### 13.2 Web Application + +```dockerfile +ADD ./server /app/server +ADD ./index.html /app/static/index.html +ADD ./style.css /app/static/style.css +ADD ./app.js /app/static/app.js +``` + +### 13.3 With Encryption + +```bash +# Create Borgfile +cat > Borgfile << 'EOF' +ADD ./secret-app /app/secret-app +ADD ./credentials.json /etc/app/credentials.json +EOF + +# Compile with encryption +borg compile -f Borgfile -e "MySecretPassword123" -o secret.stim +``` + +## 14. Future Work + +- [ ] Comment support (`#`) +- [ ] Quoted path support for spaces +- [ ] Directory recursion in ADD +- [ ] Permission preservation +- [ ] Path resolution relative to Borgfile location +- [ ] Full OCI config generation +- [ ] Variable substitution (`${VAR}`) +- [ ] Include directive +- [ ] Glob patterns in source +- [ ] COPY directive (alias for ADD) diff --git a/rfc/RFC-009-STMF.md b/rfc/RFC-009-STMF.md new file mode 100644 index 0000000..c1903dd --- /dev/null +++ b/rfc/RFC-009-STMF.md @@ -0,0 +1,365 @@ +# RFC-009: STMF Secure To-Me Form + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 + +--- + +## Abstract + +STMF (Secure To-Me Form) provides asymmetric encryption for web form submissions. It enables end-to-end encrypted form data where only the recipient can decrypt submissions, protecting sensitive data from server compromise. + +## 1. Overview + +STMF provides: +- Asymmetric encryption for form data +- X25519 key exchange +- ChaCha20-Poly1305 for payload encryption +- Browser-based encryption via WASM +- HTTP middleware for server-side decryption + +## 2. Cryptographic Primitives + +### 2.1 Key Exchange + +X25519 (Curve25519 Diffie-Hellman) + +| Parameter | Value | +|-----------|-------| +| Private key | 32 bytes | +| Public key | 32 bytes | +| Shared secret | 32 bytes | + +### 2.2 Encryption + +ChaCha20-Poly1305 + +| Parameter | Value | +|-----------|-------| +| Key | 32 bytes (SHA-256 of shared secret) | +| Nonce | 24 bytes (XChaCha variant) | +| Tag | 16 bytes | + +## 3. Protocol + +### 3.1 Setup (One-time) + +``` +Recipient (Server): +1. Generate X25519 keypair +2. Publish public key (embed in page or API) +3. Store private key securely +``` + +### 3.2 Encryption Flow (Browser) + +``` +1. Fetch recipient's public key +2. Generate ephemeral X25519 keypair +3. Compute shared secret: X25519(ephemeral_private, recipient_public) +4. Derive encryption key: SHA256(shared_secret) +5. Encrypt form data: ChaCha20-Poly1305(data, key, random_nonce) +6. Send: {ephemeral_public, nonce, ciphertext} +``` + +### 3.3 Decryption Flow (Server) + +``` +1. Receive {ephemeral_public, nonce, ciphertext} +2. Compute shared secret: X25519(recipient_private, ephemeral_public) +3. Derive encryption key: SHA256(shared_secret) +4. Decrypt: ChaCha20-Poly1305_Open(ciphertext, key, nonce) +``` + +## 4. Wire Format + +### 4.1 Container (Trix-based) + +``` +[Magic: "STMF" (4 bytes)] +[Header: Gob-encoded JSON] +[Payload: ChaCha20-Poly1305 ciphertext] +``` + +### 4.2 Header Structure + +```json +{ + "version": "1.0", + "algorithm": "x25519-chacha20poly1305", + "ephemeral_pk": "" +} +``` + +### 4.3 Transmission + +- Default form field: `_stmf_payload` +- Encoding: Base64 string +- Content-Type: `application/x-www-form-urlencoded` or `multipart/form-data` + +## 5. Data Structures + +### 5.1 FormField + +```go +type FormField struct { + Name string // Field name + Value string // Base64 for files, plaintext otherwise + Type string // "text", "password", "file" + Filename string // For file uploads + MimeType string // For file uploads +} +``` + +### 5.2 FormData + +```go +type FormData struct { + Fields []FormField // Array of form fields + Metadata map[string]string // Arbitrary key-value metadata +} +``` + +### 5.3 Builder Pattern + +```go +formData := NewFormData(). + AddField("email", "user@example.com"). + AddFieldWithType("password", "secret", "password"). + AddFile("document", base64Content, "report.pdf", "application/pdf"). + SetMetadata("timestamp", time.Now().String()) +``` + +## 6. Key Management API + +### 6.1 Key Generation + +```go +// pkg/stmf/keypair.go +func GenerateKeyPair() (*KeyPair, error) + +type KeyPair struct { + privateKey *ecdh.PrivateKey + publicKey *ecdh.PublicKey +} +``` + +### 6.2 Key Loading + +```go +// From raw bytes +func LoadPublicKey(data []byte) (*ecdh.PublicKey, error) +func LoadPrivateKey(data []byte) (*ecdh.PrivateKey, error) + +// From base64 +func LoadPublicKeyBase64(encoded string) (*ecdh.PublicKey, error) +func LoadPrivateKeyBase64(encoded string) (*ecdh.PrivateKey, error) + +// Reconstruct keypair from private key +func LoadKeyPair(privateKeyBytes []byte) (*KeyPair, error) +``` + +### 6.3 Key Export + +```go +func (kp *KeyPair) PublicKey() []byte // Raw 32 bytes +func (kp *KeyPair) PrivateKey() []byte // Raw 32 bytes +func (kp *KeyPair) PublicKeyBase64() string // Base64 encoded +func (kp *KeyPair) PrivateKeyBase64() string // Base64 encoded +``` + +## 7. WASM API + +### 7.1 BorgSTMF Namespace + +```javascript +// Generate X25519 keypair +const keypair = await BorgSTMF.generateKeyPair(); +// keypair.publicKey: base64 string +// keypair.privateKey: base64 string + +// Encrypt form data +const encrypted = await BorgSTMF.encrypt( + JSON.stringify(formData), + serverPublicKeyBase64 +); + +// Encrypt with field-level control +const encrypted = await BorgSTMF.encryptFields( + {email: "user@example.com", password: "secret"}, + serverPublicKeyBase64, + {timestamp: Date.now().toString()} // Optional metadata +); +``` + +## 8. HTTP Middleware + +### 8.1 Simple Usage + +```go +import "github.com/Snider/Borg/pkg/stmf/middleware" + +// Create middleware with private key +mw := middleware.Simple(privateKeyBytes) + +// Or from base64 +mw, err := middleware.SimpleBase64(privateKeyB64) + +// Apply to handler +http.Handle("/submit", mw(myHandler)) +``` + +### 8.2 Advanced Configuration + +```go +cfg := middleware.DefaultConfig(privateKeyBytes) +cfg.FieldName = "_custom_field" // Custom field name (default: _stmf_payload) +cfg.PopulateForm = &true // Auto-populate r.Form +cfg.OnError = customErrorHandler // Custom error handling +cfg.OnMissingPayload = customHandler // When field is absent + +mw := middleware.Middleware(cfg) +``` + +### 8.3 Context Access + +```go +func myHandler(w http.ResponseWriter, r *http.Request) { + // Get decrypted form data + formData := middleware.GetFormData(r) + + // Get metadata + metadata := middleware.GetMetadata(r) + + // Access fields + email := formData.Get("email") + password := formData.Get("password") +} +``` + +### 8.4 Middleware Behavior + +- Handles POST, PUT, PATCH requests only +- Parses multipart/form-data (32 MB limit) or application/x-www-form-urlencoded +- Looks for field `_stmf_payload` (configurable) +- Base64 decodes, then decrypts +- Populates `r.Form` and `r.PostForm` with decrypted fields +- Returns 400 Bad Request on decryption failure + +## 9. Integration Example + +### 9.1 HTML Form + +```html +
+ + + + +
+ + +``` + +### 9.2 Server Handler + +```go +func main() { + privateKey, _ := os.ReadFile("private.key") + mw := middleware.Simple(privateKey) + + http.Handle("/api/submit", mw(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + formData := middleware.GetFormData(r) + + name := formData.Get("name") + email := formData.Get("email") + ssn := formData.Get("ssn") + + // Process securely... + w.WriteHeader(http.StatusOK) + }))) + + http.ListenAndServeTLS(":443", "cert.pem", "key.pem", nil) +} +``` + +## 10. Security Properties + +### 10.1 Forward Secrecy + +- Fresh ephemeral keypair per encryption +- Compromised private key doesn't decrypt past messages +- Each ciphertext has unique shared secret + +### 10.2 Authenticity + +- Poly1305 MAC prevents tampering +- Decryption fails if ciphertext modified + +### 10.3 Confidentiality + +- ChaCha20 provides 256-bit security +- Nonces are random (24 bytes), collision unlikely +- Data encrypted before leaving browser + +### 10.4 Key Isolation + +- Private key never exposed to browser/JavaScript +- Public key can be safely distributed +- Ephemeral keys discarded after encryption + +## 11. Error Handling + +```go +var ( + ErrInvalidMagic = errors.New("invalid STMF magic") + ErrInvalidPayload = errors.New("invalid STMF payload") + ErrDecryptionFailed = errors.New("decryption failed") + ErrInvalidPublicKey = errors.New("invalid public key") + ErrInvalidPrivateKey = errors.New("invalid private key") + ErrKeyGenerationFailed = errors.New("key generation failed") +) +``` + +## 12. Implementation Reference + +- Types: `pkg/stmf/types.go` +- Key management: `pkg/stmf/keypair.go` +- Encryption: `pkg/stmf/encrypt.go` +- Decryption: `pkg/stmf/decrypt.go` +- Middleware: `pkg/stmf/middleware/http.go` +- WASM: `pkg/wasm/stmf/main.go` + +## 13. Security Considerations + +1. **Public key authenticity**: Verify public key source (HTTPS, pinning) +2. **Private key protection**: Never expose to browser, store securely +3. **Nonce uniqueness**: Random generation ensures uniqueness +4. **HTTPS required**: Transport layer must be encrypted + +## 14. Future Work + +- [ ] Multiple recipients +- [ ] Key attestation +- [ ] Offline decryption app +- [ ] Hardware key support (WebAuthn) +- [ ] Key rotation support diff --git a/rfc/RFC-010-WASM-API.md b/rfc/RFC-010-WASM-API.md new file mode 100644 index 0000000..eb4925b --- /dev/null +++ b/rfc/RFC-010-WASM-API.md @@ -0,0 +1,458 @@ +# RFC-010: WASM Decryption API + +**Status**: Draft +**Author**: [Snider](https://github.com/Snider/) +**Created**: 2026-01-13 +**License**: EUPL-1.2 +**Depends On**: RFC-002, RFC-007, RFC-009 + +--- + +## Abstract + +This RFC specifies the WebAssembly (WASM) API for browser-based decryption of SMSG content and STMF form encryption. The API is exposed through two JavaScript namespaces: `BorgSMSG` for content decryption and `BorgSTMF` for form encryption. + +## 1. Overview + +The WASM module provides: +- SMSG decryption (v1, v2, v3, chunked, ABR) +- SMSG encryption +- STMF form encryption/decryption +- Metadata extraction without decryption + +## 2. Module Loading + +### 2.1 Files Required + +``` +stmf.wasm (~5.9MB) Compiled Go WASM module +wasm_exec.js (~20KB) Go WASM runtime +``` + +### 2.2 Initialization + +```html + + +``` + +### 2.3 Ready Event + +```javascript +document.addEventListener('borgstmf:ready', (event) => { + console.log('WASM ready, version:', event.detail.version); +}); +``` + +## 3. BorgSMSG Namespace + +### 3.1 Version + +```javascript +BorgSMSG.version // "1.6.0" +BorgSMSG.ready // true when loaded +``` + +### 3.2 Metadata Functions + +#### getInfo(base64) → Promise + +Get manifest without decryption. + +```javascript +const info = await BorgSMSG.getInfo(base64Content); +// info.version, info.algorithm, info.format +// info.manifest.title, info.manifest.artist +// info.isV3Streaming, info.isChunked +// info.wrappedKeys (for v3) +``` + +#### getInfoBinary(uint8Array) → Promise + +Binary input variant (no base64 decode needed). + +```javascript +const bytes = new Uint8Array(await response.arrayBuffer()); +const info = await BorgSMSG.getInfoBinary(bytes); +``` + +### 3.3 Decryption Functions + +#### decrypt(base64, password) → Promise + +Full decryption (v1 format, base64 attachments). + +```javascript +const msg = await BorgSMSG.decrypt(base64Content, password); +// msg.body, msg.subject, msg.from +// msg.attachments[0].name, .content (base64), .mime +``` + +#### decryptStream(base64, password) → Promise + +Streaming decryption (v2 format, binary attachments). + +```javascript +const msg = await BorgSMSG.decryptStream(base64Content, password); +// msg.attachments[0].data (Uint8Array) +// msg.attachments[0].mime +``` + +#### decryptBinary(uint8Array, password) → Promise + +Binary input, binary output. + +```javascript +const bytes = new Uint8Array(await fetch(url).then(r => r.arrayBuffer())); +const msg = await BorgSMSG.decryptBinary(bytes, password); +``` + +#### quickDecrypt(base64, password) → Promise + +Returns body text only (fast path). + +```javascript +const body = await BorgSMSG.quickDecrypt(base64Content, password); +``` + +### 3.4 V3 Streaming Functions + +#### decryptV3(base64, params) → Promise + +Decrypt v3 streaming content with LTHN rolling keys. + +```javascript +const msg = await BorgSMSG.decryptV3(base64Content, { + license: "user-license-key", + fingerprint: "device-fingerprint" // optional +}); +``` + +#### getV3ChunkInfo(base64) → Promise + +Get chunk index for seeking without full decrypt. + +```javascript +const chunkInfo = await BorgSMSG.getV3ChunkInfo(base64Content); +// chunkInfo.chunkSize (default 1MB) +// chunkInfo.totalChunks +// chunkInfo.totalSize +// chunkInfo.index[i].offset, .size +``` + +#### unwrapV3CEK(base64, params) → Promise + +Unwrap CEK for manual chunk decryption. Returns base64 CEK. + +```javascript +const cekBase64 = await BorgSMSG.unwrapV3CEK(base64Content, { + license: "license", + fingerprint: "fp" +}); +``` + +#### decryptV3Chunk(base64, cekBase64, chunkIndex) → Promise + +Decrypt single chunk by index. + +```javascript +const chunk = await BorgSMSG.decryptV3Chunk(base64Content, cekBase64, 5); +``` + +#### parseV3Header(uint8Array) → Promise + +Parse header from partial data (for streaming). + +```javascript +const header = await BorgSMSG.parseV3Header(bytes); +// header.format, header.keyMethod, header.cadence +// header.payloadOffset (where chunks start) +// header.wrappedKeys, header.chunked, header.manifest +``` + +#### unwrapCEKFromHeader(wrappedKeys, params, cadence) → Promise + +Unwrap CEK from parsed header. + +```javascript +const cek = await BorgSMSG.unwrapCEKFromHeader( + header.wrappedKeys, + {license: "lic", fingerprint: "fp"}, + "daily" +); +``` + +#### decryptChunkDirect(chunkBytes, cek) → Promise + +Low-level chunk decryption with pre-unwrapped CEK. + +```javascript +const plaintext = await BorgSMSG.decryptChunkDirect(chunkBytes, cek); +``` + +### 3.5 Encryption Functions + +#### encrypt(message, password, hint?) → Promise + +Encrypt message (v1 format). Returns base64. + +```javascript +const encrypted = await BorgSMSG.encrypt({ + body: "Hello", + attachments: [{ + name: "file.txt", + content: btoa("data"), + mime: "text/plain" + }] +}, password, "optional hint"); +``` + +#### encryptWithManifest(message, password, manifest) → Promise + +Encrypt with manifest (v2 format). Returns base64. + +```javascript +const encrypted = await BorgSMSG.encryptWithManifest(message, password, { + title: "My Track", + artist: "Artist Name", + licenseType: "perpetual" +}); +``` + +### 3.6 ABR Functions + +#### parseABRManifest(jsonString) → Promise + +Parse HLS-style ABR manifest. + +```javascript +const manifest = await BorgSMSG.parseABRManifest(manifestJson); +// manifest.version, manifest.title, manifest.duration +// manifest.variants[i].name, .bandwidth, .url +// manifest.defaultIdx +``` + +#### selectVariant(manifest, bandwidthBps) → Promise + +Select best variant for bandwidth (returns index). + +```javascript +const idx = await BorgSMSG.selectVariant(manifest, measuredBandwidth); +// Uses 80% safety threshold +``` + +## 4. BorgSTMF Namespace + +### 4.1 Key Generation + +```javascript +const keypair = await BorgSTMF.generateKeyPair(); +// keypair.publicKey (base64 X25519) +// keypair.privateKey (base64 X25519) - KEEP SECRET +``` + +### 4.2 Encryption + +```javascript +// Encrypt JSON string +const encrypted = await BorgSTMF.encrypt( + JSON.stringify(formData), + serverPublicKeyBase64 +); + +// Encrypt with metadata +const encrypted = await BorgSTMF.encryptFields( + {email: "user@example.com", password: "secret"}, + serverPublicKeyBase64, + {timestamp: Date.now().toString()} // optional metadata +); +``` + +## 5. Type Definitions + +### 5.1 ManifestInfo + +```typescript +interface ManifestInfo { + version: string; + algorithm: string; + format?: string; + compression?: string; + hint?: string; + keyMethod?: string; // "LTHN" for v3 + cadence?: string; // "daily", "12h", "6h", "1h" + wrappedKeys?: WrappedKey[]; + isV3Streaming: boolean; + chunked?: ChunkInfo; + isChunked: boolean; + manifest?: Manifest; +} +``` + +### 5.2 Message / StreamMessage + +```typescript +interface Message { + from?: string; + to?: string; + subject?: string; + body: string; + timestamp?: number; + attachments: Attachment[]; + replyKey?: KeyInfo; + meta?: Record; +} + +interface Attachment { + name: string; + mime: string; + size: number; + content?: string; // base64 (v1) + data?: Uint8Array; // binary (v2/v3) +} +``` + +### 5.3 ChunkInfo + +```typescript +interface ChunkInfo { + chunkSize: number; // default 1048576 (1MB) + totalChunks: number; + totalSize: number; + index: ChunkEntry[]; +} + +interface ChunkEntry { + offset: number; + size: number; +} +``` + +### 5.4 Manifest + +```typescript +interface Manifest { + title: string; + artist?: string; + album?: string; + genre?: string; + year?: number; + releaseType?: string; // "single", "album", "ep", "mix" + duration?: number; // seconds + format?: string; + expiresAt?: number; // Unix timestamp + issuedAt?: number; // Unix timestamp + licenseType?: string; // "perpetual", "rental", "stream", "preview" + tracks?: Track[]; + tags?: string[]; + links?: Record; + extra?: Record; +} +``` + +## 6. Error Handling + +### 6.1 Pattern + +All functions throw on error: + +```javascript +try { + const msg = await BorgSMSG.decrypt(content, password); +} catch (e) { + console.error(e.message); +} +``` + +### 6.2 Common Errors + +| Error | Cause | +|-------|-------| +| `decrypt requires 2 arguments` | Wrong argument count | +| `decryption failed: {reason}` | Wrong password or corrupted | +| `invalid format` | Not a valid SMSG file | +| `unsupported version` | Unknown format version | +| `key expired` | v3 rolling key outside window | +| `invalid base64: {reason}` | Base64 decode failed | +| `chunk out of range` | Invalid chunk index | + +## 7. Performance + +### 7.1 Binary vs Base64 + +- Binary functions (`*Binary`, `decryptStream`) are ~30% faster +- Avoid double base64 encoding + +### 7.2 Large Files (>50MB) + +Use chunked streaming: + +```javascript +// Efficient: Cache CEK, stream chunks +const header = await BorgSMSG.parseV3Header(bytes); +const cek = await BorgSMSG.unwrapCEKFromHeader(header.wrappedKeys, params); + +for (let i = 0; i < header.chunked.totalChunks; i++) { + const chunk = await BorgSMSG.decryptChunkDirect(payload, cek); + player.write(chunk); + // chunk is GC'd after each iteration +} +``` + +### 7.3 Typical Execution Times + +| Operation | Size | Time | +|-----------|------|------| +| getInfo | any | ~50-100ms | +| decrypt (small) | <1MB | ~200-500ms | +| decrypt (large) | 100MB | 2-5s | +| decryptV3Chunk | 1MB | ~200-400ms | +| generateKeyPair | - | ~50-200ms | + +## 8. Browser Compatibility + +| Browser | Support | +|---------|---------| +| Chrome 57+ | Full | +| Firefox 52+ | Full | +| Safari 11+ | Full | +| Edge 16+ | Full | +| IE | Not supported | + +Requirements: +- WebAssembly support +- Async/await (ES2017) +- Uint8Array + +## 9. Memory Management + +- WASM module: ~5.9MB static +- Per-operation: Peak ~2-3x file size during decryption +- Go GC reclaims after Promise resolution +- Keys never leave WASM memory + +## 10. Implementation Reference + +- Source: `pkg/wasm/stmf/main.go` (1758 lines) +- Build: `GOOS=js GOARCH=wasm go build -o stmf.wasm ./pkg/wasm/stmf/` + +## 11. Security Considerations + +1. **Password handling**: Clear from memory after use +2. **Memory isolation**: WASM sandbox prevents JS access +3. **Constant-time crypto**: Go crypto uses safe operations +4. **Key protection**: Keys never exposed to JavaScript + +## 12. Future Work + +- [ ] WebWorker support for background decryption +- [ ] Streaming API with ReadableStream +- [ ] Smaller WASM size via TinyGo +- [ ] Native Web Crypto fallback for simple operations