docs: complete Phase 4 design — fill open sections from dAppServer archaeology

Polyfills: none needed (WebView2 is Chromium). Object store: Go-managed SQLite via gRPC, 6 operations extracted. Templated config generators: Go text/template in CoreGO. Git-based plugin marketplace: Git repo as registry, clone-based distribution, ed25519 verification, 7 marketplace RPC operations. Complete RPC surface: 40+ procedures across auth, crypto, fs, process, store, IPC, and marketplace namespaces. Heritage table expanded to 4 tiers from 20-repo archaeological survey. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 18:39:14 +00:00 · 2026-02-17 18:39:14 +00:00 · 37b50abf57
commit 37b50abf57
parent f5bf7cba81
1 changed files with 224 additions and 29 deletions
--- a/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md
+++ b/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md
@ -1,7 +1,7 @@
 # Phase 4: CoreDeno + Web Components

 **Date:** 2026-02-17
-**Status:** Approved (open sections pending dAppServer code review)
+**Status:** Approved
 **Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code

 ## Vision
@ -189,40 +189,233 @@ Angular router replaced by lightweight hash-based router mapping URLs to
 20 repos at `github.com/dAppServer/` — the original client-side server concept
 and browser↔Go communications bridge. Extract and port, not copy.

-| dAppServer repo | CoreDeno equivalent | Action |
+### Tier 1: Extract (Core Architecture)
+
+| dAppServer repo | What to extract | Phase 4 target |
 |---|---|---|
-| `server` | CoreDeno sidecar architecture | Extract |
-| `dappui` | Web Component framework | Extract |
-| `auth-server` + `mod-auth` | Signed manifest verification | Extract |
-| `mod-docker` | Process management (core/pkg/process) | Port |
-| `mod-io-process` | I/O fortress gRPC bridge | Extract |
-| `server-sdk-typescript-angular` | Deno TypeScript bindings | Port |
-| `app-marketplace` | Module registry/discovery | Extract |
-| `app-directory-browser` | File browser Web Component | Port |
-| `devops` | CI/deployment patterns | Reference |
-| `app-utils-cyberchef` | Utility module example | Reference |
-| `app-mining` | Mining module (already in ITW3) | Reference |
-| `server-sdk-python` | Not needed (Go replaces) | Skip |
+| `server` | Port 36911 bridge, ZeroMQ IPC (pub/sub + req/rep + push/pull), air-gapped PGP auth, object store, 13 test files with RPC procedures | CoreDeno sidecar, I/O fortress, auth |
+| `dappui` | Angular→WC migration, REST+WS+Wails triple, terminal (xterm.js) | Web Component framework, MCPBridge |
+| `mod-auth` | PGP zero-knowledge auth (sign→encrypt→verify→JWT), QuasiSalt, roles | Signed manifest verification, identity |
+| `mod-io-process` | Process registry, 3-layer I/O streaming (process→ZeroMQ→WS→browser) | `core/pkg/process`, event bus |
+| `app-marketplace` | Git-as-database registry, category-as-directory, install pipeline | Module registry, `.core/view.yml` loader |

-## Open Sections (Pending dAppServer Code Review)
+### Tier 2: Port (Useful Patterns)

-These sections will be refined after studying the dAppServer codebase:
+| dAppServer repo | What to port | Phase 4 target |
+|---|---|---|
+| `auth-server` | Keycloak + native PGP fallback | External auth option |
+| `mod-docker` | Docker socket client, container CRUD (8 ops) | `core/pkg/process` |
+| `app-mining` | CLI Bridge (camelCase→kebab-case), Process-as-Service, API proxy | Generic CLI wrapper |
+| `app-directory-browser` | Split-pane layout, lazy tree, filesystem CRUD RPCs | `<core-file-tree>` WC |
+| `wails-build-action` | Auto-stack detection, cross-platform signing, Deno CI | Build tooling |

-### Polyfills
-What browser APIs did dAppServer polyfill? Which are still needed in WebView2?
-WebView2 is Chromium-based so most modern APIs are available natively.
+### Tier 3: Reference
+
+| dAppServer repo | Value |
+|---|---|
+| `depends` | Bitcoin Core hermetic build; libmultiprocess + Cap'n Proto validates process-separation |
+| `app-utils-cyberchef` | Purest manifest-only pattern ("manifest IS the app") |
+| `devops` | Cross-compilation matrix (9 triples), ancestor of ADR-001 |
+| `pwa-native-action` | PWA→Wails native shell proof, ancestor of `core-gui` |
+| `docker-images` | C++ cross-compile layers |
+
+### Tier 4: Skip
+
+| dAppServer repo | Reason |
+|---|---|
+| `server-sdk-python` | Auto-generated, Go replaces |
+| `server-sdk-typescript-angular` | Auto-generated, superseded |
+| `openvpn` | Unmodified upstream fork |
+| `ansible-server-base` | Standard Ansible hardening |
+| `.github` | Org profile only |
+
+## Polyfills
+
+dAppServer polyfilled nothing at the browser level. The prototype ran inside
+Electron/WebView2 (Chromium), which already supports all required APIs natively:
+Custom Elements v1, Shadow DOM v1, ES Modules, `fetch`, `WebSocket`,
+`customElements.define()`, `structuredClone()`.
+
+**Decision**: No polyfills needed. WebView2 is Chromium-based. The minimum
+Chromium version Wails v3 targets already supports all Web Component APIs.
+
+## Object Store
+
+dAppServer used a file-based JSON key-value store at `data/objects/{group}/{object}.json`.
+Six operations discovered from test files:
+
+| Operation | dAppServer endpoint | Phase 4 equivalent |
+|-----------|--------------------|--------------------|
+| Get | `GET /config/object/{group}/{object}` | `store.get(group, key)` |
+| Set | `POST /config/object/{group}/{object}` | `store.set(group, key, value)` |
+| Clear | `DELETE /config/object/{group}/{object}` | `store.delete(group, key)` |
+| Count | `GET /config/object/{group}/count` | `store.count(group)` |
+| Remove group | `DELETE /config/object/{group}` | `store.deleteGroup(group)` |
+| Render template | `POST /config/render` | `store.render(template, vars)` |
+
+Used for: installed app registry (`conf/installed-apps.json`), menu state
+(`conf/menu.json`), per-module config, user preferences.
+
+**Decision**: Go-managed storage via gRPC. CoreGO owns persistence through
+`core/pkg/io`. Modules request storage through the I/O fortress — never
+touching the filesystem directly. SQLite backend (already a dependency in
+the blockchain layer). IndexedDB reserved for client-side cache only.
+
+## Templated Config Generators
+
+dAppServer's `config.render` endpoint accepted a template string + variable map
+and returned the rendered output. Used to generate configs for managed processes
+(e.g., xmrig config.json from user-selected pool/wallet parameters).
+
+The pattern in Phase 4:
+1. Module declares config templates in `.core/view.yml` under a `config:` key
+2. User preferences stored in the object store
+3. CoreGO renders templates at process-start time via Go `text/template`
+4. Rendered configs written to sandboxed paths the module has `write` permission for
+
+Example from mining module (camelCase→kebab-case CLI arg transformation):
+```yaml
+config:
+  xmrig:
+    template: conf/xmrig/config.json.tmpl
+    vars:
+      pool: "{{ .user.pool }}"
+      wallet: "{{ .user.wallet }}"
+```
+
+**Decision**: Go `text/template` in CoreGO. Templates live in the module's
+`.core/` directory. Variables come from the object store. No Deno involvement —
+config rendering is a Go-side I/O fortress operation.
+
+## Git-Based Plugin Marketplace
+
+### dAppServer Pattern (Extracted from `app-marketplace`)
+
+A Git repository serves as the package registry. No server infrastructure needed.
+
+```
+marketplace/                    # Git repo
+├── index.json                  # Root: {version, apps[], dirs[]}
+├── miner/
+│   └── index.json              # Category: {version, apps[], dirs[]}
+├── utils/
+│   └── index.json              # Category: {version, apps[], dirs[]}
+└── ...
+```
+
+Each `index.json` entry points to a raw `.itw3.json` URL in the plugin's own repo:
+```json
+{"code": "utils-cyberchef", "name": "CyberChef", "type": "bin",
+ "pkg": "https://raw.githubusercontent.com/dAppServer/app-utils-cyberchef/main/.itw3.json"}
+```
+
+Install pipeline: browse index → fetch manifest → download zip from `app.url` →
+extract → run hooks (rename, etc.) → register in object store → add menu entry.
+
+### Phase 4 Evolution
+
+Replace GitHub-specific URLs with Forgejo-compatible Git operations:
+
+1. **Registry**: A Git repo (`host-uk/marketplace`) with category directories
+   and `index.json` files. Cloned/pulled by CoreGO at startup and periodically.
+2. **Manifests**: Each module's `.core/view.yml` is the manifest (replaces `.itw3.json`).
+   The marketplace index points to the module's Git repo, not a raw file URL.
+3. **Distribution**: Git clone of the module repo (not zip downloads). CoreGO
+   clones into a managed modules directory with depth=1.
+4. **Verification**: ed25519 signature in `view.yml` verified before loading.
+   The marketplace index includes the expected signing public key.
+5. **Install hooks**: Declared in `view.yml` under `hooks:`. Executed by CoreGO
+   in the I/O fortress (rename, template render, permission grant).
+6. **Updates**: `git pull` on the module repo. Signature re-verified after pull.
+   If signature fails, rollback to previous commit.
+7. **Discovery**: `core marketplace list`, `core marketplace search <query>`,
+   `core marketplace install <code>`.
+
+```yaml
+# marketplace/index.json
+version: 1
+modules:
+  - code: utils-cyberchef
+    name: CyberChef Data Toolkit
+    repo: https://forge.lthn.io/host-uk/mod-cyberchef.git
+    sign_key: <ed25519 public key>
+    category: utils
+categories:
+  - miner
+  - utils
+  - network
+```
+
+### RPC Surface (from dAppServer test extraction)
+
+| Operation | CLI | RPC |
+|-----------|-----|-----|
+| Browse | `core marketplace list` | `marketplace.list(category?)` |
+| Search | `core marketplace search <q>` | `marketplace.search(query)` |
+| Install | `core marketplace install <code>` | `marketplace.install(code)` |
+| Remove | `core marketplace remove <code>` | `marketplace.remove(code)` |
+| Installed | `core marketplace installed` | `marketplace.installed()` |
+| Update | `core marketplace update <code>` | `marketplace.update(code)` |
+| Update all | `core marketplace update` | `marketplace.updateAll()` |
+
+## Complete RPC Surface (Archaeological Extraction)
+
+All procedures discovered from dAppServer test files and controllers:
+
+### Auth
+- `auth.create(username, password)` — PGP key generation + QuasiSalt hash
+- `auth.login(username, encryptedPayload)` — Zero-knowledge PGP verify → JWT
+- `auth.delete(username)` — Remove account
+
+### Crypto
+- `crypto.pgp.generateKeyPair(name, email, passphrase)` → {publicKey, privateKey}
+- `crypto.pgp.encrypt(data, publicKey)` → encryptedData
+- `crypto.pgp.decrypt(data, privateKey, passphrase)` → plaintext
+- `crypto.pgp.sign(data, privateKey, passphrase)` → signature
+- `crypto.pgp.verify(data, signature, publicKey)` → boolean
+
+### Filesystem
+- `fs.list(path, detailed?)` → FileEntry[]
+- `fs.read(path)` → content
+- `fs.write(path, content)` → boolean
+- `fs.delete(path)` → boolean
+- `fs.rename(from, to)` → boolean
+- `fs.mkdir(path)` → boolean
+- `fs.isDir(path)` → boolean
+- `fs.ensureDir(path)` → boolean
+
+### Process
+- `process.run(command, args, options)` → ProcessHandle
+- `process.add(request)` → key
+- `process.start(key)` → boolean
+- `process.stop(key)` → boolean
+- `process.kill(key)` → boolean
+- `process.list()` → string[]
+- `process.get(key)` → ProcessInfo
+- `process.stdout.subscribe(key)` → stream
+- `process.stdin.write(key, data)` → void

 ### Object Store
-dAppServer's data persistence model. How does it map to CoreDeno? Options include
-IndexedDB in WebView2, SQLite via Deno, or Go-managed storage via gRPC.
+- `store.get(group, key)` → value
+- `store.set(group, key, value)` → void
+- `store.delete(group, key)` → void
+- `store.count(group)` → number
+- `store.deleteGroup(group)` → void
+- `store.render(template, vars)` → string

-### Templated Config Generators
-The pattern for generating configs from templates. Informs `.core/view.yml`
-codegen pipeline and how complex configurations are composed from simpler parts.
+### IPC / Event Bus
+- `ipc.pub.subscribe(channel)` → stream
+- `ipc.pub.publish(channel, message)` → void
+- `ipc.req.send(channel, message)` → response
+- `ipc.push.send(message)` → void

-### Git-Based Plugin Marketplace
-Module discovery, versioning, and distribution via Git repositories. To be
-designed after dAppServer `app-marketplace` code review.
+### Marketplace
+- `marketplace.list(category?)` → ModuleEntry[]
+- `marketplace.search(query)` → ModuleEntry[]
+- `marketplace.install(code)` → boolean
+- `marketplace.remove(code)` → boolean
+- `marketplace.installed()` → InstalledModule[]
+- `marketplace.update(code)` → boolean

 ## Deliverables

@ -240,6 +433,8 @@ designed after dAppServer `app-marketplace` code review.
 ## Not In Scope (Future Phases)

 - LEM auto-loading from signed manifests
- Module marketplace server infrastructure
- Offline-first sync / object store
+- Marketplace server infrastructure (Git-based registry is sufficient)
+- Offline-first sync (IndexedDB client cache)
 - Full Angular removal (Phase 4c)
+- GlusterFS distributed storage (dAppServer aspiration, not needed yet)
+- Multi-chain support (Phase 4 is Lethean-only)