From 37b50abf5738bedab342912bdd57b5ecab34b316 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Feb 2026 18:39:14 +0000 Subject: [PATCH] =?UTF-8?q?docs:=20complete=20Phase=204=20design=20?= =?UTF-8?q?=E2=80=94=20fill=20open=20sections=20from=20dAppServer=20archae?= =?UTF-8?q?ology?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Polyfills: none needed (WebView2 is Chromium). Object store: Go-managed SQLite via gRPC, 6 operations extracted. Templated config generators: Go text/template in CoreGO. Git-based plugin marketplace: Git repo as registry, clone-based distribution, ed25519 verification, 7 marketplace RPC operations. Complete RPC surface: 40+ procedures across auth, crypto, fs, process, store, IPC, and marketplace namespaces. Heritage table expanded to 4 tiers from 20-repo archaeological survey. Co-Authored-By: Claude Opus 4.6 --- ...17-phase4-coredeno-webcomponents-design.md | 253 ++++++++++++++++-- 1 file changed, 224 insertions(+), 29 deletions(-) diff --git a/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md b/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md index a701cde..901dfa3 100644 --- a/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md +++ b/docs/plans/2026-02-17-phase4-coredeno-webcomponents-design.md @@ -1,7 +1,7 @@ # Phase 4: CoreDeno + Web Components **Date:** 2026-02-17 -**Status:** Approved (open sections pending dAppServer code review) +**Status:** Approved **Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code ## Vision @@ -189,40 +189,233 @@ Angular router replaced by lightweight hash-based router mapping URLs to 20 repos at `github.com/dAppServer/` — the original client-side server concept and browser↔Go communications bridge. Extract and port, not copy. -| dAppServer repo | CoreDeno equivalent | Action | +### Tier 1: Extract (Core Architecture) + +| dAppServer repo | What to extract | Phase 4 target | |---|---|---| -| `server` | CoreDeno sidecar architecture | Extract | -| `dappui` | Web Component framework | Extract | -| `auth-server` + `mod-auth` | Signed manifest verification | Extract | -| `mod-docker` | Process management (core/pkg/process) | Port | -| `mod-io-process` | I/O fortress gRPC bridge | Extract | -| `server-sdk-typescript-angular` | Deno TypeScript bindings | Port | -| `app-marketplace` | Module registry/discovery | Extract | -| `app-directory-browser` | File browser Web Component | Port | -| `devops` | CI/deployment patterns | Reference | -| `app-utils-cyberchef` | Utility module example | Reference | -| `app-mining` | Mining module (already in ITW3) | Reference | -| `server-sdk-python` | Not needed (Go replaces) | Skip | +| `server` | Port 36911 bridge, ZeroMQ IPC (pub/sub + req/rep + push/pull), air-gapped PGP auth, object store, 13 test files with RPC procedures | CoreDeno sidecar, I/O fortress, auth | +| `dappui` | Angular→WC migration, REST+WS+Wails triple, terminal (xterm.js) | Web Component framework, MCPBridge | +| `mod-auth` | PGP zero-knowledge auth (sign→encrypt→verify→JWT), QuasiSalt, roles | Signed manifest verification, identity | +| `mod-io-process` | Process registry, 3-layer I/O streaming (process→ZeroMQ→WS→browser) | `core/pkg/process`, event bus | +| `app-marketplace` | Git-as-database registry, category-as-directory, install pipeline | Module registry, `.core/view.yml` loader | -## Open Sections (Pending dAppServer Code Review) +### Tier 2: Port (Useful Patterns) -These sections will be refined after studying the dAppServer codebase: +| dAppServer repo | What to port | Phase 4 target | +|---|---|---| +| `auth-server` | Keycloak + native PGP fallback | External auth option | +| `mod-docker` | Docker socket client, container CRUD (8 ops) | `core/pkg/process` | +| `app-mining` | CLI Bridge (camelCase→kebab-case), Process-as-Service, API proxy | Generic CLI wrapper | +| `app-directory-browser` | Split-pane layout, lazy tree, filesystem CRUD RPCs | `` WC | +| `wails-build-action` | Auto-stack detection, cross-platform signing, Deno CI | Build tooling | -### Polyfills -What browser APIs did dAppServer polyfill? Which are still needed in WebView2? -WebView2 is Chromium-based so most modern APIs are available natively. +### Tier 3: Reference + +| dAppServer repo | Value | +|---|---| +| `depends` | Bitcoin Core hermetic build; libmultiprocess + Cap'n Proto validates process-separation | +| `app-utils-cyberchef` | Purest manifest-only pattern ("manifest IS the app") | +| `devops` | Cross-compilation matrix (9 triples), ancestor of ADR-001 | +| `pwa-native-action` | PWA→Wails native shell proof, ancestor of `core-gui` | +| `docker-images` | C++ cross-compile layers | + +### Tier 4: Skip + +| dAppServer repo | Reason | +|---|---| +| `server-sdk-python` | Auto-generated, Go replaces | +| `server-sdk-typescript-angular` | Auto-generated, superseded | +| `openvpn` | Unmodified upstream fork | +| `ansible-server-base` | Standard Ansible hardening | +| `.github` | Org profile only | + +## Polyfills + +dAppServer polyfilled nothing at the browser level. The prototype ran inside +Electron/WebView2 (Chromium), which already supports all required APIs natively: +Custom Elements v1, Shadow DOM v1, ES Modules, `fetch`, `WebSocket`, +`customElements.define()`, `structuredClone()`. + +**Decision**: No polyfills needed. WebView2 is Chromium-based. The minimum +Chromium version Wails v3 targets already supports all Web Component APIs. + +## Object Store + +dAppServer used a file-based JSON key-value store at `data/objects/{group}/{object}.json`. +Six operations discovered from test files: + +| Operation | dAppServer endpoint | Phase 4 equivalent | +|-----------|--------------------|--------------------| +| Get | `GET /config/object/{group}/{object}` | `store.get(group, key)` | +| Set | `POST /config/object/{group}/{object}` | `store.set(group, key, value)` | +| Clear | `DELETE /config/object/{group}/{object}` | `store.delete(group, key)` | +| Count | `GET /config/object/{group}/count` | `store.count(group)` | +| Remove group | `DELETE /config/object/{group}` | `store.deleteGroup(group)` | +| Render template | `POST /config/render` | `store.render(template, vars)` | + +Used for: installed app registry (`conf/installed-apps.json`), menu state +(`conf/menu.json`), per-module config, user preferences. + +**Decision**: Go-managed storage via gRPC. CoreGO owns persistence through +`core/pkg/io`. Modules request storage through the I/O fortress — never +touching the filesystem directly. SQLite backend (already a dependency in +the blockchain layer). IndexedDB reserved for client-side cache only. + +## Templated Config Generators + +dAppServer's `config.render` endpoint accepted a template string + variable map +and returned the rendered output. Used to generate configs for managed processes +(e.g., xmrig config.json from user-selected pool/wallet parameters). + +The pattern in Phase 4: +1. Module declares config templates in `.core/view.yml` under a `config:` key +2. User preferences stored in the object store +3. CoreGO renders templates at process-start time via Go `text/template` +4. Rendered configs written to sandboxed paths the module has `write` permission for + +Example from mining module (camelCase→kebab-case CLI arg transformation): +```yaml +config: + xmrig: + template: conf/xmrig/config.json.tmpl + vars: + pool: "{{ .user.pool }}" + wallet: "{{ .user.wallet }}" +``` + +**Decision**: Go `text/template` in CoreGO. Templates live in the module's +`.core/` directory. Variables come from the object store. No Deno involvement — +config rendering is a Go-side I/O fortress operation. + +## Git-Based Plugin Marketplace + +### dAppServer Pattern (Extracted from `app-marketplace`) + +A Git repository serves as the package registry. No server infrastructure needed. + +``` +marketplace/ # Git repo +├── index.json # Root: {version, apps[], dirs[]} +├── miner/ +│ └── index.json # Category: {version, apps[], dirs[]} +├── utils/ +│ └── index.json # Category: {version, apps[], dirs[]} +└── ... +``` + +Each `index.json` entry points to a raw `.itw3.json` URL in the plugin's own repo: +```json +{"code": "utils-cyberchef", "name": "CyberChef", "type": "bin", + "pkg": "https://raw.githubusercontent.com/dAppServer/app-utils-cyberchef/main/.itw3.json"} +``` + +Install pipeline: browse index → fetch manifest → download zip from `app.url` → +extract → run hooks (rename, etc.) → register in object store → add menu entry. + +### Phase 4 Evolution + +Replace GitHub-specific URLs with Forgejo-compatible Git operations: + +1. **Registry**: A Git repo (`host-uk/marketplace`) with category directories + and `index.json` files. Cloned/pulled by CoreGO at startup and periodically. +2. **Manifests**: Each module's `.core/view.yml` is the manifest (replaces `.itw3.json`). + The marketplace index points to the module's Git repo, not a raw file URL. +3. **Distribution**: Git clone of the module repo (not zip downloads). CoreGO + clones into a managed modules directory with depth=1. +4. **Verification**: ed25519 signature in `view.yml` verified before loading. + The marketplace index includes the expected signing public key. +5. **Install hooks**: Declared in `view.yml` under `hooks:`. Executed by CoreGO + in the I/O fortress (rename, template render, permission grant). +6. **Updates**: `git pull` on the module repo. Signature re-verified after pull. + If signature fails, rollback to previous commit. +7. **Discovery**: `core marketplace list`, `core marketplace search `, + `core marketplace install `. + +```yaml +# marketplace/index.json +version: 1 +modules: + - code: utils-cyberchef + name: CyberChef Data Toolkit + repo: https://forge.lthn.io/host-uk/mod-cyberchef.git + sign_key: + category: utils +categories: + - miner + - utils + - network +``` + +### RPC Surface (from dAppServer test extraction) + +| Operation | CLI | RPC | +|-----------|-----|-----| +| Browse | `core marketplace list` | `marketplace.list(category?)` | +| Search | `core marketplace search ` | `marketplace.search(query)` | +| Install | `core marketplace install ` | `marketplace.install(code)` | +| Remove | `core marketplace remove ` | `marketplace.remove(code)` | +| Installed | `core marketplace installed` | `marketplace.installed()` | +| Update | `core marketplace update ` | `marketplace.update(code)` | +| Update all | `core marketplace update` | `marketplace.updateAll()` | + +## Complete RPC Surface (Archaeological Extraction) + +All procedures discovered from dAppServer test files and controllers: + +### Auth +- `auth.create(username, password)` — PGP key generation + QuasiSalt hash +- `auth.login(username, encryptedPayload)` — Zero-knowledge PGP verify → JWT +- `auth.delete(username)` — Remove account + +### Crypto +- `crypto.pgp.generateKeyPair(name, email, passphrase)` → {publicKey, privateKey} +- `crypto.pgp.encrypt(data, publicKey)` → encryptedData +- `crypto.pgp.decrypt(data, privateKey, passphrase)` → plaintext +- `crypto.pgp.sign(data, privateKey, passphrase)` → signature +- `crypto.pgp.verify(data, signature, publicKey)` → boolean + +### Filesystem +- `fs.list(path, detailed?)` → FileEntry[] +- `fs.read(path)` → content +- `fs.write(path, content)` → boolean +- `fs.delete(path)` → boolean +- `fs.rename(from, to)` → boolean +- `fs.mkdir(path)` → boolean +- `fs.isDir(path)` → boolean +- `fs.ensureDir(path)` → boolean + +### Process +- `process.run(command, args, options)` → ProcessHandle +- `process.add(request)` → key +- `process.start(key)` → boolean +- `process.stop(key)` → boolean +- `process.kill(key)` → boolean +- `process.list()` → string[] +- `process.get(key)` → ProcessInfo +- `process.stdout.subscribe(key)` → stream +- `process.stdin.write(key, data)` → void ### Object Store -dAppServer's data persistence model. How does it map to CoreDeno? Options include -IndexedDB in WebView2, SQLite via Deno, or Go-managed storage via gRPC. +- `store.get(group, key)` → value +- `store.set(group, key, value)` → void +- `store.delete(group, key)` → void +- `store.count(group)` → number +- `store.deleteGroup(group)` → void +- `store.render(template, vars)` → string -### Templated Config Generators -The pattern for generating configs from templates. Informs `.core/view.yml` -codegen pipeline and how complex configurations are composed from simpler parts. +### IPC / Event Bus +- `ipc.pub.subscribe(channel)` → stream +- `ipc.pub.publish(channel, message)` → void +- `ipc.req.send(channel, message)` → response +- `ipc.push.send(message)` → void -### Git-Based Plugin Marketplace -Module discovery, versioning, and distribution via Git repositories. To be -designed after dAppServer `app-marketplace` code review. +### Marketplace +- `marketplace.list(category?)` → ModuleEntry[] +- `marketplace.search(query)` → ModuleEntry[] +- `marketplace.install(code)` → boolean +- `marketplace.remove(code)` → boolean +- `marketplace.installed()` → InstalledModule[] +- `marketplace.update(code)` → boolean ## Deliverables @@ -240,6 +433,8 @@ designed after dAppServer `app-marketplace` code review. ## Not In Scope (Future Phases) - LEM auto-loading from signed manifests -- Module marketplace server infrastructure -- Offline-first sync / object store +- Marketplace server infrastructure (Git-based registry is sufficient) +- Offline-first sync (IndexedDB client cache) - Full Angular removal (Phase 4c) +- GlusterFS distributed storage (dAppServer aspiration, not needed yet) +- Multi-chain support (Phase 4 is Lethean-only)