docs: complete Phase 4 design — fill open sections from dAppServer archaeology

Polyfills: none needed (WebView2 is Chromium).
Object store: Go-managed SQLite via gRPC, 6 operations extracted.
Templated config generators: Go text/template in CoreGO.
Git-based plugin marketplace: Git repo as registry, clone-based distribution,
ed25519 verification, 7 marketplace RPC operations.
Complete RPC surface: 40+ procedures across auth, crypto, fs, process,
store, IPC, and marketplace namespaces.
Heritage table expanded to 4 tiers from 20-repo archaeological survey.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Claude 2026-02-17 18:39:14 +00:00
parent f5bf7cba81
commit 37b50abf57
No known key found for this signature in database
GPG key ID: AF404715446AEB41

View file

@ -1,7 +1,7 @@
# Phase 4: CoreDeno + Web Components # Phase 4: CoreDeno + Web Components
**Date:** 2026-02-17 **Date:** 2026-02-17
**Status:** Approved (open sections pending dAppServer code review) **Status:** Approved
**Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code **Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code
## Vision ## Vision
@ -189,40 +189,233 @@ Angular router replaced by lightweight hash-based router mapping URLs to
20 repos at `github.com/dAppServer/` — the original client-side server concept 20 repos at `github.com/dAppServer/` — the original client-side server concept
and browser↔Go communications bridge. Extract and port, not copy. and browser↔Go communications bridge. Extract and port, not copy.
| dAppServer repo | CoreDeno equivalent | Action | ### Tier 1: Extract (Core Architecture)
| dAppServer repo | What to extract | Phase 4 target |
|---|---|---| |---|---|---|
| `server` | CoreDeno sidecar architecture | Extract | | `server` | Port 36911 bridge, ZeroMQ IPC (pub/sub + req/rep + push/pull), air-gapped PGP auth, object store, 13 test files with RPC procedures | CoreDeno sidecar, I/O fortress, auth |
| `dappui` | Web Component framework | Extract | | `dappui` | Angular→WC migration, REST+WS+Wails triple, terminal (xterm.js) | Web Component framework, MCPBridge |
| `auth-server` + `mod-auth` | Signed manifest verification | Extract | | `mod-auth` | PGP zero-knowledge auth (sign→encrypt→verify→JWT), QuasiSalt, roles | Signed manifest verification, identity |
| `mod-docker` | Process management (core/pkg/process) | Port | | `mod-io-process` | Process registry, 3-layer I/O streaming (process→ZeroMQ→WS→browser) | `core/pkg/process`, event bus |
| `mod-io-process` | I/O fortress gRPC bridge | Extract | | `app-marketplace` | Git-as-database registry, category-as-directory, install pipeline | Module registry, `.core/view.yml` loader |
| `server-sdk-typescript-angular` | Deno TypeScript bindings | Port |
| `app-marketplace` | Module registry/discovery | Extract |
| `app-directory-browser` | File browser Web Component | Port |
| `devops` | CI/deployment patterns | Reference |
| `app-utils-cyberchef` | Utility module example | Reference |
| `app-mining` | Mining module (already in ITW3) | Reference |
| `server-sdk-python` | Not needed (Go replaces) | Skip |
## Open Sections (Pending dAppServer Code Review) ### Tier 2: Port (Useful Patterns)
These sections will be refined after studying the dAppServer codebase: | dAppServer repo | What to port | Phase 4 target |
|---|---|---|
| `auth-server` | Keycloak + native PGP fallback | External auth option |
| `mod-docker` | Docker socket client, container CRUD (8 ops) | `core/pkg/process` |
| `app-mining` | CLI Bridge (camelCase→kebab-case), Process-as-Service, API proxy | Generic CLI wrapper |
| `app-directory-browser` | Split-pane layout, lazy tree, filesystem CRUD RPCs | `<core-file-tree>` WC |
| `wails-build-action` | Auto-stack detection, cross-platform signing, Deno CI | Build tooling |
### Polyfills ### Tier 3: Reference
What browser APIs did dAppServer polyfill? Which are still needed in WebView2?
WebView2 is Chromium-based so most modern APIs are available natively. | dAppServer repo | Value |
|---|---|
| `depends` | Bitcoin Core hermetic build; libmultiprocess + Cap'n Proto validates process-separation |
| `app-utils-cyberchef` | Purest manifest-only pattern ("manifest IS the app") |
| `devops` | Cross-compilation matrix (9 triples), ancestor of ADR-001 |
| `pwa-native-action` | PWA→Wails native shell proof, ancestor of `core-gui` |
| `docker-images` | C++ cross-compile layers |
### Tier 4: Skip
| dAppServer repo | Reason |
|---|---|
| `server-sdk-python` | Auto-generated, Go replaces |
| `server-sdk-typescript-angular` | Auto-generated, superseded |
| `openvpn` | Unmodified upstream fork |
| `ansible-server-base` | Standard Ansible hardening |
| `.github` | Org profile only |
## Polyfills
dAppServer polyfilled nothing at the browser level. The prototype ran inside
Electron/WebView2 (Chromium), which already supports all required APIs natively:
Custom Elements v1, Shadow DOM v1, ES Modules, `fetch`, `WebSocket`,
`customElements.define()`, `structuredClone()`.
**Decision**: No polyfills needed. WebView2 is Chromium-based. The minimum
Chromium version Wails v3 targets already supports all Web Component APIs.
## Object Store
dAppServer used a file-based JSON key-value store at `data/objects/{group}/{object}.json`.
Six operations discovered from test files:
| Operation | dAppServer endpoint | Phase 4 equivalent |
|-----------|--------------------|--------------------|
| Get | `GET /config/object/{group}/{object}` | `store.get(group, key)` |
| Set | `POST /config/object/{group}/{object}` | `store.set(group, key, value)` |
| Clear | `DELETE /config/object/{group}/{object}` | `store.delete(group, key)` |
| Count | `GET /config/object/{group}/count` | `store.count(group)` |
| Remove group | `DELETE /config/object/{group}` | `store.deleteGroup(group)` |
| Render template | `POST /config/render` | `store.render(template, vars)` |
Used for: installed app registry (`conf/installed-apps.json`), menu state
(`conf/menu.json`), per-module config, user preferences.
**Decision**: Go-managed storage via gRPC. CoreGO owns persistence through
`core/pkg/io`. Modules request storage through the I/O fortress — never
touching the filesystem directly. SQLite backend (already a dependency in
the blockchain layer). IndexedDB reserved for client-side cache only.
## Templated Config Generators
dAppServer's `config.render` endpoint accepted a template string + variable map
and returned the rendered output. Used to generate configs for managed processes
(e.g., xmrig config.json from user-selected pool/wallet parameters).
The pattern in Phase 4:
1. Module declares config templates in `.core/view.yml` under a `config:` key
2. User preferences stored in the object store
3. CoreGO renders templates at process-start time via Go `text/template`
4. Rendered configs written to sandboxed paths the module has `write` permission for
Example from mining module (camelCase→kebab-case CLI arg transformation):
```yaml
config:
xmrig:
template: conf/xmrig/config.json.tmpl
vars:
pool: "{{ .user.pool }}"
wallet: "{{ .user.wallet }}"
```
**Decision**: Go `text/template` in CoreGO. Templates live in the module's
`.core/` directory. Variables come from the object store. No Deno involvement —
config rendering is a Go-side I/O fortress operation.
## Git-Based Plugin Marketplace
### dAppServer Pattern (Extracted from `app-marketplace`)
A Git repository serves as the package registry. No server infrastructure needed.
```
marketplace/ # Git repo
├── index.json # Root: {version, apps[], dirs[]}
├── miner/
│ └── index.json # Category: {version, apps[], dirs[]}
├── utils/
│ └── index.json # Category: {version, apps[], dirs[]}
└── ...
```
Each `index.json` entry points to a raw `.itw3.json` URL in the plugin's own repo:
```json
{"code": "utils-cyberchef", "name": "CyberChef", "type": "bin",
"pkg": "https://raw.githubusercontent.com/dAppServer/app-utils-cyberchef/main/.itw3.json"}
```
Install pipeline: browse index → fetch manifest → download zip from `app.url`
extract → run hooks (rename, etc.) → register in object store → add menu entry.
### Phase 4 Evolution
Replace GitHub-specific URLs with Forgejo-compatible Git operations:
1. **Registry**: A Git repo (`host-uk/marketplace`) with category directories
and `index.json` files. Cloned/pulled by CoreGO at startup and periodically.
2. **Manifests**: Each module's `.core/view.yml` is the manifest (replaces `.itw3.json`).
The marketplace index points to the module's Git repo, not a raw file URL.
3. **Distribution**: Git clone of the module repo (not zip downloads). CoreGO
clones into a managed modules directory with depth=1.
4. **Verification**: ed25519 signature in `view.yml` verified before loading.
The marketplace index includes the expected signing public key.
5. **Install hooks**: Declared in `view.yml` under `hooks:`. Executed by CoreGO
in the I/O fortress (rename, template render, permission grant).
6. **Updates**: `git pull` on the module repo. Signature re-verified after pull.
If signature fails, rollback to previous commit.
7. **Discovery**: `core marketplace list`, `core marketplace search <query>`,
`core marketplace install <code>`.
```yaml
# marketplace/index.json
version: 1
modules:
- code: utils-cyberchef
name: CyberChef Data Toolkit
repo: https://forge.lthn.io/host-uk/mod-cyberchef.git
sign_key: <ed25519 public key>
category: utils
categories:
- miner
- utils
- network
```
### RPC Surface (from dAppServer test extraction)
| Operation | CLI | RPC |
|-----------|-----|-----|
| Browse | `core marketplace list` | `marketplace.list(category?)` |
| Search | `core marketplace search <q>` | `marketplace.search(query)` |
| Install | `core marketplace install <code>` | `marketplace.install(code)` |
| Remove | `core marketplace remove <code>` | `marketplace.remove(code)` |
| Installed | `core marketplace installed` | `marketplace.installed()` |
| Update | `core marketplace update <code>` | `marketplace.update(code)` |
| Update all | `core marketplace update` | `marketplace.updateAll()` |
## Complete RPC Surface (Archaeological Extraction)
All procedures discovered from dAppServer test files and controllers:
### Auth
- `auth.create(username, password)` — PGP key generation + QuasiSalt hash
- `auth.login(username, encryptedPayload)` — Zero-knowledge PGP verify → JWT
- `auth.delete(username)` — Remove account
### Crypto
- `crypto.pgp.generateKeyPair(name, email, passphrase)` → {publicKey, privateKey}
- `crypto.pgp.encrypt(data, publicKey)` → encryptedData
- `crypto.pgp.decrypt(data, privateKey, passphrase)` → plaintext
- `crypto.pgp.sign(data, privateKey, passphrase)` → signature
- `crypto.pgp.verify(data, signature, publicKey)` → boolean
### Filesystem
- `fs.list(path, detailed?)` → FileEntry[]
- `fs.read(path)` → content
- `fs.write(path, content)` → boolean
- `fs.delete(path)` → boolean
- `fs.rename(from, to)` → boolean
- `fs.mkdir(path)` → boolean
- `fs.isDir(path)` → boolean
- `fs.ensureDir(path)` → boolean
### Process
- `process.run(command, args, options)` → ProcessHandle
- `process.add(request)` → key
- `process.start(key)` → boolean
- `process.stop(key)` → boolean
- `process.kill(key)` → boolean
- `process.list()` → string[]
- `process.get(key)` → ProcessInfo
- `process.stdout.subscribe(key)` → stream
- `process.stdin.write(key, data)` → void
### Object Store ### Object Store
dAppServer's data persistence model. How does it map to CoreDeno? Options include - `store.get(group, key)` → value
IndexedDB in WebView2, SQLite via Deno, or Go-managed storage via gRPC. - `store.set(group, key, value)` → void
- `store.delete(group, key)` → void
- `store.count(group)` → number
- `store.deleteGroup(group)` → void
- `store.render(template, vars)` → string
### Templated Config Generators ### IPC / Event Bus
The pattern for generating configs from templates. Informs `.core/view.yml` - `ipc.pub.subscribe(channel)` → stream
codegen pipeline and how complex configurations are composed from simpler parts. - `ipc.pub.publish(channel, message)` → void
- `ipc.req.send(channel, message)` → response
- `ipc.push.send(message)` → void
### Git-Based Plugin Marketplace ### Marketplace
Module discovery, versioning, and distribution via Git repositories. To be - `marketplace.list(category?)` → ModuleEntry[]
designed after dAppServer `app-marketplace` code review. - `marketplace.search(query)` → ModuleEntry[]
- `marketplace.install(code)` → boolean
- `marketplace.remove(code)` → boolean
- `marketplace.installed()` → InstalledModule[]
- `marketplace.update(code)` → boolean
## Deliverables ## Deliverables
@ -240,6 +433,8 @@ designed after dAppServer `app-marketplace` code review.
## Not In Scope (Future Phases) ## Not In Scope (Future Phases)
- LEM auto-loading from signed manifests - LEM auto-loading from signed manifests
- Module marketplace server infrastructure - Marketplace server infrastructure (Git-based registry is sufficient)
- Offline-first sync / object store - Offline-first sync (IndexedDB client cache)
- Full Angular removal (Phase 4c) - Full Angular removal (Phase 4c)
- GlusterFS distributed storage (dAppServer aspiration, not needed yet)
- Multi-chain support (Phase 4 is Lethean-only)