docs: complete Phase 4 design — fill open sections from dAppServer archaeology
Polyfills: none needed (WebView2 is Chromium). Object store: Go-managed SQLite via gRPC, 6 operations extracted. Templated config generators: Go text/template in CoreGO. Git-based plugin marketplace: Git repo as registry, clone-based distribution, ed25519 verification, 7 marketplace RPC operations. Complete RPC surface: 40+ procedures across auth, crypto, fs, process, store, IPC, and marketplace namespaces. Heritage table expanded to 4 tiers from 20-repo archaeological survey. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
f5bf7cba81
commit
37b50abf57
1 changed files with 224 additions and 29 deletions
|
|
@ -1,7 +1,7 @@
|
||||||
# Phase 4: CoreDeno + Web Components
|
# Phase 4: CoreDeno + Web Components
|
||||||
|
|
||||||
**Date:** 2026-02-17
|
**Date:** 2026-02-17
|
||||||
**Status:** Approved (open sections pending dAppServer code review)
|
**Status:** Approved
|
||||||
**Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code
|
**Heritage:** dAppServer prototype (20 repos), Chandler/Dreaming in Code
|
||||||
|
|
||||||
## Vision
|
## Vision
|
||||||
|
|
@ -189,40 +189,233 @@ Angular router replaced by lightweight hash-based router mapping URLs to
|
||||||
20 repos at `github.com/dAppServer/` — the original client-side server concept
|
20 repos at `github.com/dAppServer/` — the original client-side server concept
|
||||||
and browser↔Go communications bridge. Extract and port, not copy.
|
and browser↔Go communications bridge. Extract and port, not copy.
|
||||||
|
|
||||||
| dAppServer repo | CoreDeno equivalent | Action |
|
### Tier 1: Extract (Core Architecture)
|
||||||
|
|
||||||
|
| dAppServer repo | What to extract | Phase 4 target |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `server` | CoreDeno sidecar architecture | Extract |
|
| `server` | Port 36911 bridge, ZeroMQ IPC (pub/sub + req/rep + push/pull), air-gapped PGP auth, object store, 13 test files with RPC procedures | CoreDeno sidecar, I/O fortress, auth |
|
||||||
| `dappui` | Web Component framework | Extract |
|
| `dappui` | Angular→WC migration, REST+WS+Wails triple, terminal (xterm.js) | Web Component framework, MCPBridge |
|
||||||
| `auth-server` + `mod-auth` | Signed manifest verification | Extract |
|
| `mod-auth` | PGP zero-knowledge auth (sign→encrypt→verify→JWT), QuasiSalt, roles | Signed manifest verification, identity |
|
||||||
| `mod-docker` | Process management (core/pkg/process) | Port |
|
| `mod-io-process` | Process registry, 3-layer I/O streaming (process→ZeroMQ→WS→browser) | `core/pkg/process`, event bus |
|
||||||
| `mod-io-process` | I/O fortress gRPC bridge | Extract |
|
| `app-marketplace` | Git-as-database registry, category-as-directory, install pipeline | Module registry, `.core/view.yml` loader |
|
||||||
| `server-sdk-typescript-angular` | Deno TypeScript bindings | Port |
|
|
||||||
| `app-marketplace` | Module registry/discovery | Extract |
|
|
||||||
| `app-directory-browser` | File browser Web Component | Port |
|
|
||||||
| `devops` | CI/deployment patterns | Reference |
|
|
||||||
| `app-utils-cyberchef` | Utility module example | Reference |
|
|
||||||
| `app-mining` | Mining module (already in ITW3) | Reference |
|
|
||||||
| `server-sdk-python` | Not needed (Go replaces) | Skip |
|
|
||||||
|
|
||||||
## Open Sections (Pending dAppServer Code Review)
|
### Tier 2: Port (Useful Patterns)
|
||||||
|
|
||||||
These sections will be refined after studying the dAppServer codebase:
|
| dAppServer repo | What to port | Phase 4 target |
|
||||||
|
|---|---|---|
|
||||||
|
| `auth-server` | Keycloak + native PGP fallback | External auth option |
|
||||||
|
| `mod-docker` | Docker socket client, container CRUD (8 ops) | `core/pkg/process` |
|
||||||
|
| `app-mining` | CLI Bridge (camelCase→kebab-case), Process-as-Service, API proxy | Generic CLI wrapper |
|
||||||
|
| `app-directory-browser` | Split-pane layout, lazy tree, filesystem CRUD RPCs | `<core-file-tree>` WC |
|
||||||
|
| `wails-build-action` | Auto-stack detection, cross-platform signing, Deno CI | Build tooling |
|
||||||
|
|
||||||
### Polyfills
|
### Tier 3: Reference
|
||||||
What browser APIs did dAppServer polyfill? Which are still needed in WebView2?
|
|
||||||
WebView2 is Chromium-based so most modern APIs are available natively.
|
| dAppServer repo | Value |
|
||||||
|
|---|---|
|
||||||
|
| `depends` | Bitcoin Core hermetic build; libmultiprocess + Cap'n Proto validates process-separation |
|
||||||
|
| `app-utils-cyberchef` | Purest manifest-only pattern ("manifest IS the app") |
|
||||||
|
| `devops` | Cross-compilation matrix (9 triples), ancestor of ADR-001 |
|
||||||
|
| `pwa-native-action` | PWA→Wails native shell proof, ancestor of `core-gui` |
|
||||||
|
| `docker-images` | C++ cross-compile layers |
|
||||||
|
|
||||||
|
### Tier 4: Skip
|
||||||
|
|
||||||
|
| dAppServer repo | Reason |
|
||||||
|
|---|---|
|
||||||
|
| `server-sdk-python` | Auto-generated, Go replaces |
|
||||||
|
| `server-sdk-typescript-angular` | Auto-generated, superseded |
|
||||||
|
| `openvpn` | Unmodified upstream fork |
|
||||||
|
| `ansible-server-base` | Standard Ansible hardening |
|
||||||
|
| `.github` | Org profile only |
|
||||||
|
|
||||||
|
## Polyfills
|
||||||
|
|
||||||
|
dAppServer polyfilled nothing at the browser level. The prototype ran inside
|
||||||
|
Electron/WebView2 (Chromium), which already supports all required APIs natively:
|
||||||
|
Custom Elements v1, Shadow DOM v1, ES Modules, `fetch`, `WebSocket`,
|
||||||
|
`customElements.define()`, `structuredClone()`.
|
||||||
|
|
||||||
|
**Decision**: No polyfills needed. WebView2 is Chromium-based. The minimum
|
||||||
|
Chromium version Wails v3 targets already supports all Web Component APIs.
|
||||||
|
|
||||||
|
## Object Store
|
||||||
|
|
||||||
|
dAppServer used a file-based JSON key-value store at `data/objects/{group}/{object}.json`.
|
||||||
|
Six operations discovered from test files:
|
||||||
|
|
||||||
|
| Operation | dAppServer endpoint | Phase 4 equivalent |
|
||||||
|
|-----------|--------------------|--------------------|
|
||||||
|
| Get | `GET /config/object/{group}/{object}` | `store.get(group, key)` |
|
||||||
|
| Set | `POST /config/object/{group}/{object}` | `store.set(group, key, value)` |
|
||||||
|
| Clear | `DELETE /config/object/{group}/{object}` | `store.delete(group, key)` |
|
||||||
|
| Count | `GET /config/object/{group}/count` | `store.count(group)` |
|
||||||
|
| Remove group | `DELETE /config/object/{group}` | `store.deleteGroup(group)` |
|
||||||
|
| Render template | `POST /config/render` | `store.render(template, vars)` |
|
||||||
|
|
||||||
|
Used for: installed app registry (`conf/installed-apps.json`), menu state
|
||||||
|
(`conf/menu.json`), per-module config, user preferences.
|
||||||
|
|
||||||
|
**Decision**: Go-managed storage via gRPC. CoreGO owns persistence through
|
||||||
|
`core/pkg/io`. Modules request storage through the I/O fortress — never
|
||||||
|
touching the filesystem directly. SQLite backend (already a dependency in
|
||||||
|
the blockchain layer). IndexedDB reserved for client-side cache only.
|
||||||
|
|
||||||
|
## Templated Config Generators
|
||||||
|
|
||||||
|
dAppServer's `config.render` endpoint accepted a template string + variable map
|
||||||
|
and returned the rendered output. Used to generate configs for managed processes
|
||||||
|
(e.g., xmrig config.json from user-selected pool/wallet parameters).
|
||||||
|
|
||||||
|
The pattern in Phase 4:
|
||||||
|
1. Module declares config templates in `.core/view.yml` under a `config:` key
|
||||||
|
2. User preferences stored in the object store
|
||||||
|
3. CoreGO renders templates at process-start time via Go `text/template`
|
||||||
|
4. Rendered configs written to sandboxed paths the module has `write` permission for
|
||||||
|
|
||||||
|
Example from mining module (camelCase→kebab-case CLI arg transformation):
|
||||||
|
```yaml
|
||||||
|
config:
|
||||||
|
xmrig:
|
||||||
|
template: conf/xmrig/config.json.tmpl
|
||||||
|
vars:
|
||||||
|
pool: "{{ .user.pool }}"
|
||||||
|
wallet: "{{ .user.wallet }}"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Decision**: Go `text/template` in CoreGO. Templates live in the module's
|
||||||
|
`.core/` directory. Variables come from the object store. No Deno involvement —
|
||||||
|
config rendering is a Go-side I/O fortress operation.
|
||||||
|
|
||||||
|
## Git-Based Plugin Marketplace
|
||||||
|
|
||||||
|
### dAppServer Pattern (Extracted from `app-marketplace`)
|
||||||
|
|
||||||
|
A Git repository serves as the package registry. No server infrastructure needed.
|
||||||
|
|
||||||
|
```
|
||||||
|
marketplace/ # Git repo
|
||||||
|
├── index.json # Root: {version, apps[], dirs[]}
|
||||||
|
├── miner/
|
||||||
|
│ └── index.json # Category: {version, apps[], dirs[]}
|
||||||
|
├── utils/
|
||||||
|
│ └── index.json # Category: {version, apps[], dirs[]}
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Each `index.json` entry points to a raw `.itw3.json` URL in the plugin's own repo:
|
||||||
|
```json
|
||||||
|
{"code": "utils-cyberchef", "name": "CyberChef", "type": "bin",
|
||||||
|
"pkg": "https://raw.githubusercontent.com/dAppServer/app-utils-cyberchef/main/.itw3.json"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Install pipeline: browse index → fetch manifest → download zip from `app.url` →
|
||||||
|
extract → run hooks (rename, etc.) → register in object store → add menu entry.
|
||||||
|
|
||||||
|
### Phase 4 Evolution
|
||||||
|
|
||||||
|
Replace GitHub-specific URLs with Forgejo-compatible Git operations:
|
||||||
|
|
||||||
|
1. **Registry**: A Git repo (`host-uk/marketplace`) with category directories
|
||||||
|
and `index.json` files. Cloned/pulled by CoreGO at startup and periodically.
|
||||||
|
2. **Manifests**: Each module's `.core/view.yml` is the manifest (replaces `.itw3.json`).
|
||||||
|
The marketplace index points to the module's Git repo, not a raw file URL.
|
||||||
|
3. **Distribution**: Git clone of the module repo (not zip downloads). CoreGO
|
||||||
|
clones into a managed modules directory with depth=1.
|
||||||
|
4. **Verification**: ed25519 signature in `view.yml` verified before loading.
|
||||||
|
The marketplace index includes the expected signing public key.
|
||||||
|
5. **Install hooks**: Declared in `view.yml` under `hooks:`. Executed by CoreGO
|
||||||
|
in the I/O fortress (rename, template render, permission grant).
|
||||||
|
6. **Updates**: `git pull` on the module repo. Signature re-verified after pull.
|
||||||
|
If signature fails, rollback to previous commit.
|
||||||
|
7. **Discovery**: `core marketplace list`, `core marketplace search <query>`,
|
||||||
|
`core marketplace install <code>`.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# marketplace/index.json
|
||||||
|
version: 1
|
||||||
|
modules:
|
||||||
|
- code: utils-cyberchef
|
||||||
|
name: CyberChef Data Toolkit
|
||||||
|
repo: https://forge.lthn.io/host-uk/mod-cyberchef.git
|
||||||
|
sign_key: <ed25519 public key>
|
||||||
|
category: utils
|
||||||
|
categories:
|
||||||
|
- miner
|
||||||
|
- utils
|
||||||
|
- network
|
||||||
|
```
|
||||||
|
|
||||||
|
### RPC Surface (from dAppServer test extraction)
|
||||||
|
|
||||||
|
| Operation | CLI | RPC |
|
||||||
|
|-----------|-----|-----|
|
||||||
|
| Browse | `core marketplace list` | `marketplace.list(category?)` |
|
||||||
|
| Search | `core marketplace search <q>` | `marketplace.search(query)` |
|
||||||
|
| Install | `core marketplace install <code>` | `marketplace.install(code)` |
|
||||||
|
| Remove | `core marketplace remove <code>` | `marketplace.remove(code)` |
|
||||||
|
| Installed | `core marketplace installed` | `marketplace.installed()` |
|
||||||
|
| Update | `core marketplace update <code>` | `marketplace.update(code)` |
|
||||||
|
| Update all | `core marketplace update` | `marketplace.updateAll()` |
|
||||||
|
|
||||||
|
## Complete RPC Surface (Archaeological Extraction)
|
||||||
|
|
||||||
|
All procedures discovered from dAppServer test files and controllers:
|
||||||
|
|
||||||
|
### Auth
|
||||||
|
- `auth.create(username, password)` — PGP key generation + QuasiSalt hash
|
||||||
|
- `auth.login(username, encryptedPayload)` — Zero-knowledge PGP verify → JWT
|
||||||
|
- `auth.delete(username)` — Remove account
|
||||||
|
|
||||||
|
### Crypto
|
||||||
|
- `crypto.pgp.generateKeyPair(name, email, passphrase)` → {publicKey, privateKey}
|
||||||
|
- `crypto.pgp.encrypt(data, publicKey)` → encryptedData
|
||||||
|
- `crypto.pgp.decrypt(data, privateKey, passphrase)` → plaintext
|
||||||
|
- `crypto.pgp.sign(data, privateKey, passphrase)` → signature
|
||||||
|
- `crypto.pgp.verify(data, signature, publicKey)` → boolean
|
||||||
|
|
||||||
|
### Filesystem
|
||||||
|
- `fs.list(path, detailed?)` → FileEntry[]
|
||||||
|
- `fs.read(path)` → content
|
||||||
|
- `fs.write(path, content)` → boolean
|
||||||
|
- `fs.delete(path)` → boolean
|
||||||
|
- `fs.rename(from, to)` → boolean
|
||||||
|
- `fs.mkdir(path)` → boolean
|
||||||
|
- `fs.isDir(path)` → boolean
|
||||||
|
- `fs.ensureDir(path)` → boolean
|
||||||
|
|
||||||
|
### Process
|
||||||
|
- `process.run(command, args, options)` → ProcessHandle
|
||||||
|
- `process.add(request)` → key
|
||||||
|
- `process.start(key)` → boolean
|
||||||
|
- `process.stop(key)` → boolean
|
||||||
|
- `process.kill(key)` → boolean
|
||||||
|
- `process.list()` → string[]
|
||||||
|
- `process.get(key)` → ProcessInfo
|
||||||
|
- `process.stdout.subscribe(key)` → stream
|
||||||
|
- `process.stdin.write(key, data)` → void
|
||||||
|
|
||||||
### Object Store
|
### Object Store
|
||||||
dAppServer's data persistence model. How does it map to CoreDeno? Options include
|
- `store.get(group, key)` → value
|
||||||
IndexedDB in WebView2, SQLite via Deno, or Go-managed storage via gRPC.
|
- `store.set(group, key, value)` → void
|
||||||
|
- `store.delete(group, key)` → void
|
||||||
|
- `store.count(group)` → number
|
||||||
|
- `store.deleteGroup(group)` → void
|
||||||
|
- `store.render(template, vars)` → string
|
||||||
|
|
||||||
### Templated Config Generators
|
### IPC / Event Bus
|
||||||
The pattern for generating configs from templates. Informs `.core/view.yml`
|
- `ipc.pub.subscribe(channel)` → stream
|
||||||
codegen pipeline and how complex configurations are composed from simpler parts.
|
- `ipc.pub.publish(channel, message)` → void
|
||||||
|
- `ipc.req.send(channel, message)` → response
|
||||||
|
- `ipc.push.send(message)` → void
|
||||||
|
|
||||||
### Git-Based Plugin Marketplace
|
### Marketplace
|
||||||
Module discovery, versioning, and distribution via Git repositories. To be
|
- `marketplace.list(category?)` → ModuleEntry[]
|
||||||
designed after dAppServer `app-marketplace` code review.
|
- `marketplace.search(query)` → ModuleEntry[]
|
||||||
|
- `marketplace.install(code)` → boolean
|
||||||
|
- `marketplace.remove(code)` → boolean
|
||||||
|
- `marketplace.installed()` → InstalledModule[]
|
||||||
|
- `marketplace.update(code)` → boolean
|
||||||
|
|
||||||
## Deliverables
|
## Deliverables
|
||||||
|
|
||||||
|
|
@ -240,6 +433,8 @@ designed after dAppServer `app-marketplace` code review.
|
||||||
## Not In Scope (Future Phases)
|
## Not In Scope (Future Phases)
|
||||||
|
|
||||||
- LEM auto-loading from signed manifests
|
- LEM auto-loading from signed manifests
|
||||||
- Module marketplace server infrastructure
|
- Marketplace server infrastructure (Git-based registry is sufficient)
|
||||||
- Offline-first sync / object store
|
- Offline-first sync (IndexedDB client cache)
|
||||||
- Full Angular removal (Phase 4c)
|
- Full Angular removal (Phase 4c)
|
||||||
|
- GlusterFS distributed storage (dAppServer aspiration, not needed yet)
|
||||||
|
- Multi-chain support (Phase 4 is Lethean-only)
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue