docs: remove implemented plan/spec files
Co-Authored-By: Virgil <virgil@lethean.io>
This commit is contained in:
Snider
parent
b9500bf866
commit
d828c6356a
10 changed files with 0 additions and 4529 deletions
|
|
@ -1,788 +0,0 @@
|
|||
# core/api Polyglot Merge — Implementation Plan
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Design:** [2026-03-14-api-polyglot-merge-design.md](../specs/2026-03-14-api-polyglot-merge-design.md)
|
||||
**Pattern:** Follows core/mcp merge (2026-03-09)
|
||||
**Co-author:** Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
|
||||
## Source Inventory
|
||||
|
||||
### Go side (`core/go-api`)
|
||||
|
||||
Module: `forge.lthn.ai/core/go-api` (Go 1.26)
|
||||
|
||||
Root-level `.go` files (50 files):
|
||||
- Core: `api.go`, `group.go`, `options.go`, `response.go`, `middleware.go`
|
||||
- Features: `authentik.go`, `bridge.go`, `brotli.go`, `cache.go`, `codegen.go`, `export.go`, `graphql.go`, `i18n.go`, `openapi.go`, `sse.go`, `swagger.go`, `tracing.go`, `websocket.go`
|
||||
- Tests: `*_test.go` (26 files including `race_test.go`, `norace_test.go`)
|
||||
- Subdirs: `cmd/api/` (CLI commands), `docs/` (4 markdown files), `.core/` (build.yaml, release.yaml)
|
||||
|
||||
Key dependencies: `forge.lthn.ai/core/cli`, `github.com/gin-gonic/gin`, OpenTelemetry, Casbin, OIDC
|
||||
|
||||
### PHP side (`core/php-api`)
|
||||
|
||||
Package: `lthn/php-api` (PHP 8.2+, requires `lthn/php`)
|
||||
|
||||
Three namespace roots:
|
||||
- `Core\Api\` (`src/Api/`) — Boot, Controllers, Middleware, Models, Services, Documentation, RateLimit, Tests
|
||||
- `Core\Front\Api\` (`src/Front/Api/`) — API versioning frontage, auto-discovered provider
|
||||
- `Core\Website\Api\` (`src/Website/Api/`) — Documentation UI, Blade views, web routes
|
||||
|
||||
### Consumer repos (Go)
|
||||
|
||||
| Repo | Dependency | Go source imports | Notes |
|
||||
|------|-----------|-------------------|-------|
|
||||
| `core/go-ml` | direct | `api/routes.go`, `api/routes_test.go` | Aliased as `goapi` |
|
||||
| `core/mcp` | direct | `pkg/mcp/bridge.go`, `pkg/mcp/bridge_test.go` | Aliased as `api` |
|
||||
| `core/go-ai` | direct (go.mod only) | None | go.mod + go.sum + docs reference |
|
||||
| `core/ide` | indirect | None | Transitive via go-ai or mcp |
|
||||
| `core/agent` | indirect (v0.0.3) | None | Transitive |
|
||||
|
||||
### Consumer repos (PHP)
|
||||
|
||||
| Repo / App | Dependency | Notes |
|
||||
|------------|-----------|-------|
|
||||
| `host.uk.com` (Laravel app) | `core/php-api: *` | composer.json + repositories block |
|
||||
| `core/php-template` | `lthn/php-api: dev-main` | Starter template |
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — Clone core/api and copy Go files from go-api
|
||||
|
||||
Clone the empty target repo and copy all Go source files at root level (Option B from the design spec — no `pkg/api/` nesting).
|
||||
|
||||
```bash
|
||||
cd ~/Code/core
|
||||
git clone ssh://git@forge.lthn.ai:2223/core/api.git
|
||||
cd api
|
||||
|
||||
# Copy all Go source files (root level)
|
||||
cp ~/Code/core/go-api/*.go .
|
||||
|
||||
# Copy cmd/ directory
|
||||
cp -r ~/Code/core/go-api/cmd .
|
||||
|
||||
# Copy docs/
|
||||
cp -r ~/Code/core/go-api/docs .
|
||||
|
||||
# Copy .core/ build config
|
||||
mkdir -p .core
|
||||
cp ~/Code/core/go-api/.core/build.yaml .core/build.yaml
|
||||
cp ~/Code/core/go-api/.core/release.yaml .core/release.yaml
|
||||
|
||||
# Copy go.sum as starting point
|
||||
cp ~/Code/core/go-api/go.sum .
|
||||
```
|
||||
|
||||
Update `.core/build.yaml`:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
|
||||
project:
|
||||
name: core-api
|
||||
description: REST API framework (Go + PHP)
|
||||
binary: ""
|
||||
|
||||
build:
|
||||
cgo: false
|
||||
flags:
|
||||
- -trimpath
|
||||
ldflags:
|
||||
- -s
|
||||
- -w
|
||||
|
||||
targets:
|
||||
- os: linux
|
||||
arch: amd64
|
||||
- os: linux
|
||||
arch: arm64
|
||||
- os: darwin
|
||||
arch: arm64
|
||||
- os: windows
|
||||
arch: amd64
|
||||
```
|
||||
|
||||
Update `.core/release.yaml`:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
|
||||
project:
|
||||
name: core-api
|
||||
repository: core/api
|
||||
|
||||
publishers: []
|
||||
|
||||
changelog:
|
||||
include:
|
||||
- feat
|
||||
- fix
|
||||
- perf
|
||||
- refactor
|
||||
exclude:
|
||||
- chore
|
||||
- docs
|
||||
- style
|
||||
- test
|
||||
- ci
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
ls *.go | wc -l # Should be 50 files
|
||||
ls cmd/api/ # Should have cmd.go, cmd_sdk.go, cmd_spec.go, cmd_test.go
|
||||
ls docs/ # architecture.md, development.md, history.md, index.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — Copy PHP files from php-api into src/php/
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/api
|
||||
|
||||
# Create PHP directory structure
|
||||
mkdir -p src/php/src
|
||||
mkdir -p src/php/tests
|
||||
|
||||
# Copy PHP source namespaces
|
||||
cp -r ~/Code/core/php-api/src/Api src/php/src/Api
|
||||
cp -r ~/Code/core/php-api/src/Front src/php/src/Front
|
||||
cp -r ~/Code/core/php-api/src/Website src/php/src/Website
|
||||
|
||||
# Copy PHP test infrastructure
|
||||
cp -r ~/Code/core/php-api/tests/Feature src/php/tests/Feature 2>/dev/null || true
|
||||
cp -r ~/Code/core/php-api/tests/Unit src/php/tests/Unit 2>/dev/null || true
|
||||
cp ~/Code/core/php-api/tests/TestCase.php src/php/tests/TestCase.php 2>/dev/null || true
|
||||
|
||||
# Copy phpunit.xml (adjust paths for new location)
|
||||
cp ~/Code/core/php-api/phpunit.xml src/php/phpunit.xml
|
||||
```
|
||||
|
||||
Edit `src/php/phpunit.xml` — update the `<directory>` paths:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
|
||||
bootstrap="vendor/autoload.php"
|
||||
colors="true"
|
||||
>
|
||||
<testsuites>
|
||||
<testsuite name="Unit">
|
||||
<directory>tests/Unit</directory>
|
||||
</testsuite>
|
||||
<testsuite name="Feature">
|
||||
<directory>tests/Feature</directory>
|
||||
</testsuite>
|
||||
</testsuites>
|
||||
<source>
|
||||
<include>
|
||||
<directory>src</directory>
|
||||
</include>
|
||||
</source>
|
||||
<php>
|
||||
<env name="APP_ENV" value="testing"/>
|
||||
<env name="APP_MAINTENANCE_DRIVER" value="file"/>
|
||||
<env name="BCRYPT_ROUNDS" value="4"/>
|
||||
<env name="CACHE_STORE" value="array"/>
|
||||
<env name="DB_CONNECTION" value="sqlite"/>
|
||||
<env name="DB_DATABASE" value=":memory:"/>
|
||||
<env name="MAIL_MAILER" value="array"/>
|
||||
<env name="PULSE_ENABLED" value="false"/>
|
||||
<env name="QUEUE_CONNECTION" value="sync"/>
|
||||
<env name="SESSION_DRIVER" value="array"/>
|
||||
<env name="TELESCOPE_ENABLED" value="false"/>
|
||||
</php>
|
||||
</phpunit>
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
ls src/php/src/ # Api, Front, Website
|
||||
ls src/php/src/Api/ # Boot.php, Controllers/, Middleware/, Models/, etc.
|
||||
ls src/php/src/Front/Api/ # Boot.php, ApiVersionService.php, Middleware/, etc.
|
||||
ls src/php/src/Website/Api/ # Boot.php, Controllers/, Views/, etc.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — Create go.mod, composer.json, and .gitattributes
|
||||
|
||||
### go.mod
|
||||
|
||||
Create `go.mod` with the new module path. Start from the existing go.mod and change the module line:
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/api
|
||||
sed 's|module forge.lthn.ai/core/go-api|module forge.lthn.ai/core/api|' \
|
||||
~/Code/core/go-api/go.mod > go.mod
|
||||
```
|
||||
|
||||
The first line should now read:
|
||||
|
||||
```
|
||||
module forge.lthn.ai/core/api
|
||||
```
|
||||
|
||||
Then tidy:
|
||||
|
||||
```bash
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### composer.json
|
||||
|
||||
Create `composer.json` following the core/mcp pattern:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "lthn/api",
|
||||
"description": "REST API module — Laravel API layer + standalone Go binary",
|
||||
"keywords": ["api", "rest", "laravel", "openapi"],
|
||||
"license": "EUPL-1.2",
|
||||
"require": {
|
||||
"php": "^8.2",
|
||||
"lthn/php": "*",
|
||||
"symfony/yaml": "^7.0"
|
||||
},
|
||||
"autoload": {
|
||||
"psr-4": {
|
||||
"Core\\Api\\": "src/php/src/Api/",
|
||||
"Core\\Front\\Api\\": "src/php/src/Front/Api/",
|
||||
"Core\\Website\\Api\\": "src/php/src/Website/Api/"
|
||||
}
|
||||
},
|
||||
"autoload-dev": {
|
||||
"psr-4": {
|
||||
"Core\\Api\\Tests\\": "src/php/tests/"
|
||||
}
|
||||
},
|
||||
"extra": {
|
||||
"laravel": {
|
||||
"providers": [
|
||||
"Core\\Front\\Api\\Boot"
|
||||
]
|
||||
}
|
||||
},
|
||||
"config": {
|
||||
"sort-packages": true
|
||||
},
|
||||
"minimum-stability": "dev",
|
||||
"prefer-stable": true,
|
||||
"replace": {
|
||||
"core/php-api": "self.version",
|
||||
"lthn/php-api": "self.version"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### .gitattributes
|
||||
|
||||
Create `.gitattributes` matching the core/mcp pattern — exclude Go files from Composer installs:
|
||||
|
||||
```
|
||||
*.go export-ignore
|
||||
go.mod export-ignore
|
||||
go.sum export-ignore
|
||||
cmd/ export-ignore
|
||||
pkg/ export-ignore
|
||||
.core/ export-ignore
|
||||
src/php/tests/ export-ignore
|
||||
```
|
||||
|
||||
### .gitignore
|
||||
|
||||
```
|
||||
# Binaries
|
||||
core-api
|
||||
*.exe
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
|
||||
# Go
|
||||
vendor/
|
||||
|
||||
# PHP
|
||||
/vendor/
|
||||
node_modules/
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
head -1 go.mod # module forge.lthn.ai/core/api
|
||||
cat composer.json | python3 -m json.tool # Valid JSON
|
||||
cat .gitattributes # 7 export-ignore lines
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — Create CLAUDE.md for the polyglot repo
|
||||
|
||||
Create `CLAUDE.md` at the repo root:
|
||||
|
||||
```markdown
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Core API is the REST framework for the Lethean ecosystem, providing both a **Go HTTP engine** (Gin-based, with OpenAPI generation, WebSocket/SSE, ToolBridge) and a **PHP Laravel package** (rate limiting, webhooks, API key management, OpenAPI documentation). Both halves serve the same purpose in their respective stacks.
|
||||
|
||||
Module: `forge.lthn.ai/core/api` | Package: `lthn/api` | Licence: EUPL-1.2
|
||||
|
||||
## Build and Test Commands
|
||||
|
||||
### Go
|
||||
|
||||
```bash
|
||||
core build # Build binary (if cmd/ has main)
|
||||
go build ./... # Build library
|
||||
|
||||
core go test # Run all Go tests
|
||||
core go test --run TestName # Run a single test
|
||||
core go cov # Coverage report
|
||||
core go cov --open # Open HTML coverage in browser
|
||||
core go qa # Format + vet + lint + test
|
||||
core go qa full # Also race detector, vuln scan, security audit
|
||||
core go fmt # gofmt
|
||||
core go lint # golangci-lint
|
||||
core go vet # go vet
|
||||
```
|
||||
|
||||
### PHP (from repo root)
|
||||
|
||||
```bash
|
||||
composer test # Run all PHP tests (Pest)
|
||||
composer test -- --filter=ApiKey # Single test
|
||||
composer lint # Laravel Pint (PSR-12)
|
||||
./vendor/bin/pint --dirty # Format changed files
|
||||
```
|
||||
|
||||
Tests live in `src/php/src/Api/Tests/Feature/` (in-source) and `src/php/tests/` (standalone).
|
||||
|
||||
## Architecture
|
||||
|
||||
### Go Engine (root-level .go files)
|
||||
|
||||
`Engine` is the central type, configured via functional `Option` functions passed to `New()`:
|
||||
|
||||
```go
|
||||
engine, _ := api.New(api.WithAddr(":8080"), api.WithCORS("*"), api.WithSwagger(...))
|
||||
engine.Register(myRouteGroup)
|
||||
engine.Serve(ctx)
|
||||
```
|
||||
|
||||
**Extension interfaces** (`group.go`):
|
||||
- `RouteGroup` — minimum: `Name()`, `BasePath()`, `RegisterRoutes(*gin.RouterGroup)`
|
||||
- `StreamGroup` — optional: `Channels() []string` for WebSocket
|
||||
- `DescribableGroup` — extends RouteGroup with `Describe() []RouteDescription` for OpenAPI
|
||||
|
||||
**ToolBridge** (`bridge.go`): Converts `ToolDescriptor` structs into `POST /{tool_name}` REST endpoints with auto-generated OpenAPI paths.
|
||||
|
||||
**Authentication** (`authentik.go`): Authentik OIDC integration + static bearer token. Permissive middleware with `RequireAuth()` / `RequireGroup()` guards.
|
||||
|
||||
**OpenAPI** (`openapi.go`, `export.go`, `codegen.go`): `SpecBuilder.Build()` generates OpenAPI 3.1 JSON. `SDKGenerator` wraps openapi-generator-cli for 11 languages.
|
||||
|
||||
**CLI** (`cmd/api/`): Registers `core api spec` and `core api sdk` commands.
|
||||
|
||||
### PHP Package (`src/php/`)
|
||||
|
||||
Three namespace roots:
|
||||
|
||||
| Namespace | Path | Role |
|
||||
|-----------|------|------|
|
||||
| `Core\Front\Api` | `src/php/src/Front/Api/` | API frontage — middleware, versioning, auto-discovered provider |
|
||||
| `Core\Api` | `src/php/src/Api/` | Backend — auth, scopes, models, webhooks, OpenAPI docs |
|
||||
| `Core\Website\Api` | `src/php/src/Website/Api/` | Documentation UI — controllers, Blade views, web routes |
|
||||
|
||||
Boot chain: `Front\Api\Boot` (auto-discovered) fires `ApiRoutesRegistering` → `Api\Boot` registers middleware and routes.
|
||||
|
||||
Key services: `WebhookService`, `RateLimitService`, `IpRestrictionService`, `OpenApiBuilder`, `ApiKeyService`.
|
||||
|
||||
## Conventions
|
||||
|
||||
- **UK English** in all user-facing strings and docs (colour, organisation, unauthorised)
|
||||
- **SPDX headers** in Go files: `// SPDX-License-Identifier: EUPL-1.2`
|
||||
- **`declare(strict_types=1);`** in every PHP file
|
||||
- **Full type hints** on all PHP parameters and return types
|
||||
- **Pest syntax** for PHP tests (not PHPUnit)
|
||||
- **Flux Pro** components in Livewire views; **Font Awesome** icons
|
||||
- **Conventional commits**: `type(scope): description`
|
||||
- **Co-Author**: `Co-Authored-By: Virgil <virgil@lethean.io>`
|
||||
- Go test names use `_Good` / `_Bad` / `_Ugly` suffixes
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
| Go module | Role |
|
||||
|-----------|------|
|
||||
| `forge.lthn.ai/core/cli` | CLI command registration |
|
||||
| `github.com/gin-gonic/gin` | HTTP router |
|
||||
| `github.com/casbin/casbin/v2` | Authorisation policies |
|
||||
| `github.com/coreos/go-oidc/v3` | OIDC / Authentik |
|
||||
| `go.opentelemetry.io/otel` | OpenTelemetry tracing |
|
||||
|
||||
PHP: `lthn/php` (Core framework), Laravel 12, `symfony/yaml`.
|
||||
|
||||
Go workspace: this module is part of `~/Code/go.work`. Requires Go 1.26+, PHP 8.2+.
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
wc -l CLAUDE.md # Should be ~100 lines
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — Update consumer repos (Go import paths)
|
||||
|
||||
Change `forge.lthn.ai/core/go-api` to `forge.lthn.ai/core/api` in all consumers.
|
||||
|
||||
### 5a — core/go-ml (2 source files + go.mod)
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/go-ml
|
||||
|
||||
# Update Go source imports
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' \
|
||||
api/routes.go api/routes_test.go
|
||||
|
||||
# Update go.mod
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' go.mod
|
||||
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Verify the import alias in `api/routes.go` reads:
|
||||
```go
|
||||
goapi "forge.lthn.ai/core/api"
|
||||
```
|
||||
|
||||
### 5b — core/mcp (2 source files + go.mod)
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/mcp
|
||||
|
||||
# Update Go source imports
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' \
|
||||
pkg/mcp/bridge.go pkg/mcp/bridge_test.go
|
||||
|
||||
# Update go.mod
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' go.mod
|
||||
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Verify the import alias in `pkg/mcp/bridge.go` reads:
|
||||
```go
|
||||
api "forge.lthn.ai/core/api"
|
||||
```
|
||||
|
||||
### 5c — core/go-ai (go.mod only — no Go source imports)
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/go-ai
|
||||
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' go.mod
|
||||
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
Also update the docs reference:
|
||||
```bash
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' docs/index.md
|
||||
```
|
||||
|
||||
### 5d — core/ide (indirect only — go.mod)
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/ide
|
||||
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' go.mod
|
||||
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### 5e — core/agent (indirect only — go.mod)
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/agent
|
||||
|
||||
sed -i '' 's|forge.lthn.ai/core/go-api|forge.lthn.ai/core/api|g' go.mod
|
||||
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### 5f — Update CLAUDE.md references in core/mcp
|
||||
|
||||
In `~/Code/core/mcp/CLAUDE.md`, update the dependency table:
|
||||
|
||||
```
|
||||
| `forge.lthn.ai/core/go-api` | REST framework + `ToolBridge` |
|
||||
```
|
||||
becomes:
|
||||
```
|
||||
| `forge.lthn.ai/core/api` | REST framework + `ToolBridge` |
|
||||
```
|
||||
|
||||
### 5g — PHP consumers
|
||||
|
||||
**host.uk.com Laravel app** (`~/Code/lab/host.uk.com/composer.json`):
|
||||
|
||||
Replace the `core/php-api` requirement and repository entry:
|
||||
- Change `"core/php-api": "*"` to `"lthn/api": "dev-main"`
|
||||
- Change repository URL from `ssh://git@forge.lthn.ai:2223/core/php-api.git` to `ssh://git@forge.lthn.ai:2223/core/api.git`
|
||||
|
||||
The `replace` block in the new `composer.json` (`"core/php-api": "self.version"`, `"lthn/php-api": "self.version"`) ensures backward compatibility.
|
||||
|
||||
**core/php-template** (`~/Code/core/php-template/composer.json`):
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/php-template
|
||||
sed -i '' 's|"lthn/php-api": "dev-main"|"lthn/api": "dev-main"|g' composer.json
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
# Confirm no stale references remain
|
||||
grep -r 'forge.lthn.ai/core/go-api' ~/Code/core/go-ml/ --include='*.go' --include='go.mod'
|
||||
grep -r 'forge.lthn.ai/core/go-api' ~/Code/core/mcp/ --include='*.go' --include='go.mod'
|
||||
grep -r 'forge.lthn.ai/core/go-api' ~/Code/core/go-ai/ --include='*.go' --include='go.mod'
|
||||
grep -r 'forge.lthn.ai/core/go-api' ~/Code/core/ide/ --include='*.go' --include='go.mod'
|
||||
grep -r 'forge.lthn.ai/core/go-api' ~/Code/core/agent/ --include='*.go' --include='go.mod'
|
||||
# All should return nothing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6 — Update go.work
|
||||
|
||||
Edit `~/Code/go.work`:
|
||||
|
||||
```bash
|
||||
cd ~/Code
|
||||
|
||||
# Remove go-api entry, add api entry
|
||||
sed -i '' 's|./core/go-api|./core/api|' go.work
|
||||
|
||||
# Sync workspace
|
||||
go work sync
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
```bash
|
||||
grep 'core/api' ~/Code/go.work # Should show ./core/api
|
||||
grep 'core/go-api' ~/Code/go.work # Should return nothing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7 — Build verification across all affected repos
|
||||
|
||||
Run in sequence — each depends on the workspace being consistent:
|
||||
|
||||
```bash
|
||||
cd ~/Code
|
||||
|
||||
# 1. Verify core/api itself builds and tests pass
|
||||
cd ~/Code/core/api
|
||||
go build ./...
|
||||
go test ./...
|
||||
go vet ./...
|
||||
|
||||
# 2. Verify core/go-ml
|
||||
cd ~/Code/core/go-ml
|
||||
go build ./...
|
||||
go test ./...
|
||||
|
||||
# 3. Verify core/mcp
|
||||
cd ~/Code/core/mcp
|
||||
go build ./...
|
||||
go test ./...
|
||||
|
||||
# 4. Verify core/go-ai
|
||||
cd ~/Code/core/go-ai
|
||||
go build ./...
|
||||
|
||||
# 5. Verify core/ide
|
||||
cd ~/Code/core/ide
|
||||
go build ./...
|
||||
|
||||
# 6. Verify core/agent
|
||||
cd ~/Code/core/agent
|
||||
go build ./...
|
||||
```
|
||||
|
||||
If any fail, fix import paths and re-run `go mod tidy` in the affected repo.
|
||||
|
||||
### Verification
|
||||
|
||||
All six builds should succeed with zero errors. Test failures unrelated to the import path change can be ignored (pre-existing).
|
||||
|
||||
---
|
||||
|
||||
## Task 8 — Archive go-api and php-api on Forge
|
||||
|
||||
Archive both source repos on Forge. This marks them read-only but keeps them accessible for existing consumers that haven't updated.
|
||||
|
||||
```bash
|
||||
# Archive via Forgejo API (requires GITEA_TOKEN or gh-equivalent)
|
||||
# Option A: Forgejo web UI
|
||||
# 1. Navigate to https://forge.lthn.ai/core/go-api/settings
|
||||
# 2. Click "Archive this repository"
|
||||
# 3. Repeat for https://forge.lthn.ai/core/php-api/settings
|
||||
|
||||
# Option B: Forgejo API
|
||||
curl -X PATCH \
|
||||
-H "Authorization: token ${FORGE_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"archived": true}' \
|
||||
"https://forge.lthn.ai/api/v1/repos/core/go-api"
|
||||
|
||||
curl -X PATCH \
|
||||
-H "Authorization: token ${FORGE_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"archived": true}' \
|
||||
"https://forge.lthn.ai/api/v1/repos/core/php-api"
|
||||
```
|
||||
|
||||
Update repo descriptions to point to the merged repo:
|
||||
|
||||
```bash
|
||||
curl -X PATCH \
|
||||
-H "Authorization: token ${FORGE_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"description": "ARCHIVED — merged into core/api"}' \
|
||||
"https://forge.lthn.ai/api/v1/repos/core/go-api"
|
||||
|
||||
curl -X PATCH \
|
||||
-H "Authorization: token ${FORGE_TOKEN}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"description": "ARCHIVED — merged into core/api"}' \
|
||||
"https://forge.lthn.ai/api/v1/repos/core/php-api"
|
||||
```
|
||||
|
||||
### Local cleanup
|
||||
|
||||
```bash
|
||||
# Optionally remove the old local clones (or keep for reference)
|
||||
# rm -rf ~/Code/core/go-api ~/Code/core/php-api
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Commit Strategy
|
||||
|
||||
Initial commit in core/api:
|
||||
|
||||
```bash
|
||||
cd ~/Code/core/api
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
feat(api): merge go-api + php-api into polyglot repo
|
||||
|
||||
Go source at root level (Option B), PHP under src/php/.
|
||||
Module path: forge.lthn.ai/core/api
|
||||
Package name: lthn/api
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
git push origin main
|
||||
```
|
||||
|
||||
Consumer updates (one commit per repo):
|
||||
|
||||
```bash
|
||||
# core/go-ml
|
||||
cd ~/Code/core/go-ml
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(api): update import path from go-api to core/api
|
||||
|
||||
Part of the polyglot merge — forge.lthn.ai/core/go-api is now
|
||||
forge.lthn.ai/core/api.
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
|
||||
# core/mcp
|
||||
cd ~/Code/core/mcp
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(api): update import path from go-api to core/api
|
||||
|
||||
Part of the polyglot merge — forge.lthn.ai/core/go-api is now
|
||||
forge.lthn.ai/core/api.
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
|
||||
# core/go-ai
|
||||
cd ~/Code/core/go-ai
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(api): update import path from go-api to core/api
|
||||
|
||||
Part of the polyglot merge — forge.lthn.ai/core/go-api is now
|
||||
forge.lthn.ai/core/api.
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
|
||||
# core/ide
|
||||
cd ~/Code/core/ide
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(api): update import path from go-api to core/api
|
||||
|
||||
Part of the polyglot merge — forge.lthn.ai/core/go-api is now
|
||||
forge.lthn.ai/core/api.
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
|
||||
# core/agent
|
||||
cd ~/Code/core/agent
|
||||
git add -A
|
||||
git commit -m "$(cat <<'EOF'
|
||||
refactor(api): update import path from go-api to core/api
|
||||
|
||||
Part of the polyglot merge — forge.lthn.ai/core/go-api is now
|
||||
forge.lthn.ai/core/api.
|
||||
|
||||
Co-Authored-By: Virgil <virgil@lethean.io>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| # | Task | Files touched | Risk |
|
||||
|---|------|--------------|------|
|
||||
| 1 | Clone core/api, copy Go files | 50 .go + cmd/ + docs/ + .core/ | Low |
|
||||
| 2 | Copy PHP files into src/php/ | ~80 PHP files | Low |
|
||||
| 3 | Create go.mod, composer.json, .gitattributes | 4 new files | Low |
|
||||
| 4 | Create CLAUDE.md | 1 new file | Low |
|
||||
| 5 | Update 5 Go consumers + 2 PHP consumers | ~12 files across 7 repos | Medium |
|
||||
| 6 | Update go.work | 1 file | Low |
|
||||
| 7 | Build verification | 0 files (validation only) | N/A |
|
||||
| 8 | Archive go-api + php-api | 0 local files (Forge API) | Low |
|
||||
|
||||
Estimated effort: 30–45 minutes of execution time.
|
||||
|
|
@ -1,452 +0,0 @@
|
|||
# core/ide Modernisation — Implementation Plan
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Spec:** [2026-03-14-ide-modernisation-design.md](../specs/2026-03-14-ide-modernisation-design.md)
|
||||
|
||||
---
|
||||
|
||||
## Task 1 — Delete the 7 obsolete files
|
||||
|
||||
Remove all hand-rolled services that are now provided by ecosystem packages.
|
||||
|
||||
```bash
|
||||
cd /Users/snider/Code/core/ide
|
||||
|
||||
rm mcp_bridge.go # Replaced by core/mcp Service + transports
|
||||
rm webview_svc.go # Replaced by core/gui/pkg/display + webview
|
||||
rm brain_mcp.go # Replaced by core/mcp/pkg/mcp/brain
|
||||
rm claude_bridge.go # Replaced by core/mcp/pkg/mcp/ide Bridge
|
||||
rm headless_mcp.go # Not needed — core/mcp runs in-process
|
||||
rm headless.go # config gui.enabled flag; jobrunner belongs in core/agent
|
||||
rm greetservice.go # Scaffold placeholder
|
||||
```
|
||||
|
||||
**Verification:** `ls *.go` should show only `main.go`.
|
||||
|
||||
---
|
||||
|
||||
## Task 2 — Rewrite main.go
|
||||
|
||||
Replace the current main.go with a thin Wails shell that wires `core.Core` with
|
||||
ecosystem services. The full file contents:
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"embed"
|
||||
"io/fs"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"runtime"
|
||||
"syscall"
|
||||
|
||||
"forge.lthn.ai/core/config"
|
||||
"forge.lthn.ai/core/go-ws"
|
||||
"forge.lthn.ai/core/go/pkg/core"
|
||||
guiMCP "forge.lthn.ai/core/gui/pkg/mcp"
|
||||
"forge.lthn.ai/core/gui/pkg/display"
|
||||
"forge.lthn.ai/core/ide/icons"
|
||||
"forge.lthn.ai/core/mcp/pkg/mcp"
|
||||
"forge.lthn.ai/core/mcp/pkg/mcp/brain"
|
||||
"forge.lthn.ai/core/mcp/pkg/mcp/ide"
|
||||
"github.com/wailsapp/wails/v3/pkg/application"
|
||||
)
|
||||
|
||||
//go:embed all:frontend/dist/wails-angular-template/browser
|
||||
var assets embed.FS
|
||||
|
||||
func main() {
|
||||
// ── Flags ──────────────────────────────────────────────────
|
||||
mcpOnly := false
|
||||
for _, arg := range os.Args[1:] {
|
||||
if arg == "--mcp" {
|
||||
mcpOnly = true
|
||||
}
|
||||
}
|
||||
|
||||
// ── Configuration ──────────────────────────────────────────
|
||||
cfg, _ := config.New()
|
||||
|
||||
cwd, err := os.Getwd()
|
||||
if err != nil {
|
||||
log.Fatalf("failed to get working directory: %v", err)
|
||||
}
|
||||
|
||||
// ── Shared resources (built before Core) ───────────────────
|
||||
hub := ws.NewHub()
|
||||
|
||||
bridgeCfg := ide.DefaultConfig()
|
||||
bridgeCfg.WorkspaceRoot = cwd
|
||||
if url := os.Getenv("CORE_API_URL"); url != "" {
|
||||
bridgeCfg.LaravelWSURL = url
|
||||
}
|
||||
if token := os.Getenv("CORE_API_TOKEN"); token != "" {
|
||||
bridgeCfg.Token = token
|
||||
}
|
||||
bridge := ide.NewBridge(hub, bridgeCfg)
|
||||
|
||||
// ── Core framework ─────────────────────────────────────────
|
||||
c, err := core.New(
|
||||
core.WithName("ws", func(c *core.Core) (any, error) {
|
||||
return hub, nil
|
||||
}),
|
||||
core.WithService(display.Register(nil)), // nil platform until Wails starts
|
||||
core.WithName("mcp", func(c *core.Core) (any, error) {
|
||||
return mcp.New(
|
||||
mcp.WithWorkspaceRoot(cwd),
|
||||
mcp.WithWSHub(hub),
|
||||
mcp.WithSubsystem(brain.New(bridge)),
|
||||
mcp.WithSubsystem(guiMCP.New(c)),
|
||||
)
|
||||
}),
|
||||
)
|
||||
if err != nil {
|
||||
log.Fatalf("failed to create core: %v", err)
|
||||
}
|
||||
|
||||
// Retrieve the MCP service for transport control
|
||||
mcpSvc, err := core.ServiceFor[*mcp.Service](c, "mcp")
|
||||
if err != nil {
|
||||
log.Fatalf("failed to get MCP service: %v", err)
|
||||
}
|
||||
|
||||
// ── Mode selection ─────────────────────────────────────────
|
||||
if mcpOnly {
|
||||
// stdio mode — Claude Code connects via --mcp flag
|
||||
ctx, cancel := signal.NotifyContext(context.Background(),
|
||||
syscall.SIGINT, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
// Start Core lifecycle manually
|
||||
if err := c.ServiceStartup(ctx, nil); err != nil {
|
||||
log.Fatalf("core startup failed: %v", err)
|
||||
}
|
||||
bridge.Start(ctx)
|
||||
|
||||
if err := mcpSvc.ServeStdio(ctx); err != nil {
|
||||
log.Printf("MCP stdio error: %v", err)
|
||||
}
|
||||
|
||||
_ = mcpSvc.Shutdown(ctx)
|
||||
_ = c.ServiceShutdown(ctx)
|
||||
return
|
||||
}
|
||||
|
||||
if !guiEnabled(cfg) {
|
||||
// No GUI — run Core with MCP transport in background
|
||||
ctx, cancel := signal.NotifyContext(context.Background(),
|
||||
syscall.SIGINT, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
if err := c.ServiceStartup(ctx, nil); err != nil {
|
||||
log.Fatalf("core startup failed: %v", err)
|
||||
}
|
||||
bridge.Start(ctx)
|
||||
|
||||
go func() {
|
||||
if err := mcpSvc.Run(ctx); err != nil {
|
||||
log.Printf("MCP error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-ctx.Done()
|
||||
shutdownCtx := context.Background()
|
||||
_ = mcpSvc.Shutdown(shutdownCtx)
|
||||
_ = c.ServiceShutdown(shutdownCtx)
|
||||
return
|
||||
}
|
||||
|
||||
// ── GUI mode ───────────────────────────────────────────────
|
||||
staticAssets, err := fs.Sub(assets, "frontend/dist/wails-angular-template/browser")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
app := application.New(application.Options{
|
||||
Name: "Core IDE",
|
||||
Description: "Host UK Core IDE - Development Environment",
|
||||
Services: []application.Service{
|
||||
application.NewService(c),
|
||||
},
|
||||
Assets: application.AssetOptions{
|
||||
Handler: application.AssetFileServerFS(staticAssets),
|
||||
},
|
||||
Mac: application.MacOptions{
|
||||
ActivationPolicy: application.ActivationPolicyAccessory,
|
||||
},
|
||||
OnShutdown: func() {
|
||||
ctx := context.Background()
|
||||
_ = mcpSvc.Shutdown(ctx)
|
||||
bridge.Shutdown()
|
||||
},
|
||||
})
|
||||
|
||||
// System tray
|
||||
systray := app.SystemTray.New()
|
||||
systray.SetTooltip("Core IDE")
|
||||
|
||||
if runtime.GOOS == "darwin" {
|
||||
systray.SetTemplateIcon(icons.AppTray)
|
||||
} else {
|
||||
systray.SetDarkModeIcon(icons.AppTray)
|
||||
systray.SetIcon(icons.AppTray)
|
||||
}
|
||||
|
||||
// Tray panel window
|
||||
trayWindow := app.Window.NewWithOptions(application.WebviewWindowOptions{
|
||||
Name: "tray-panel",
|
||||
Title: "Core IDE",
|
||||
Width: 380,
|
||||
Height: 480,
|
||||
URL: "/tray",
|
||||
Hidden: true,
|
||||
Frameless: true,
|
||||
BackgroundColour: application.NewRGB(26, 27, 38),
|
||||
})
|
||||
systray.AttachWindow(trayWindow).WindowOffset(5)
|
||||
|
||||
// Tray menu
|
||||
trayMenu := app.Menu.New()
|
||||
trayMenu.Add("Quit").OnClick(func(ctx *application.Context) {
|
||||
app.Quit()
|
||||
})
|
||||
systray.SetMenu(trayMenu)
|
||||
|
||||
// Start MCP transport alongside Wails
|
||||
go func() {
|
||||
ctx := context.Background()
|
||||
bridge.Start(ctx)
|
||||
if err := mcpSvc.Run(ctx); err != nil {
|
||||
log.Printf("MCP error: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
log.Println("Starting Core IDE...")
|
||||
|
||||
if err := app.Run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// guiEnabled checks whether the GUI should start.
|
||||
// Returns false if config says gui.enabled: false, or if no display is available.
|
||||
func guiEnabled(cfg *config.Config) bool {
|
||||
if cfg != nil {
|
||||
var guiCfg struct {
|
||||
Enabled *bool `mapstructure:"enabled"`
|
||||
}
|
||||
if err := cfg.Get("gui", &guiCfg); err == nil && guiCfg.Enabled != nil {
|
||||
return *guiCfg.Enabled
|
||||
}
|
||||
}
|
||||
// Fall back to display detection
|
||||
if runtime.GOOS == "windows" || runtime.GOOS == "darwin" {
|
||||
return true
|
||||
}
|
||||
return os.Getenv("DISPLAY") != "" || os.Getenv("WAYLAND_DISPLAY") != ""
|
||||
}
|
||||
```
|
||||
|
||||
### Key changes from the old main.go
|
||||
|
||||
1. **Three modes**: `--mcp` (stdio for Claude Code), no-GUI (headless with MCP_ADDR), GUI (Wails systray).
|
||||
2. **`core.Core` is the Wails service** — its `ServiceStartup`/`ServiceShutdown` drive all sub-service lifecycles.
|
||||
3. **Ecosystem wiring**: `display.Register(nil)` for GUI tools, `brain.New(bridge)` for OpenBrain, `guiMCP.New(c)` for 74 GUI MCP tools, `mcp.WithWSHub(hub)` for WS streaming.
|
||||
4. **No hand-rolled HTTP server** — `mcp.Service` owns the MCP protocol (stdio/TCP/Unix). The WS hub is injected via option.
|
||||
5. **IDE bridge** uses `ide.DefaultConfig()` from `core/mcp/pkg/mcp/ide` with env var overrides.
|
||||
|
||||
---
|
||||
|
||||
## Task 3 — Update go.mod
|
||||
|
||||
### 3a. Replace go.mod with updated dependencies
|
||||
|
||||
```
|
||||
module forge.lthn.ai/core/ide
|
||||
|
||||
go 1.26.0
|
||||
|
||||
require (
|
||||
forge.lthn.ai/core/go v0.2.2
|
||||
forge.lthn.ai/core/config v0.1.2
|
||||
forge.lthn.ai/core/go-ws v0.1.3
|
||||
forge.lthn.ai/core/gui v0.1.0
|
||||
forge.lthn.ai/core/mcp v0.1.0
|
||||
github.com/wailsapp/wails/v3 v3.0.0-alpha.74
|
||||
)
|
||||
```
|
||||
|
||||
**Removed** (no longer directly imported):
|
||||
- `forge.lthn.ai/core/agent` — brain tools now via `core/mcp/pkg/mcp/brain`
|
||||
- `forge.lthn.ai/core/go-process` — daemon/PID logic was in headless.go
|
||||
- `forge.lthn.ai/core/go-scm` — Forgejo client was in headless.go
|
||||
- `github.com/gorilla/websocket` — replaced by `core/mcp/pkg/mcp/ide` (uses it internally)
|
||||
|
||||
**Added**:
|
||||
- `forge.lthn.ai/core/gui` — display service + 74 MCP tools
|
||||
- `forge.lthn.ai/core/mcp` — MCP server, brain subsystem, IDE bridge
|
||||
|
||||
### 3b. Sync workspace and tidy
|
||||
|
||||
```bash
|
||||
cd /Users/snider/Code/core/ide
|
||||
go mod tidy
|
||||
cd /Users/snider/Code
|
||||
go work sync
|
||||
```
|
||||
|
||||
The `go mod tidy` will pull indirect dependencies and populate the `require`
|
||||
block. The `go work sync` aligns versions across the workspace.
|
||||
|
||||
---
|
||||
|
||||
## Task 4 — Update CLAUDE.md
|
||||
|
||||
Replace the existing CLAUDE.md with content reflecting the new architecture:
|
||||
|
||||
```markdown
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Build & Development Commands
|
||||
|
||||
\`\`\`bash
|
||||
# Development (hot-reload GUI + Go rebuild)
|
||||
wails3 dev
|
||||
|
||||
# Production build (preferred)
|
||||
core build
|
||||
|
||||
# Frontend-only development
|
||||
cd frontend && npm install && npm run dev
|
||||
|
||||
# Go tests
|
||||
core go test # All tests
|
||||
core go test --run TestName # Single test
|
||||
core go cov # Coverage report
|
||||
core go cov --open # Coverage in browser
|
||||
|
||||
# Quality assurance
|
||||
core go qa # Format + vet + lint + test
|
||||
core go qa full # + race detector, vuln scan, security audit
|
||||
core go fmt # Format only
|
||||
core go lint # Lint only
|
||||
|
||||
# Frontend tests
|
||||
cd frontend && npm run test
|
||||
\`\`\`
|
||||
|
||||
## Architecture
|
||||
|
||||
**Thin Wails shell** wiring ecosystem packages via `core.Core` dependency injection. Three operating modes:
|
||||
|
||||
### GUI Mode (default)
|
||||
`main()` → Wails 3 application with embedded Angular frontend, system tray (macOS: accessory app, no Dock icon). Core framework manages all services:
|
||||
- **display** (`core/gui`) — window management, webview automation, 74 MCP tools across 14 categories
|
||||
- **MCP** (`core/mcp`) — Model Context Protocol server with file ops, brain subsystem, GUI subsystem
|
||||
- **IDE bridge** (`core/mcp/pkg/mcp/ide`) — WebSocket bridge to Laravel core-agentic backend
|
||||
- **WS hub** (`core/go-ws`) — WebSocket hub for Angular frontend communication
|
||||
|
||||
### MCP Mode (`--mcp`)
|
||||
`core-ide --mcp` → stdio MCP server for Claude Code integration. No GUI, no HTTP. Configure in `.claude/.mcp.json`:
|
||||
\`\`\`json
|
||||
{
|
||||
"mcpServers": {
|
||||
"core-ide": {
|
||||
"type": "stdio",
|
||||
"command": "core-ide",
|
||||
"args": ["--mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
### Headless Mode (no display or `gui.enabled: false`)
|
||||
Core framework runs all services without Wails. MCP transport determined by `MCP_ADDR` env var (TCP if set, stdio otherwise).
|
||||
|
||||
### Frontend
|
||||
Angular 20+ app embedded via `//go:embed`. Two routes: `/tray` (system tray panel, 380x480 frameless) and `/ide` (full IDE layout).
|
||||
|
||||
## Configuration
|
||||
|
||||
\`\`\`yaml
|
||||
# .core/config.yaml
|
||||
gui:
|
||||
enabled: true # false = no Wails, Core still runs
|
||||
mcp:
|
||||
transport: stdio # stdio | tcp | unix
|
||||
tcp:
|
||||
port: 9877
|
||||
brain:
|
||||
api_url: http://localhost:8000
|
||||
api_token: "" # or CORE_API_TOKEN env var
|
||||
\`\`\`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `CORE_API_URL` | `http://localhost:8000` | Laravel backend WebSocket URL |
|
||||
| `CORE_API_TOKEN` | (empty) | Bearer token for Laravel backend auth |
|
||||
| `MCP_ADDR` | (empty) | TCP address for MCP server (headless mode) |
|
||||
|
||||
## Workspace Dependencies
|
||||
|
||||
This module uses a Go workspace (`~/Code/go.work`) with `replace` directives for sibling modules:
|
||||
- `../go` → `forge.lthn.ai/core/go`
|
||||
- `../gui` → `forge.lthn.ai/core/gui`
|
||||
- `../mcp` → `forge.lthn.ai/core/mcp`
|
||||
- `../config` → `forge.lthn.ai/core/config`
|
||||
- `../go-ws` → `forge.lthn.ai/core/go-ws`
|
||||
|
||||
## Conventions
|
||||
|
||||
- **UK English** in documentation and user-facing strings (colour, organisation, centre).
|
||||
- **Conventional commits**: `type(scope): description` with co-author line `Co-Authored-By: Virgil <virgil@lethean.io>`.
|
||||
- **Licence**: EUPL-1.2.
|
||||
- All Go code is in `package main` (single-package application).
|
||||
- Services are registered via `core.WithService` or `core.WithName` factory functions.
|
||||
- MCP subsystems implement `mcp.Subsystem` interface from `core/mcp`.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5 — Build verification
|
||||
|
||||
```bash
|
||||
cd /Users/snider/Code/core/ide
|
||||
go build ./...
|
||||
```
|
||||
|
||||
If the build fails, the likely causes are:
|
||||
|
||||
1. **Missing workspace entries** — ensure `go.work` includes `./core/mcp` and `./core/gui`:
|
||||
```bash
|
||||
cd /Users/snider/Code
|
||||
go work use ./core/mcp ./core/gui
|
||||
go work sync
|
||||
```
|
||||
|
||||
2. **Stale go.sum** — clear and regenerate:
|
||||
```bash
|
||||
cd /Users/snider/Code/core/ide
|
||||
rm go.sum
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
3. **Version mismatches** — check that indirect dependency versions align:
|
||||
```bash
|
||||
go mod graph | grep -i conflict
|
||||
```
|
||||
|
||||
Once `go build ./...` succeeds, run the quality check:
|
||||
|
||||
```bash
|
||||
core go qa
|
||||
```
|
||||
|
||||
This confirms formatting, vetting, linting, and tests all pass.
|
||||
|
|
@ -1,105 +0,0 @@
|
|||
# Runtime Provider Loading — Implementation Plan
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Spec:** ../specs/2026-03-14-runtime-provider-loading-design.md
|
||||
**Status:** Complete
|
||||
|
||||
## Task 1: ProxyProvider in core/api (`pkg/provider/proxy.go`)
|
||||
|
||||
**File:** `/Users/snider/Code/core/api/pkg/provider/proxy.go`
|
||||
|
||||
Replace the Phase 3 stub with a working ProxyProvider that:
|
||||
|
||||
- Takes a `ProxyConfig` struct: Name, BasePath, Upstream URL, ElementSpec, SpecFile path
|
||||
- Implements `Provider` (Name, BasePath, RegisterRoutes)
|
||||
- Implements `Renderable` (Element)
|
||||
- `RegisterRoutes` creates a catch-all `/*path` handler using `net/http/httputil.ReverseProxy`
|
||||
- Strips the base path before proxying so the upstream sees clean paths
|
||||
- Upstream is always `127.0.0.1` (local process)
|
||||
|
||||
**Test file:** `pkg/provider/proxy_test.go`
|
||||
- Proxy routes requests to a test upstream server
|
||||
- Health check passthrough
|
||||
- Element() returns configured ElementSpec
|
||||
- Name/BasePath return configured values
|
||||
|
||||
## Task 2: Manifest Extensions in go-scm (`manifest/manifest.go`)
|
||||
|
||||
**File:** `/Users/snider/Code/core/go-scm/manifest/manifest.go`
|
||||
|
||||
Extend the Manifest struct with provider-specific fields:
|
||||
|
||||
- `Namespace` — API route prefix (e.g. `/api/v1/cool-widget`)
|
||||
- `Port` — listen port (0 = auto-assign)
|
||||
- `Binary` — path to binary relative to provider dir
|
||||
- `Args` — additional CLI args
|
||||
- `Element` — UI element spec (tag + source)
|
||||
- `Spec` — path to OpenAPI spec file
|
||||
|
||||
These fields are optional — existing manifests without them remain valid.
|
||||
|
||||
**Test:** Add parse test with provider fields.
|
||||
|
||||
## Task 3: Provider Discovery in go-scm (`marketplace/discovery.go`)
|
||||
|
||||
**File:** `/Users/snider/Code/core/go-scm/marketplace/discovery.go`
|
||||
|
||||
- `DiscoverProviders(dir string)` — scans `dir/*/manifest.yaml` (using `os` directly, not Medium, since this is filesystem discovery)
|
||||
- Returns `[]DiscoveredProvider` with Dir + Manifest
|
||||
- Skips directories without manifests (logs warning, continues)
|
||||
|
||||
**File:** `/Users/snider/Code/core/go-scm/marketplace/registry_file.go`
|
||||
|
||||
- `ProviderRegistry` — read/write `registry.yaml` tracking installed providers
|
||||
- `LoadRegistry(path)`, `SaveRegistry(path)`, `Add()`, `Remove()`, `Get()`, `List()`
|
||||
|
||||
**Test file:** `marketplace/discovery_test.go`
|
||||
- Discover finds providers in temp dirs with manifests
|
||||
- Discover skips dirs without manifests
|
||||
- Registry CRUD operations
|
||||
|
||||
## Task 4: RuntimeManager in core/ide (`runtime.go`)
|
||||
|
||||
**File:** `/Users/snider/Code/core/ide/runtime.go`
|
||||
|
||||
- `RuntimeManager` struct holding discovered providers, engine reference, process registry
|
||||
- `NewRuntimeManager(engine, processRegistry)` constructor
|
||||
- `StartAll(ctx)` — discover providers in `~/.core/providers/`, start each binary via `os/exec.Cmd`, wait for health, register ProxyProvider
|
||||
- `StopAll()` — SIGTERM each provider process, clean up
|
||||
- `List()` — return running providers
|
||||
- Free port allocation via `net.Listen(":0")`
|
||||
- Health check polling with timeout
|
||||
|
||||
## Task 5: Wire into core/ide main.go
|
||||
|
||||
**File:** `/Users/snider/Code/core/ide/main.go`
|
||||
|
||||
- After creating the api.Engine (`engine, _ := api.New(...)`), create a RuntimeManager
|
||||
- In each mode (mcpOnly, headless, GUI), call `rm.StartAll(ctx)` before serving
|
||||
- In shutdown paths, call `rm.StopAll()`
|
||||
- Import `forge.lthn.ai/core/go-scm/marketplace` (add dependency)
|
||||
|
||||
## Task 6: Build Verification
|
||||
|
||||
- `cd core/api && go build ./...`
|
||||
- `cd go-scm && go build ./...`
|
||||
- `cd ide && go build ./...` (may need go.mod updates)
|
||||
|
||||
## Task 7: Tests
|
||||
|
||||
- `cd core/api && go test ./pkg/provider/...`
|
||||
- `cd go-scm && go test ./marketplace/... ./manifest/...`
|
||||
|
||||
## Repos Affected
|
||||
|
||||
| Repo | Changes |
|
||||
|------|---------|
|
||||
| core/api | `pkg/provider/proxy.go` (implement), `pkg/provider/proxy_test.go` (new) |
|
||||
| go-scm | `manifest/manifest.go` (extend), `manifest/manifest_test.go` (extend), `marketplace/discovery.go` (new), `marketplace/registry_file.go` (new), `marketplace/discovery_test.go` (new) |
|
||||
| core/ide | `runtime.go` (new), `main.go` (wire), `go.mod` (add go-scm dep) |
|
||||
|
||||
## Commit Strategy
|
||||
|
||||
1. core/api — `feat(provider): implement ProxyProvider reverse proxy`
|
||||
2. go-scm — `feat(marketplace): add provider discovery and registry`
|
||||
3. core/ide — `feat(runtime): add RuntimeManager for provider loading`
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -1,154 +0,0 @@
|
|||
# core/api Polyglot Merge — Go + PHP API in One Repo
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
**Pattern:** Same as core/mcp merge (2026-03-09)
|
||||
|
||||
## Problem
|
||||
|
||||
The API layer is split across two repos:
|
||||
- `core/go-api` — Gin REST framework, OpenAPI, ToolBridge, SDK codegen, WS/SSE
|
||||
- `core/php-api` — Laravel REST module, rate limiting, webhooks, OAuth, API docs
|
||||
|
||||
These serve the same purpose in their respective stacks. The provider framework
|
||||
(previous spec) needs a single `core/api` that both Go and PHP packages register
|
||||
into. Same problem core/mcp solved by merging.
|
||||
|
||||
## Solution
|
||||
|
||||
Merge both into `core/api` (`ssh://git@forge.lthn.ai:2223/core/api.git`),
|
||||
following the exact pattern established by core/mcp.
|
||||
|
||||
## Target Structure
|
||||
|
||||
```
|
||||
core/api/
|
||||
├── go.mod # module forge.lthn.ai/core/api
|
||||
├── composer.json # name: lthn/api
|
||||
├── .gitattributes # export-ignore Go files for Composer
|
||||
├── CLAUDE.md
|
||||
├── pkg/ # Go packages
|
||||
│ ├── api/ # Engine, RouteGroup, middleware (from go-api root)
|
||||
│ ├── provider/ # Provider framework (new, from previous spec)
|
||||
│ └── openapi/ # SpecBuilder, codegen (from go-api root)
|
||||
├── cmd/
|
||||
│ └── core-api/ # Standalone binary (if needed)
|
||||
├── src/
|
||||
│ └── php/
|
||||
│ └── src/
|
||||
│ ├── Api/ # Boot, Controllers, Middleware, etc. (from php-api/src/Api/)
|
||||
│ ├── Front/Api/ # Frontend API routes (from php-api/src/Front/)
|
||||
│ └── Website/Api/ # Website API routes (from php-api/src/Website/)
|
||||
└── docs/
|
||||
```
|
||||
|
||||
## Go Module
|
||||
|
||||
```go
|
||||
module forge.lthn.ai/core/api
|
||||
```
|
||||
|
||||
All current consumers importing `forge.lthn.ai/core/go-api` will need to update
|
||||
to `forge.lthn.ai/core/api/pkg/api` (or `forge.lthn.ai/core/api` if we keep
|
||||
packages at root level — see decision below).
|
||||
|
||||
### Package Layout Decision
|
||||
|
||||
**Option A**: Packages under `pkg/api/` — clean but changes all import paths.
|
||||
**Option B**: Packages at root (like go-api currently) — no import path change
|
||||
if module path stays `forge.lthn.ai/core/api`.
|
||||
|
||||
**Recommendation: B** — keep Go files at repo root. The module path changes from
|
||||
`forge.lthn.ai/core/go-api` to `forge.lthn.ai/core/api`. This is a one-line
|
||||
change in each consumer's import. The PHP code lives under `src/php/` and
|
||||
doesn't conflict.
|
||||
|
||||
```
|
||||
core/api/
|
||||
├── api.go, group.go, openapi.go, ... # Go source at root (same as go-api today)
|
||||
├── pkg/provider/ # New provider framework
|
||||
├── cmd/core-api/ # Binary
|
||||
├── src/php/ # PHP source
|
||||
├── composer.json
|
||||
├── go.mod
|
||||
└── .gitattributes
|
||||
```
|
||||
|
||||
## Composer Package
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "lthn/api",
|
||||
"autoload": {
|
||||
"psr-4": {
|
||||
"Core\\Api\\": "src/php/src/Api/",
|
||||
"Core\\Website\\Api\\": "src/php/src/Website/Api/"
|
||||
}
|
||||
},
|
||||
"replace": {
|
||||
"lthn/php-api": "self.version",
|
||||
"core/php-api": "self.version"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: No `Core\Front\Api\` namespace — php-api has no `src/Front/` directory.
|
||||
Add it later if a frontend API boot provider is needed.
|
||||
|
||||
## .gitattributes
|
||||
|
||||
Same pattern as core/mcp — exclude Go files from Composer installs:
|
||||
|
||||
```
|
||||
*.go export-ignore
|
||||
go.mod export-ignore
|
||||
go.sum export-ignore
|
||||
cmd/ export-ignore
|
||||
pkg/ export-ignore
|
||||
.core/ export-ignore
|
||||
src/php/tests/ export-ignore
|
||||
```
|
||||
|
||||
## Migration Steps
|
||||
|
||||
1. **Clone core/api** (empty repo, just created)
|
||||
2. **Copy Go code** from go-api → core/api root (all .go files, cmd/, docs/)
|
||||
3. **Copy PHP code** from php-api/src/ → core/api/src/php/src/
|
||||
4. **Copy PHP config** — composer.json, phpunit.xml, tests
|
||||
5. **Update go.mod** — change module path to `forge.lthn.ai/core/api`
|
||||
6. **Create .gitattributes** — export-ignore Go files
|
||||
7. **Create composer.json** — lthn/api with PSR-4 autoload
|
||||
8. **Update consumers** — sed import paths in all repos that import go-api
|
||||
9. **Add provider framework** — `pkg/provider/` from the previous spec
|
||||
10. **Archive go-api and php-api** on forge (keep for backward compat period)
|
||||
|
||||
## Consumer Updates
|
||||
|
||||
Repos that import `forge.lthn.ai/core/go-api`:
|
||||
- core/go-ai (ToolBridge, API routes)
|
||||
- core/go-ml (API route group)
|
||||
- core/mcp (bridge.go)
|
||||
- core/agent (indirect, v0.0.3)
|
||||
- core/ide (via provider framework)
|
||||
|
||||
Each needs: `s/forge.lthn.ai\/core\/go-api/forge.lthn.ai\/core\/api/g` in
|
||||
imports and go.mod.
|
||||
|
||||
PHP consumers via Composer — update `lthn/php-api` → `lthn/api` in
|
||||
composer.json. Namespace stays `Core\Api\` (unchanged).
|
||||
|
||||
## Go Workspace
|
||||
|
||||
Add `core/api` to `~/Code/go.work`, remove `core/go-api` entry.
|
||||
|
||||
## Testing
|
||||
|
||||
- `go test ./...` in core/api
|
||||
- `composer test` in core/api (runs PHP tests via phpunit)
|
||||
- Build verification across consumer repos after import path update
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Provider framework implementation (separate spec, separate plan)
|
||||
- New API features — this is a structural merge only
|
||||
- TypeScript API layer (future Phase 3)
|
||||
|
|
@ -1,226 +0,0 @@
|
|||
# GUI App Shell — Framework-Level Application Frame
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
**Depends on:** Service Provider Framework, Runtime Provider Loading
|
||||
**Source:** Port from core-gui/cmd/lthn-desktop/frontend
|
||||
|
||||
## Problem
|
||||
|
||||
core/ide has a bare Angular frontend with placeholder routes. The real app
|
||||
shell exists in the archived `core-gui/cmd/lthn-desktop/frontend` with
|
||||
HLCRF layout, Web Awesome components, sidebar navigation, feature flags,
|
||||
i18n, and custom element support. It needs to live in `core/gui/ui/` as a
|
||||
framework component that any Wails app can import.
|
||||
|
||||
## Solution
|
||||
|
||||
Port the application frame from lthn-desktop into `core/gui/ui/` as a
|
||||
reusable Angular library. Add a `ProviderDiscoveryService` that dynamically
|
||||
populates navigation and loads custom elements from registered providers.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
core/gui/ui/ <- framework (npm package)
|
||||
src/
|
||||
frame/
|
||||
application-frame.ts <- HLCRF shell (header, sidebar, content, footer)
|
||||
application-frame.html <- wa-page template with slots
|
||||
system-tray-frame.ts <- tray panel (380x480 frameless)
|
||||
services/
|
||||
provider-discovery.ts <- fetch providers, load custom elements
|
||||
websocket.ts <- WS connection with reconnect
|
||||
api-config.ts <- API base URL configuration
|
||||
i18n.ts <- translation service
|
||||
components/
|
||||
provider-host.ts <- wrapper that hosts a custom element
|
||||
provider-nav.ts <- dynamic sidebar from providers
|
||||
status-bar.ts <- footer with provider status
|
||||
index.ts <- public API exports
|
||||
package.json
|
||||
tsconfig.json
|
||||
|
||||
core/ide/frontend/ <- application (imports framework)
|
||||
src/
|
||||
app/
|
||||
app.routes.ts <- routes using framework components
|
||||
app.config.ts <- provider registrations
|
||||
main.ts
|
||||
angular.json
|
||||
```
|
||||
|
||||
## Provider Discovery Service
|
||||
|
||||
The core service that makes everything dynamic:
|
||||
|
||||
```typescript
|
||||
@Injectable({ providedIn: 'root' })
|
||||
export class ProviderDiscoveryService {
|
||||
private providers = signal<ProviderInfo[]>([]);
|
||||
readonly providers$ = this.providers.asReadonly();
|
||||
|
||||
constructor(private apiConfig: ApiConfigService) {}
|
||||
|
||||
async discover(): Promise<void> {
|
||||
const res = await fetch(`${this.apiConfig.baseUrl}/api/v1/providers`);
|
||||
const data = await res.json();
|
||||
this.providers.set(data.providers);
|
||||
|
||||
// Load custom elements for Renderable providers
|
||||
for (const p of data.providers) {
|
||||
if (p.element?.tag && p.element?.source) {
|
||||
await this.loadElement(p.element.tag, p.element.source);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async loadElement(tag: string, source: string): Promise<void> {
|
||||
if (customElements.get(tag)) return;
|
||||
const script = document.createElement('script');
|
||||
script.type = 'module';
|
||||
script.src = source;
|
||||
document.head.appendChild(script);
|
||||
await customElements.whenDefined(tag);
|
||||
}
|
||||
}
|
||||
|
||||
interface ProviderInfo {
|
||||
name: string;
|
||||
basePath: string;
|
||||
status?: string;
|
||||
element?: { tag: string; source: string };
|
||||
channels?: string[];
|
||||
}
|
||||
```
|
||||
|
||||
## Application Frame
|
||||
|
||||
Ported from lthn-desktop with these changes:
|
||||
|
||||
1. **Navigation is dynamic** — populated from ProviderDiscoveryService
|
||||
2. **Content area hosts custom elements** — ProviderHostComponent wraps any custom element
|
||||
3. **Feature flags from config** — reads from core/config API
|
||||
4. **Web Awesome** — keeps wa-page, wa-button design system
|
||||
5. **Font Awesome Pro** — keeps icon system
|
||||
6. **i18n** — keeps translation service
|
||||
7. **HLCRF slots** — header, navigation, main, footer map to wa-page slots
|
||||
|
||||
### Dynamic Navigation
|
||||
|
||||
```typescript
|
||||
// In application-frame.ts
|
||||
async ngOnInit() {
|
||||
await this.providerService.discover();
|
||||
|
||||
this.navigation = this.providerService.providers$()
|
||||
.filter(p => p.element)
|
||||
.map(p => ({
|
||||
name: p.name,
|
||||
href: p.name.toLowerCase(),
|
||||
icon: 'fa-regular fa-puzzle-piece',
|
||||
element: p.element
|
||||
}));
|
||||
}
|
||||
```
|
||||
|
||||
### Provider Host Component
|
||||
|
||||
Renders any custom element by tag name using Angular's Renderer2 for safe DOM manipulation:
|
||||
|
||||
```typescript
|
||||
@Component({
|
||||
selector: 'provider-host',
|
||||
template: '<div #container></div>',
|
||||
schemas: [CUSTOM_ELEMENTS_SCHEMA]
|
||||
})
|
||||
export class ProviderHostComponent implements OnChanges {
|
||||
@Input() tag!: string;
|
||||
@Input() apiUrl = '';
|
||||
@ViewChild('container') container!: ElementRef;
|
||||
|
||||
constructor(private renderer: Renderer2) {}
|
||||
|
||||
ngOnChanges() {
|
||||
const native = this.container.nativeElement;
|
||||
// Clear previous element safely
|
||||
while (native.firstChild) {
|
||||
this.renderer.removeChild(native, native.firstChild);
|
||||
}
|
||||
// Create and append custom element
|
||||
const el = this.renderer.createElement(this.tag);
|
||||
if (this.apiUrl) this.renderer.setAttribute(el, 'api-url', this.apiUrl);
|
||||
this.renderer.appendChild(native, el);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Routes
|
||||
|
||||
```typescript
|
||||
export const routes: Routes = [
|
||||
{ path: 'tray', component: SystemTrayFrame },
|
||||
{
|
||||
path: '',
|
||||
component: ApplicationFrame,
|
||||
children: [
|
||||
{ path: ':provider', component: ProviderHostComponent },
|
||||
{ path: '', redirectTo: 'process', pathMatch: 'full' }
|
||||
]
|
||||
}
|
||||
];
|
||||
```
|
||||
|
||||
## System Tray Frame
|
||||
|
||||
380x480 frameless panel showing:
|
||||
- Provider status cards (from discovery service)
|
||||
- Quick stats from Streamable providers (via WS)
|
||||
- Brain connection status
|
||||
- MCP server status
|
||||
|
||||
## What to Port from lthn-desktop
|
||||
|
||||
| Source | Target | Changes |
|
||||
|--------|--------|---------|
|
||||
| frame/application.frame.ts | core/gui/ui/src/frame/ | Dynamic nav from providers |
|
||||
| frame/application.frame.html | core/gui/ui/src/frame/ | Keep wa-page template |
|
||||
| frame/system-tray.frame.ts | core/gui/ui/src/frame/ | Add provider status cards |
|
||||
| services/translation.service.ts | core/gui/ui/src/services/ | Keep as-is |
|
||||
| services/i18n.service.ts | core/gui/ui/src/services/ | Keep as-is |
|
||||
|
||||
## What NOT to Port
|
||||
|
||||
- blockchain/ — that is a provider, not framework
|
||||
- mining/ — that is a provider (already has its own elements)
|
||||
- developer/ — that is a provider
|
||||
- system/setup* — future setup wizard provider
|
||||
- Wails bindings (@lthn/core/*) — replaced by REST API calls
|
||||
|
||||
## Dependencies
|
||||
|
||||
### core/gui/ui (npm)
|
||||
- @angular/core, @angular/router, @angular/common
|
||||
- @awesome.me/webawesome (design system)
|
||||
- @fortawesome/fontawesome-pro (icons)
|
||||
|
||||
### core/ide/frontend (npm)
|
||||
- core/gui/ui (local dependency)
|
||||
- Angular 20+
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
# Build framework
|
||||
cd core/gui/ui && npm run build
|
||||
|
||||
# Build IDE app (imports framework)
|
||||
cd core/ide/frontend && npm run build
|
||||
```
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Setup wizard (future provider)
|
||||
- Monaco editor integration (future provider)
|
||||
- Blockchain dashboard (future provider)
|
||||
- Theming system (future — Web Awesome handles dark mode)
|
||||
|
|
@ -1,235 +0,0 @@
|
|||
# core/ide Modernisation — Lego Rewire
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
|
||||
## Problem
|
||||
|
||||
core/ide was built before the ecosystem packages existed. It hand-rolls an MCP bridge, webview service, brain tools, and WebSocket relay that are now properly implemented in dedicated packages. The codebase is ~1,200 lines of duplicated logic.
|
||||
|
||||
## Solution
|
||||
|
||||
Replace all hand-rolled services with imports from the existing ecosystem. The IDE becomes a thin Wails shell that wires up `core.Core` with registered services.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Wails 3 │
|
||||
│ ┌──────────┐ ┌──────────────────────────┐ │
|
||||
│ │ Systray │ │ Angular Frontend │ │
|
||||
│ │ (lifecycle)│ │ /tray (control pane) │ │
|
||||
│ │ │ │ /ide (full IDE) │ │
|
||||
│ └────┬─────┘ └──────────┬───────────────┘ │
|
||||
│ │ │ WS │
|
||||
└───────┼───────────────────┼─────────────────┘
|
||||
│ │
|
||||
┌───────┼───────────────────┼─────────────────┐
|
||||
│ │ core.Core │ │
|
||||
│ ┌────┴─────┐ ┌─────────┴───────┐ │
|
||||
│ │ display │ │ go-ws Hub │ │
|
||||
│ │ Service │ │ (Angular comms) │ │
|
||||
│ │(core/gui)│ └─────────────────┘ │
|
||||
│ └──────────┘ │
|
||||
│ ┌──────────────────────────────────────┐ │
|
||||
│ │ core/mcp Service │ │
|
||||
│ │ ├─ brain subsystem (OpenBrain) │ │
|
||||
│ │ ├─ gui subsystem (74 tools) │ │
|
||||
│ │ ├─ file ops (go-io sandboxed) │ │
|
||||
│ │ └─ transports: stdio, TCP, Unix │ │
|
||||
│ └──────────────────────────────────────┘ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ config │ │go-process│ │ go-log │ │
|
||||
│ └──────────┘ └──────────┘ └──────────┘ │
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
│ stdio (--mcp flag)
|
||||
│
|
||||
┌───────┴─────────────────────────────────────┐
|
||||
│ Claude Code / other MCP clients │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Application Lifecycle
|
||||
|
||||
- **Systray** is the app lifecycle. Quit from tray menu = shutdown Core.
|
||||
- **Tray panel** is a 380x480 frameless Angular window attached to the tray icon. Renders a control pane (MCP status, connected agents, brain stats).
|
||||
- **Windows** are disposable views — closing a window does not terminate the app.
|
||||
- **macOS**: Accessory app (no Dock icon). Template tray icon.
|
||||
- **config** reads `.core/config.yaml`. If `gui.enabled: false` (or no display), Wails is skipped entirely. Core still runs all services — MCP server, brain, etc.
|
||||
|
||||
## Service Wiring
|
||||
|
||||
Services are registered as factory functions via `core.WithService`. Each factory
|
||||
receives the `*core.Core` instance and returns the service. Construction order
|
||||
matters — the WS hub and IDE bridge must be built before the MCP service that
|
||||
depends on them.
|
||||
|
||||
```go
|
||||
func main() {
|
||||
cfg, _ := config.New()
|
||||
|
||||
// Shared resources built before Core
|
||||
hub := ws.NewHub()
|
||||
bridgeCfg := ide.ConfigFromEnv()
|
||||
bridge := ide.NewBridge(hub, bridgeCfg)
|
||||
|
||||
// Core framework — services run regardless of GUI
|
||||
c, _ := core.New(
|
||||
core.WithService(func(c *core.Core) (any, error) {
|
||||
return hub, nil // go-ws hub for Angular + bridge
|
||||
}),
|
||||
core.WithService(display.Register(nil)), // core/gui — nil platform until Wails starts
|
||||
core.WithService(func(c *core.Core) (any, error) {
|
||||
// MCP service with subsystems
|
||||
return mcp.New(
|
||||
mcp.WithWorkspaceRoot(cwd),
|
||||
mcp.WithSubsystem(brain.New(bridge)),
|
||||
mcp.WithSubsystem(guiMCP.New(c)),
|
||||
)
|
||||
}),
|
||||
)
|
||||
|
||||
if guiEnabled(cfg) {
|
||||
startWailsApp(c, hub, bridge)
|
||||
} else {
|
||||
// No GUI — start Core lifecycle manually
|
||||
ctx, cancel := signal.NotifyContext(context.Background(),
|
||||
syscall.SIGINT, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
startMCP(ctx, c) // goroutine: mcpSvc.ServeStdio or ServeTCP
|
||||
bridge.Start(ctx)
|
||||
<-ctx.Done()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### MCP Lifecycle
|
||||
|
||||
`mcp.Service` does not implement `core.Startable`/`core.Stoppable`. The MCP
|
||||
transport must be started explicitly:
|
||||
|
||||
- **GUI mode**: `startWailsApp` starts `mcpSvc.ServeStdio(ctx)` or
|
||||
`mcpSvc.ServeTCP(ctx, addr)` in a goroutine alongside the Wails event loop.
|
||||
Wails quit triggers `mcpSvc.Shutdown(ctx)`.
|
||||
- **No-GUI mode**: `startMCP(ctx, c)` starts the transport in a goroutine.
|
||||
Context cancellation (SIGINT/SIGTERM) triggers shutdown.
|
||||
|
||||
### Version Alignment
|
||||
|
||||
After updating `go.mod` to import `core/mcp`, `core/gui`, etc., run
|
||||
`go work sync` to align indirect dependency versions across the workspace.
|
||||
|
||||
## MCP Server
|
||||
|
||||
The `core/mcp` package provides the standard MCP protocol server with three transports:
|
||||
|
||||
- **stdio** — for Claude Code integration via `.claude/.mcp.json`
|
||||
- **TCP** — for network MCP clients (port from config, default 9877)
|
||||
- **Unix socket** — for local IPC
|
||||
|
||||
Claude Code connects via stdio:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"core-ide": {
|
||||
"type": "stdio",
|
||||
"command": "core-ide",
|
||||
"args": ["--mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Brain Subsystem
|
||||
|
||||
Already implemented in `core/mcp/pkg/mcp/brain`. Proxies to Laravel OpenBrain API via the IDE bridge (`core/mcp/pkg/mcp/ide`). Tools:
|
||||
|
||||
- `brain_remember` — store a memory (content, type, project, agent_id, tags)
|
||||
- `brain_recall` — semantic search (query, top_k, project, type, agent_id)
|
||||
- `brain_forget` — remove a memory by ID
|
||||
- `brain_ensure_collection` — ensure Qdrant collection exists
|
||||
|
||||
Environment: `CORE_API_URL` (default `http://localhost:8000`), `CORE_API_TOKEN`.
|
||||
|
||||
## GUI Subsystem
|
||||
|
||||
The `core/gui/pkg/mcp.Subsystem` provides 74 MCP tools across 14 categories (webview, window, layout, screen, clipboard, dialog, notification, tray, environment, browser, contextmenu, keybinding, dock, lifecycle). These are only active when a display platform is registered.
|
||||
|
||||
The `core/gui/pkg/display.Service` bridges Core IPC to the Wails platform — window management, DOM interaction, event forwarding, WS event bridging for Angular.
|
||||
|
||||
## Files
|
||||
|
||||
### Delete (7 files)
|
||||
|
||||
| File | Replaced by |
|
||||
|------|-------------|
|
||||
| `mcp_bridge.go` | `core/mcp` Service + transports |
|
||||
| `webview_svc.go` | `core/gui/pkg/webview` + `core/gui/pkg/display` |
|
||||
| `brain_mcp.go` | `core/mcp/pkg/mcp/brain` |
|
||||
| `claude_bridge.go` | `core/mcp/pkg/mcp/ide` Bridge |
|
||||
| `headless_mcp.go` | Not needed — core/mcp runs in-process |
|
||||
| `headless.go` | config `gui.enabled` flag. Jobrunner/poller extracted to core/agent (not in scope). |
|
||||
| `greetservice.go` | Scaffold placeholder, not needed |
|
||||
|
||||
### Rewrite (1 file)
|
||||
|
||||
| File | Changes |
|
||||
|------|---------|
|
||||
| `main.go` | Thin shell: config, core.New with services, Wails systray, `--mcp` flag for stdio |
|
||||
|
||||
### Keep (unchanged)
|
||||
|
||||
| File/Dir | Reason |
|
||||
|----------|--------|
|
||||
| `frontend/` | Angular app — tray panel + IDE routes |
|
||||
| `icons/` | Tray icons |
|
||||
| `.core/build.yaml` | Build configuration |
|
||||
| `CLAUDE.md` | Update after implementation |
|
||||
| `go.mod` | Update dependencies |
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Add
|
||||
|
||||
- `forge.lthn.ai/core/go` — DI framework, core.Core
|
||||
- `forge.lthn.ai/core/mcp` — MCP server, brain subsystem, IDE bridge
|
||||
- `forge.lthn.ai/core/gui` — display service, 74 MCP tools
|
||||
- `forge.lthn.ai/core/go-io` — sandboxed filesystem
|
||||
- `forge.lthn.ai/core/go-log` — structured logging + errors
|
||||
- `forge.lthn.ai/core/config` — configuration
|
||||
|
||||
### Remove (indirect cleanup)
|
||||
|
||||
- `forge.lthn.ai/core/agent` — no longer imported directly (brain tools via core/mcp)
|
||||
- Direct `gorilla/websocket` — replaced by go-ws
|
||||
|
||||
## Configuration
|
||||
|
||||
`.core/config.yaml`:
|
||||
|
||||
```yaml
|
||||
gui:
|
||||
enabled: true # false = no Wails, Core still runs
|
||||
mcp:
|
||||
transport: stdio # stdio | tcp | unix
|
||||
tcp:
|
||||
port: 9877
|
||||
brain:
|
||||
api_url: http://localhost:8000
|
||||
api_token: "" # or CORE_API_TOKEN env var
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
- Core framework tests live in their respective packages (core/mcp, core/gui)
|
||||
- IDE-specific tests: Wails service startup, config loading, systray lifecycle
|
||||
- Integration: `core-ide --mcp` stdio round-trip with a test MCP client
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Angular frontend changes (existing routes work as-is)
|
||||
- New MCP tool categories beyond what core/mcp and core/gui already provide
|
||||
- Jobrunner/Forgejo poller (was in headless.go — separate concern, belongs in core/agent)
|
||||
- CoreDeno/TypeScript runtime integration (future phase)
|
||||
|
|
@ -1,336 +0,0 @@
|
|||
# Runtime Provider Loading — Plugin Ecosystem
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
**Depends on:** Service Provider Framework, SCM Provider
|
||||
|
||||
## Problem
|
||||
|
||||
All providers are currently compiled into the binary. Users cannot install,
|
||||
remove, or update providers without rebuilding core-ide. There's no plugin
|
||||
ecosystem — every provider is a Go import in main.go.
|
||||
|
||||
## Solution
|
||||
|
||||
Runtime provider discovery using the Mining namespace pattern. Installed
|
||||
providers run as managed processes with `--namespace` flags. The IDE's Gin
|
||||
router proxies to them. JS bundles load dynamically in Angular. No recompile.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
~/.core/providers/
|
||||
├── cool-widget/
|
||||
│ ├── manifest.yaml # Name, namespace, element, permissions
|
||||
│ ├── cool-widget # Binary (or path to system binary)
|
||||
│ ├── openapi.json # OpenAPI spec
|
||||
│ └── assets/
|
||||
│ └── core-cool-widget.js # Custom element bundle
|
||||
├── data-viz/
|
||||
│ ├── manifest.yaml
|
||||
│ ├── data-viz
|
||||
│ └── assets/
|
||||
│ └── core-data-viz.js
|
||||
└── registry.yaml # Installed providers list
|
||||
```
|
||||
|
||||
### Runtime Flow
|
||||
|
||||
```
|
||||
core-ide (main process)
|
||||
┌─────────────────────────────────┐
|
||||
│ Gin Router │
|
||||
│ /api/v1/scm/* → compiled in │
|
||||
│ /api/v1/brain/* → compiled in │
|
||||
│ /api/v1/cool-widget/* → proxy ──┼──→ :9901 (cool-widget binary)
|
||||
│ /api/v1/data-viz/* → proxy ──┼──→ :9902 (data-viz binary)
|
||||
│ │
|
||||
│ Angular Shell │
|
||||
│ <core-scm-panel> → compiled │
|
||||
│ <core-cool-widget> → dynamic │
|
||||
│ <core-data-viz> → dynamic │
|
||||
└─────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Manifest Format
|
||||
|
||||
`.core/manifest.yaml` (same as go-scm's manifest loader):
|
||||
|
||||
```yaml
|
||||
code: cool-widget
|
||||
name: Cool Widget Dashboard
|
||||
version: 1.0.0
|
||||
author: someone
|
||||
licence: EUPL-1.2
|
||||
|
||||
# Provider configuration
|
||||
namespace: /api/v1/cool-widget
|
||||
port: 0 # 0 = auto-assign
|
||||
binary: ./cool-widget # Relative to provider dir
|
||||
args: [] # Additional CLI args
|
||||
|
||||
# UI
|
||||
element:
|
||||
tag: core-cool-widget
|
||||
source: ./assets/core-cool-widget.js
|
||||
|
||||
# Layout
|
||||
layout: HCF
|
||||
slots:
|
||||
H: toolbar
|
||||
C: dashboard
|
||||
F: status
|
||||
|
||||
# OpenAPI spec
|
||||
spec: ./openapi.json
|
||||
|
||||
# Permissions (for TIM sandbox — future)
|
||||
permissions:
|
||||
network: ["api.example.com"]
|
||||
filesystem: ["~/.core/providers/cool-widget/data/"]
|
||||
|
||||
# Signature
|
||||
sign: <ed25519 signature>
|
||||
```
|
||||
|
||||
## Provider Lifecycle
|
||||
|
||||
### Discovery
|
||||
|
||||
On startup, the IDE scans `~/.core/providers/*/manifest.yaml`:
|
||||
|
||||
```go
|
||||
func DiscoverProviders(dir string) ([]RuntimeProvider, error) {
|
||||
entries, _ := os.ReadDir(dir)
|
||||
var providers []RuntimeProvider
|
||||
for _, e := range entries {
|
||||
if !e.IsDir() { continue }
|
||||
m, err := manifest.Load(filepath.Join(dir, e.Name()))
|
||||
if err != nil { continue }
|
||||
providers = append(providers, RuntimeProvider{
|
||||
Dir: filepath.Join(dir, e.Name()),
|
||||
Manifest: m,
|
||||
})
|
||||
}
|
||||
return providers, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Start
|
||||
|
||||
For each discovered provider:
|
||||
|
||||
1. Assign a free port (if `port: 0` in manifest)
|
||||
2. Start the binary via go-process: `./cool-widget --namespace /api/v1/cool-widget --port 9901`
|
||||
3. Wait for health check: `GET http://localhost:9901/health`
|
||||
4. Register a `ProxyProvider` in the API engine that reverse-proxies to that port
|
||||
5. Serve the JS bundle as a static asset at `/assets/{code}.js`
|
||||
|
||||
```go
|
||||
type RuntimeProvider struct {
|
||||
Dir string
|
||||
Manifest *manifest.Manifest
|
||||
Process *process.Daemon
|
||||
Port int
|
||||
}
|
||||
|
||||
func (rp *RuntimeProvider) Start(engine *api.Engine, hub *ws.Hub) error {
|
||||
// Start binary
|
||||
rp.Port = findFreePort()
|
||||
rp.Process = process.NewDaemon(process.DaemonOptions{
|
||||
Command: filepath.Join(rp.Dir, rp.Manifest.Binary),
|
||||
Args: append(rp.Manifest.Args, "--namespace", rp.Manifest.Namespace, "--port", strconv.Itoa(rp.Port)),
|
||||
PIDFile: filepath.Join(rp.Dir, "provider.pid"),
|
||||
})
|
||||
rp.Process.Start()
|
||||
|
||||
// Wait for health
|
||||
waitForHealth(rp.Port)
|
||||
|
||||
// Register proxy provider
|
||||
proxy := provider.NewProxy(provider.ProxyConfig{
|
||||
Name: rp.Manifest.Code,
|
||||
BasePath: rp.Manifest.Namespace,
|
||||
Upstream: fmt.Sprintf("http://127.0.0.1:%d", rp.Port),
|
||||
Element: rp.Manifest.Element,
|
||||
SpecFile: filepath.Join(rp.Dir, rp.Manifest.Spec),
|
||||
})
|
||||
engine.Register(proxy)
|
||||
|
||||
// Serve JS assets
|
||||
engine.Router().Static("/assets/"+rp.Manifest.Code, filepath.Join(rp.Dir, "assets"))
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Stop
|
||||
|
||||
On IDE quit or provider removal:
|
||||
1. Send SIGTERM to provider process
|
||||
2. Remove proxy routes from Gin
|
||||
3. Unload JS bundle from Angular
|
||||
|
||||
### Hot Reload (Development)
|
||||
|
||||
During development (`core dev` in a provider dir):
|
||||
1. Watch for binary changes → restart process
|
||||
2. Watch for JS changes → reload in Angular
|
||||
3. Watch for manifest changes → re-register proxy
|
||||
|
||||
## Install / Remove
|
||||
|
||||
### Install
|
||||
|
||||
```bash
|
||||
core install forge.lthn.ai/someone/cool-widget
|
||||
```
|
||||
|
||||
1. Clone or download the provider repo
|
||||
2. Verify Ed25519 signature in manifest
|
||||
3. If Go source: `go build -o cool-widget .` in the provider dir
|
||||
4. Copy to `~/.core/providers/cool-widget/`
|
||||
5. Update `~/.core/providers/registry.yaml`
|
||||
6. If IDE is running: hot-load the provider (no restart needed)
|
||||
|
||||
### Remove
|
||||
|
||||
```bash
|
||||
core remove cool-widget
|
||||
```
|
||||
|
||||
1. Stop the provider process
|
||||
2. Remove from `~/.core/providers/cool-widget/`
|
||||
3. Update `~/.core/providers/registry.yaml`
|
||||
4. If IDE is running: unload the proxy + UI
|
||||
|
||||
### Update
|
||||
|
||||
```bash
|
||||
core update cool-widget
|
||||
```
|
||||
|
||||
1. Pull latest from git
|
||||
2. Verify new signature
|
||||
3. Rebuild if source-based
|
||||
4. Stop old process, start new
|
||||
5. Reload JS bundle
|
||||
|
||||
## Registry
|
||||
|
||||
`~/.core/providers/registry.yaml`:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
providers:
|
||||
cool-widget:
|
||||
installed: "2026-03-14T12:00:00Z"
|
||||
version: 1.0.0
|
||||
source: forge.lthn.ai/someone/cool-widget
|
||||
auto_start: true
|
||||
data-viz:
|
||||
installed: "2026-03-14T13:00:00Z"
|
||||
version: 0.2.0
|
||||
source: github.com/user/data-viz
|
||||
auto_start: true
|
||||
```
|
||||
|
||||
## Custom Binary Build
|
||||
|
||||
```bash
|
||||
core build --brand "My Product" --include cool-widget,data-viz
|
||||
```
|
||||
|
||||
Instead of runtime proxy, this compiles the selected providers directly
|
||||
into the binary:
|
||||
|
||||
1. Read each provider's Go source
|
||||
2. Import as compiled providers (not proxied)
|
||||
3. Embed JS bundles via `//go:embed`
|
||||
4. Set binary name, icon, and metadata from brand config
|
||||
5. Output: single binary with everything compiled in
|
||||
|
||||
Same providers, two modes: proxied (plugin) or compiled (product).
|
||||
|
||||
## Provider Binary Contract
|
||||
|
||||
A provider binary must:
|
||||
|
||||
1. Accept `--namespace` flag (API route prefix)
|
||||
2. Accept `--port` flag (HTTP listen port)
|
||||
3. Serve `GET /health` → `{"status": "ok"}`
|
||||
4. Serve its API under the namespace path
|
||||
5. Optionally accept `--ws-url` flag to connect to IDE's WS hub for events
|
||||
|
||||
The element-template already scaffolds this pattern.
|
||||
|
||||
## Swagger Aggregation
|
||||
|
||||
The IDE's `SpecBuilder` aggregates OpenAPI specs from:
|
||||
1. Compiled providers (via `DescribableGroup.Describe()`)
|
||||
2. Runtime providers (via their `openapi.json` files)
|
||||
|
||||
Merged into one spec at `/swagger/doc.json`. The Swagger UI shows all
|
||||
providers' endpoints in one place.
|
||||
|
||||
## Angular Dynamic Loading
|
||||
|
||||
Custom elements load at runtime without Angular knowing about them at
|
||||
build time:
|
||||
|
||||
```typescript
|
||||
// In the IDE's Angular shell
|
||||
async function loadProviderElement(tag: string, scriptUrl: string) {
|
||||
if (customElements.get(tag)) return; // Already loaded
|
||||
|
||||
const script = document.createElement('script');
|
||||
script.type = 'module';
|
||||
script.src = scriptUrl;
|
||||
document.head.appendChild(script);
|
||||
|
||||
// Wait for registration
|
||||
await customElements.whenDefined(tag);
|
||||
}
|
||||
```
|
||||
|
||||
The tray panel and IDE layout call this for each Renderable provider
|
||||
discovered at startup. Angular wraps the custom element in a host component
|
||||
for the HLCRF slot assignment.
|
||||
|
||||
## Security
|
||||
|
||||
### Signature Verification
|
||||
|
||||
All manifests must be signed (Ed25519). Unsigned providers are rejected
|
||||
unless `--allow-unsigned` is passed (development only).
|
||||
|
||||
### Process Isolation
|
||||
|
||||
Provider processes run as the current user with no special privileges.
|
||||
Future: TIM containers for full sandbox (filesystem + network isolation
|
||||
per the manifest's permissions declaration).
|
||||
|
||||
### Network
|
||||
|
||||
Providers listen on `127.0.0.1` only. No external network exposure.
|
||||
The IDE's Gin router is the only entry point.
|
||||
|
||||
## Implementation Location
|
||||
|
||||
| Component | Package | New/Existing |
|
||||
|-----------|---------|-------------|
|
||||
| Provider discovery | go-scm/marketplace | Extend existing |
|
||||
| Process management | go-process | Existing daemon API |
|
||||
| Proxy provider | core/api/pkg/provider | New: proxy.go |
|
||||
| Install/remove CLI | core/cli cmd/ | New commands |
|
||||
| Runtime loader | core/ide | New: runtime.go |
|
||||
| JS dynamic loading | core/ide frontend/ | New: provider-loader service |
|
||||
| Registry file | go-scm/marketplace | Extend existing |
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- TIM container sandbox (future — Phase 4 from provider framework spec)
|
||||
- Provider marketplace server (git-based discovery is sufficient)
|
||||
- Revenue sharing / paid providers (future — SMSG licensing)
|
||||
- Angular module federation (future — current pattern is custom elements)
|
||||
- Multi-language provider SDKs (future — element-template is Go-first)
|
||||
|
|
@ -1,258 +0,0 @@
|
|||
# go-scm Service Provider + Custom Elements
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
**Depends on:** Service Provider Framework, core/api
|
||||
|
||||
## Problem
|
||||
|
||||
go-scm has marketplace, manifest, and registry functionality but no REST API
|
||||
or UI. The IDE can't browse providers, install apps, or inspect manifests
|
||||
without CLI commands.
|
||||
|
||||
## Solution
|
||||
|
||||
Add a service provider to go-scm with REST endpoints and a Lit custom element
|
||||
bundle containing multiple composable elements.
|
||||
|
||||
## Provider (Go)
|
||||
|
||||
### ScmProvider
|
||||
|
||||
Lives in `go-scm/pkg/api/provider.go`. Implements `Provider` + `Streamable` +
|
||||
`Describable` + `Renderable`.
|
||||
|
||||
```go
|
||||
Name() → "scm"
|
||||
BasePath() → "/api/v1/scm"
|
||||
Element() → ElementSpec{Tag: "core-scm-panel", Source: "/assets/core-scm.js"}
|
||||
Channels() → []string{"scm.marketplace.*", "scm.manifest.*", "scm.registry.*"}
|
||||
```
|
||||
|
||||
### REST Endpoints
|
||||
|
||||
#### Marketplace
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | /marketplace | List available providers from git registry |
|
||||
| GET | /marketplace/:code | Get provider details |
|
||||
| POST | /marketplace/:code/install | Install provider |
|
||||
| DELETE | /marketplace/:code | Remove installed provider |
|
||||
| POST | /marketplace/refresh | Pull latest marketplace index (requires new FetchIndex function) |
|
||||
|
||||
#### Manifest
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | /manifest | Read .core/manifest.yaml from current directory |
|
||||
| POST | /manifest/verify | Verify Ed25519 signature (body: {public_key: hex}) |
|
||||
| POST | /manifest/sign | Sign manifest with private key (body: {private_key: hex}) |
|
||||
| GET | /manifest/permissions | List declared permissions |
|
||||
|
||||
#### Installed
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | /installed | List installed providers |
|
||||
| POST | /installed/:code/update | Apply update (pulls latest from git) |
|
||||
|
||||
Note: Single-item `GET /installed/:code` and update-check require new methods
|
||||
on `Installer` (`Get(code)` and `CheckUpdate(code)`). Deferred — the list
|
||||
endpoint is sufficient for Phase 1. Update applies `Installer.Update()`.
|
||||
|
||||
#### Registry
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | /registry | List repos from repos.yaml |
|
||||
| GET | /registry/:name/status | Git status for a repo |
|
||||
|
||||
Note: Registry endpoints are read-only. Pull/push actions are handled by
|
||||
`core dev` CLI commands, not the REST API. The `<core-scm-registry>` element
|
||||
shows status only — no write buttons.
|
||||
|
||||
### WS Events
|
||||
|
||||
| Event | Data | Trigger |
|
||||
|-------|------|---------|
|
||||
| scm.marketplace.refreshed | {count} | After marketplace pull |
|
||||
| scm.marketplace.installed | {code, name, version, installed_at} | After install |
|
||||
| scm.marketplace.removed | {code} | After remove |
|
||||
| scm.manifest.verified | {code, valid, signer} | After signature check |
|
||||
| scm.registry.changed | {name, status} | Repo status change |
|
||||
|
||||
## Custom Elements (Lit)
|
||||
|
||||
### Bundle Structure
|
||||
|
||||
```
|
||||
go-scm/ui/
|
||||
├── src/
|
||||
│ ├── index.ts # Bundle entry — exports all elements
|
||||
│ ├── scm-panel.ts # <core-scm-panel> — HLCRF layout
|
||||
│ ├── scm-marketplace.ts # <core-scm-marketplace> — browse/install
|
||||
│ ├── scm-manifest.ts # <core-scm-manifest> — view/verify
|
||||
│ ├── scm-installed.ts # <core-scm-installed> — manage installed
|
||||
│ ├── scm-registry.ts # <core-scm-registry> — repo status
|
||||
│ └── shared/
|
||||
│ ├── api.ts # Fetch wrapper for /api/v1/scm/*
|
||||
│ └── events.ts # WS event listener helpers
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
└── index.html # Demo page
|
||||
```
|
||||
|
||||
### Elements
|
||||
|
||||
#### `<core-scm-panel>`
|
||||
|
||||
Top-level element. Arranges child elements in HLCRF layout `H[LC]CF`:
|
||||
- H: Title bar with refresh button
|
||||
- H-L: Navigation tabs (Marketplace / Installed / Registry)
|
||||
- H-C: Search input
|
||||
- C: Active tab content (one of the child elements)
|
||||
- F: Status bar (connection state, last refresh)
|
||||
|
||||
#### `<core-scm-marketplace>`
|
||||
|
||||
Browse the git-based provider marketplace.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| api-url | string | API base URL (default: current origin) |
|
||||
| category | string | Filter by category |
|
||||
|
||||
Displays:
|
||||
- Provider cards with name, description, version, author
|
||||
- Install/Remove button per card
|
||||
- Category filter tabs
|
||||
- Search (filters client-side)
|
||||
|
||||
#### `<core-scm-manifest>`
|
||||
|
||||
View and verify a .core/manifest.yaml file.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| api-url | string | API base URL |
|
||||
| path | string | Directory to read manifest from |
|
||||
|
||||
Displays:
|
||||
- Manifest fields (code, name, version, layout variant)
|
||||
- HLCRF slot assignments
|
||||
- Permission declarations
|
||||
- Signature status badge (verified/unsigned/invalid)
|
||||
- Sign button (calls POST /manifest/sign)
|
||||
|
||||
#### `<core-scm-installed>`
|
||||
|
||||
Manage installed providers.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| api-url | string | API base URL |
|
||||
|
||||
Displays:
|
||||
- Installed provider list with version, status
|
||||
- Update available indicator
|
||||
- Update/Remove buttons
|
||||
- Provider detail panel on click
|
||||
|
||||
#### `<core-scm-registry>`
|
||||
|
||||
Show repos.yaml registry status.
|
||||
|
||||
| Attribute | Type | Description |
|
||||
|-----------|------|-------------|
|
||||
| api-url | string | API base URL |
|
||||
|
||||
Displays:
|
||||
- Repo list from registry
|
||||
- Git status per repo (clean/dirty/ahead/behind)
|
||||
|
||||
### Shared
|
||||
|
||||
#### `api.ts`
|
||||
|
||||
```typescript
|
||||
export class ScmApi {
|
||||
constructor(private baseUrl: string = '') {}
|
||||
|
||||
private get base() {
|
||||
return `${this.baseUrl}/api/v1/scm`;
|
||||
}
|
||||
|
||||
marketplace() { return fetch(`${this.base}/marketplace`).then(r => r.json()); }
|
||||
install(code: string){ return fetch(`${this.base}/marketplace/${code}/install`, {method:'POST'}).then(r => r.json()); }
|
||||
remove(code: string) { return fetch(`${this.base}/marketplace/${code}`, {method:'DELETE'}).then(r => r.json()); }
|
||||
installed() { return fetch(`${this.base}/installed`).then(r => r.json()); }
|
||||
manifest() { return fetch(`${this.base}/manifest`).then(r => r.json()); }
|
||||
verify(publicKey: string) { return fetch(`${this.base}/manifest/verify`, {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({public_key: publicKey})}).then(r => r.json()); }
|
||||
sign(privateKey: string) { return fetch(`${this.base}/manifest/sign`, {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({private_key: privateKey})}).then(r => r.json()); }
|
||||
registry() { return fetch(`${this.base}/registry`).then(r => r.json()); }
|
||||
}
|
||||
```
|
||||
|
||||
#### `events.ts`
|
||||
|
||||
```typescript
|
||||
export function connectScmEvents(wsUrl: string, handler: (event: any) => void) {
|
||||
const ws = new WebSocket(wsUrl);
|
||||
ws.onmessage = (e) => {
|
||||
const event = JSON.parse(e.data);
|
||||
if (event.type?.startsWith('scm.')) handler(event);
|
||||
};
|
||||
return ws;
|
||||
}
|
||||
```
|
||||
|
||||
## Build
|
||||
|
||||
```bash
|
||||
cd go-scm/ui
|
||||
npm install
|
||||
npm run build # → dist/core-scm.js (single bundle, all elements)
|
||||
```
|
||||
|
||||
The built JS is embedded in go-scm's Go binary via `//go:embed` and served
|
||||
as a static asset by the provider.
|
||||
|
||||
## Dependencies
|
||||
|
||||
### Go (go-scm)
|
||||
- core/api (provider interfaces, Gin)
|
||||
- go-ws (WS hub for events)
|
||||
- Existing: manifest, marketplace, repos packages (already in go-scm)
|
||||
|
||||
### TypeScript (ui/)
|
||||
- lit (Web Components)
|
||||
- No other runtime deps
|
||||
|
||||
## Files
|
||||
|
||||
### Create in go-scm
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `pkg/api/provider.go` | ScmProvider with all REST endpoints |
|
||||
| `pkg/api/provider_test.go` | Endpoint tests |
|
||||
| `pkg/api/embed.go` | `//go:embed` for UI assets |
|
||||
| `ui/src/index.ts` | Bundle entry |
|
||||
| `ui/src/scm-panel.ts` | Top-level HLCRF panel |
|
||||
| `ui/src/scm-marketplace.ts` | Marketplace browser |
|
||||
| `ui/src/scm-manifest.ts` | Manifest viewer |
|
||||
| `ui/src/scm-installed.ts` | Installed provider manager |
|
||||
| `ui/src/scm-registry.ts` | Registry status |
|
||||
| `ui/src/shared/api.ts` | API client |
|
||||
| `ui/src/shared/events.ts` | WS event helpers |
|
||||
| `ui/package.json` | Lit + TypeScript deps |
|
||||
| `ui/tsconfig.json` | TypeScript config |
|
||||
| `ui/index.html` | Demo page |
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Actual STIM packaging/distribution (future — uses Borg)
|
||||
- Provider sandbox enforcement (future — uses TIM/CoreDeno)
|
||||
- Marketplace git server (uses existing forge)
|
||||
- Angular wrappers (IDE dynamically loads custom elements)
|
||||
|
|
@ -1,290 +0,0 @@
|
|||
# Service Provider Framework — Polyglot API + UI
|
||||
|
||||
**Date:** 2026-03-14
|
||||
**Status:** Approved
|
||||
**Depends on:** IDE Modernisation (2026-03-14)
|
||||
|
||||
## Problem
|
||||
|
||||
Each package in the ecosystem (Go, PHP, TypeScript) builds its own API endpoints,
|
||||
WebSocket events, and UI components independently. There's no standard way for a
|
||||
package to say "I provide these capabilities" and have them automatically
|
||||
assembled into an API router, MCP server, or GUI.
|
||||
|
||||
The Mining repo proves the pattern works — Gin routes + WS events + Angular
|
||||
custom element = full process management UI. But it's hand-wired for one use case.
|
||||
|
||||
## Vision
|
||||
|
||||
Any package in any language can register as a service provider. The contract is
|
||||
OpenAPI. Go packages implement a Go interface directly. PHP and TypeScript
|
||||
packages publish an OpenAPI spec and run their own HTTP handler — the API layer
|
||||
reverse-proxies or aggregates. The result:
|
||||
|
||||
- Every provider automatically gets a REST API
|
||||
- Every provider with a custom element automatically gets a GUI panel
|
||||
- Every provider with tool descriptions automatically gets MCP tools
|
||||
- The language the provider is written in is irrelevant
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
.core/config.yaml
|
||||
│
|
||||
├─→ core/go-api (Service Registry)
|
||||
│ ├─ Go providers: implement Provider interface directly
|
||||
│ ├─ PHP providers: OpenAPI spec + reverse proxy to FrankenPHP
|
||||
│ ├─ TS providers: OpenAPI spec + reverse proxy to CoreDeno
|
||||
│ ├─ Assembled Gin router (all routes merged)
|
||||
│ └─ WS hub (all events merged)
|
||||
│
|
||||
├─→ core/gui (Display Layer)
|
||||
│ ├─ Discovers Renderable providers
|
||||
│ ├─ Loads custom elements into Angular shell
|
||||
│ └─ HLCRF layout from .core/ config
|
||||
│
|
||||
├─→ core/mcp (Tool Layer)
|
||||
│ ├─ Discovers Describable providers
|
||||
│ ├─ Registers as MCP tools
|
||||
│ └─ stdio/TCP/Unix transports
|
||||
│
|
||||
└─→ core/ide (Application Shell)
|
||||
├─ Wails systray + Angular frontend
|
||||
├─ Hosts go-api router
|
||||
└─ Hosts core/mcp server
|
||||
```
|
||||
|
||||
## The Provider Interface (Go)
|
||||
|
||||
Lives in `core/go-api/pkg/provider/`. Built on top of the existing `RouteGroup`
|
||||
and `DescribableGroup` interfaces — providers ARE route groups, not a parallel
|
||||
system.
|
||||
|
||||
```go
|
||||
// Provider extends RouteGroup with a provider identity.
|
||||
// Every Provider is a RouteGroup and registers through api.Engine.Register().
|
||||
type Provider interface {
|
||||
api.RouteGroup // Name(), BasePath(), RegisterRoutes(*gin.RouterGroup)
|
||||
}
|
||||
|
||||
// Streamable providers emit real-time events via WebSocket.
|
||||
// The hub is injected at construction time. Channels() declares the
|
||||
// event prefixes this provider will emit (e.g. "brain.*").
|
||||
type Streamable interface {
|
||||
Provider
|
||||
Channels() []string // Event prefixes emitted by this provider
|
||||
}
|
||||
|
||||
// Describable providers expose structured route descriptions for OpenAPI.
|
||||
// This extends the existing DescribableGroup interface.
|
||||
type Describable interface {
|
||||
Provider
|
||||
api.DescribableGroup // Describe() []RouteDescription
|
||||
}
|
||||
|
||||
// Renderable providers declare a custom element for GUI display.
|
||||
type Renderable interface {
|
||||
Provider
|
||||
Element() ElementSpec
|
||||
}
|
||||
|
||||
type ElementSpec struct {
|
||||
Tag string // e.g. "core-brain-panel"
|
||||
Source string // URL or embedded path to the JS bundle
|
||||
}
|
||||
```
|
||||
|
||||
Note: `Manageable` (Start/Stop/Status) is deferred to Phase 2. In Phase 1,
|
||||
provider lifecycle is handled by `core.Core`'s existing `Startable`/`Stoppable`
|
||||
interfaces — providers that need lifecycle management implement those directly
|
||||
when registered as Core services.
|
||||
|
||||
### Registration
|
||||
|
||||
Providers register through the existing `api.Engine`, not a parallel router.
|
||||
This gives them middleware, CORS, Swagger, health checks, and OpenAPI for free.
|
||||
|
||||
```go
|
||||
engine, _ := api.New(
|
||||
api.WithCORS(),
|
||||
api.WithSwagger(),
|
||||
api.WithWSHub(hub),
|
||||
)
|
||||
|
||||
// Register providers as route groups — they get middleware, OpenAPI, etc.
|
||||
engine.Register(brain.NewProvider(bridge, hub))
|
||||
engine.Register(daemon.NewProvider(registry))
|
||||
engine.Register(build.NewProvider())
|
||||
|
||||
// Providers that are Streamable have the hub injected at construction.
|
||||
// They call hub.SendToChannel("brain.recall.complete", event) internally.
|
||||
```
|
||||
|
||||
The `Registry` type is a convenience wrapper that collects providers and
|
||||
calls `engine.Register()` for each:
|
||||
|
||||
```go
|
||||
reg := provider.NewRegistry()
|
||||
reg.Add(brain.NewProvider(bridge, hub))
|
||||
reg.Add(build.NewProvider())
|
||||
reg.MountAll(engine) // calls engine.Register() for each
|
||||
```
|
||||
|
||||
## Polyglot Providers (PHP, TypeScript)
|
||||
|
||||
Non-Go providers don't implement the Go interface. They:
|
||||
|
||||
1. Publish an OpenAPI spec in `.core/providers/{name}.yaml`
|
||||
2. Run their own HTTP server (FrankenPHP, CoreDeno, or any process)
|
||||
3. The Go API layer discovers the spec and creates a reverse proxy route group
|
||||
|
||||
```yaml
|
||||
# .core/providers/studio.yaml
|
||||
name: studio
|
||||
language: php
|
||||
spec: openapi-3.json # Path to OpenAPI spec
|
||||
endpoint: http://localhost:8000 # Where the PHP handler listens
|
||||
element:
|
||||
tag: core-studio-panel
|
||||
source: /assets/studio-panel.js
|
||||
events:
|
||||
- studio.render.started
|
||||
- studio.render.complete
|
||||
```
|
||||
|
||||
The Go registry wraps this as a `ProxyProvider` — it implements `Provider` by
|
||||
reverse-proxying to the endpoint, `Describable` by reading the spec file,
|
||||
and `Renderable` by reading the element config.
|
||||
|
||||
For real-time events, the upstream process connects to the Go WS hub as a
|
||||
client (using `ws.ReconnectingClient`) or pushes events via the go-ws Redis
|
||||
pub/sub backend. The `ProxyProvider` declares the expected channels from the
|
||||
YAML config. The mechanism choice depends on deployment: Redis for multi-host,
|
||||
direct WS for single-binary.
|
||||
|
||||
### OpenAPI as Contract
|
||||
|
||||
The OpenAPI spec is the single source of truth for:
|
||||
- **go-api**: Route mounting and request validation
|
||||
- **core/mcp**: Automatic MCP tool generation from endpoints
|
||||
- **core/gui**: Form generation for Manageable providers
|
||||
- **SDK codegen**: TypeScript/Python/PHP client generation (already in go-api)
|
||||
|
||||
A PHP package that publishes a valid OpenAPI spec gets all four for free.
|
||||
|
||||
## Discovery
|
||||
|
||||
Provider discovery follows the `.core/` convention:
|
||||
|
||||
1. **Static config** — `.core/config.yaml` lists enabled providers
|
||||
2. **Directory scan** — `.core/providers/*.yaml` for polyglot provider specs
|
||||
3. **Go registration** — `core.WithService(provider.Register(registry))` in main.go
|
||||
|
||||
```yaml
|
||||
# .core/config.yaml
|
||||
providers:
|
||||
brain:
|
||||
enabled: true
|
||||
studio:
|
||||
enabled: true
|
||||
endpoint: http://localhost:8000
|
||||
gallery:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
## GUI Integration
|
||||
|
||||
`core/gui`'s display service queries the registry for `Renderable` providers.
|
||||
For each one, it:
|
||||
|
||||
1. Loads the custom element JS bundle (from `ElementSpec.Source`)
|
||||
2. Creates an Angular wrapper component that hosts the custom element
|
||||
3. Registers it in the available panels list
|
||||
4. Layout is configured via `.core/config.yaml` or defaults to auto-arrangement
|
||||
|
||||
The Angular shell doesn't know about providers at build time. Custom elements
|
||||
are loaded dynamically at runtime. This is the same pattern as Mining's
|
||||
`<mbe-mining-dashboard>` — a self-contained web component that talks to the
|
||||
Gin API via fetch/WS.
|
||||
|
||||
### Tray Panel
|
||||
|
||||
The systray control pane shows:
|
||||
- List of registered providers with status indicators
|
||||
- Start/Stop controls for Manageable providers
|
||||
- Quick stats for Streamable providers
|
||||
- Click to open full panel in a new window
|
||||
|
||||
## WS Event Protocol
|
||||
|
||||
All providers share a single WS hub. Events are namespaced by provider:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "brain.recall.complete",
|
||||
"timestamp": "2026-03-14T10:30:00Z",
|
||||
"data": { "query": "...", "results": 5 }
|
||||
}
|
||||
```
|
||||
|
||||
Angular services filter by prefix (`brain.*`, `studio.*`, etc.).
|
||||
This is identical to Mining's `WebSocketService` pattern but generalised.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Go Provider Framework (this spec)
|
||||
- `Provider` interface (extends `RouteGroup`) + `Registry` in `core/go-api/pkg/provider/`
|
||||
- Providers register through existing `api.Engine` — get middleware, OpenAPI, Swagger for free
|
||||
- Streamable providers receive WS hub at construction, declare channel prefixes
|
||||
- **go-process as first provider** — daemon registry, PID files, health checks → `<core-process-panel>`
|
||||
- Brain as second provider
|
||||
- core/ide consumes the registry
|
||||
- Element template: [core-element-template](https://github.com/Snider/core-element-template) — Go CLI + Lit custom element scaffold for new providers
|
||||
|
||||
### Phase 2: GUI Consumer
|
||||
- core/gui discovers Renderable providers
|
||||
- Dynamic custom element loading in Angular shell
|
||||
- Tray panel with provider status
|
||||
- HLCRF layout configuration
|
||||
|
||||
### Phase 3: Polyglot Providers
|
||||
- `ProxyProvider` for PHP/TS providers
|
||||
- `.core/providers/*.yaml` discovery
|
||||
- OpenAPI spec → MCP tool auto-generation
|
||||
- PHP packages (core/php-*) expose providers via FrankenPHP
|
||||
- TS packages (core/ts) expose providers via CoreDeno
|
||||
|
||||
### Phase 4: SDK + Marketplace
|
||||
- Auto-generate client SDKs from assembled OpenAPI spec
|
||||
- Provider marketplace (git-based, same pattern as dAppServer)
|
||||
- Signed provider manifests (ed25519, from `.core/view.yml` spec)
|
||||
|
||||
## Files (Phase 1)
|
||||
|
||||
### Create in core/go-api
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `pkg/provider/provider.go` | Provider (extends RouteGroup), Streamable, Describable, Renderable interfaces |
|
||||
| `pkg/provider/registry.go` | Registry: Add, MountAll(engine), List |
|
||||
| `pkg/provider/proxy.go` | ProxyProvider for polyglot (Phase 3, stub for now) |
|
||||
| `pkg/provider/registry_test.go` | Unit tests |
|
||||
|
||||
### Update in core/ide
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `main.go` | Create registry, register providers, mount router |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `core/go-api` — Gin, route groups, OpenAPI (already there)
|
||||
- `core/go-ws` — WS hub (already there)
|
||||
- No new external dependencies
|
||||
|
||||
## Not In Scope
|
||||
|
||||
- Angular component library (Phase 2)
|
||||
- PHP/TS provider runtime (Phase 3)
|
||||
- Provider marketplace (Phase 4)
|
||||
- Authentication/authorisation per provider (future — Authentik integration)
|
||||
Loading…
Add table
Reference in a new issue