From f71f4c7d66b1756bac8a596a567cdc57f6f6d3d4 Mon Sep 17 00:00:00 2001 From: Snider Date: Fri, 6 Mar 2026 14:24:35 +0000 Subject: [PATCH] docs: remove stale CLI/ecosystem docs, keep framework-only content MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Massive cleanup after module extraction sprint. core/go is now a pure DI framework — docs should reflect that, not document CLI commands. - Delete 130+ CLI command/example docs (already in core/cli) - Delete 6 obsolete pkg-batch*-analysis.md files - Delete plans/, skill/, static/, mcp/ (moved to correct repos) - Rewrite index.md for DI framework (not CLI) - Fix PACKAGE_STANDARDS.md: framework.* → core.* references - Fix log.md: correct framework integration example Remaining docs: index.md, pkg/PACKAGE_STANDARDS.md, pkg/log.md Co-Authored-By: Virgil --- docs/cmd/ai/example.md | 100 - docs/cmd/ai/index.md | 262 -- docs/cmd/build/example.md | 83 - docs/cmd/build/index.md | 176 -- docs/cmd/build/sdk/example.md | 56 - docs/cmd/build/sdk/index.md | 27 - docs/cmd/ci/changelog/example.md | 36 - docs/cmd/ci/changelog/index.md | 28 - docs/cmd/ci/example.md | 90 - docs/cmd/ci/index.md | 79 - docs/cmd/ci/init/example.md | 17 - docs/cmd/ci/init/index.md | 11 - docs/cmd/ci/version/example.md | 18 - docs/cmd/ci/version/index.md | 21 - docs/cmd/dev/ci/index.md | 61 - docs/cmd/dev/commit/index.md | 46 - docs/cmd/dev/example.md | 203 -- docs/cmd/dev/health/index.md | 52 - docs/cmd/dev/impact/index.md | 65 - docs/cmd/dev/index.md | 388 --- docs/cmd/dev/issues/index.md | 57 - docs/cmd/dev/pull/index.md | 47 - docs/cmd/dev/push/index.md | 52 - docs/cmd/dev/reviews/index.md | 61 - docs/cmd/dev/work/example.md | 33 - docs/cmd/dev/work/index.md | 293 -- docs/cmd/docs/example.md | 14 - docs/cmd/docs/index.md | 110 - docs/cmd/doctor/example.md | 20 - docs/cmd/doctor/index.md | 81 - docs/cmd/go/cov/example.md | 18 - docs/cmd/go/cov/index.md | 28 - docs/cmd/go/example.md | 89 - docs/cmd/go/fmt/example.md | 12 - docs/cmd/go/fmt/index.md | 25 - docs/cmd/go/index.md | 15 - docs/cmd/go/install/example.md | 15 - docs/cmd/go/install/index.md | 25 - docs/cmd/go/lint/example.md | 22 - docs/cmd/go/lint/index.md | 22 - docs/cmd/go/mod/download/index.md | 29 - docs/cmd/go/mod/example.md | 15 - docs/cmd/go/mod/graph/index.md | 44 - docs/cmd/go/mod/index.md | 21 - docs/cmd/go/mod/tidy/index.md | 29 - docs/cmd/go/mod/verify/index.md | 41 - docs/cmd/go/test/example.md | 27 - docs/cmd/go/test/index.md | 31 - docs/cmd/go/work/index.md | 19 - docs/cmd/go/work/init/index.md | 40 - docs/cmd/go/work/sync/index.md | 35 - docs/cmd/go/work/use/index.md | 46 - docs/cmd/index.md | 31 - docs/cmd/php/example.md | 111 - docs/cmd/php/index.md | 413 --- docs/cmd/pkg/example.md | 36 - docs/cmd/pkg/index.md | 144 - docs/cmd/pkg/search/example.md | 23 - docs/cmd/pkg/search/index.md | 75 - docs/cmd/sdk/example.md | 35 - docs/cmd/sdk/index.md | 106 - docs/cmd/setup/example.md | 293 -- docs/cmd/setup/index.md | 213 -- docs/cmd/test/example.md | 8 - docs/cmd/test/index.md | 74 - docs/cmd/vm/example.md | 52 - docs/cmd/vm/index.md | 163 -- docs/cmd/vm/templates/example.md | 53 - docs/cmd/vm/templates/index.md | 124 - docs/configuration.md | 380 --- docs/ecosystem.md | 457 --- docs/examples/build-cpp.yaml | 83 - docs/examples/build-docker-go.yaml | 42 - docs/examples/build-docker.yaml | 40 - docs/examples/build-full.yaml | 121 - docs/examples/build-go-cli.yaml | 39 - docs/examples/build-go-library.yaml | 23 - docs/examples/build-go-wails.yaml | 46 - docs/examples/build-linuxkit.yaml | 33 - docs/examples/build-minimal.yaml | 7 - docs/examples/build-multi-binary.yaml | 51 - docs/examples/build-php-laravel.yaml | 50 - docs/examples/linuxkit-docker.yml | 29 - docs/examples/linuxkit-server.yml | 51 - docs/examples/publish-all.yaml | 68 - docs/examples/publish-aur.yaml | 28 - docs/examples/publish-chocolatey.yaml | 29 - docs/examples/publish-docker.yaml | 38 - docs/examples/publish-github.yaml | 14 - docs/examples/publish-homebrew.yaml | 29 - docs/examples/publish-linuxkit.yaml | 36 - docs/examples/publish-npm.yaml | 21 - docs/examples/publish-scoop.yaml | 23 - docs/examples/release-full.yaml | 98 - docs/examples/release-go-cli.yaml | 59 - docs/examples/release-go-wails.yaml | 36 - docs/examples/release-minimal.yaml | 11 - docs/examples/release-official-repos.yaml | 51 - docs/examples/release-php-laravel.yaml | 42 - docs/examples/sdk-full.yaml | 43 - docs/faq.md | 97 - docs/getting-started.md | 191 -- docs/glossary.md | 112 - docs/index.md | 229 +- docs/mcp/angular-testing.md | 470 --- docs/migration.md | 233 -- docs/pkg-batch1-analysis.md | 213 -- docs/pkg-batch2-analysis.md | 255 -- docs/pkg-batch3-analysis.md | 384 --- docs/pkg-batch4-analysis.md | 366 --- docs/pkg-batch5-analysis.md | 303 -- docs/pkg-batch6-analysis.md | 520 ---- docs/pkg/PACKAGE_STANDARDS.md | 60 +- docs/pkg/i18n/EXTENDING.md | 399 --- docs/pkg/i18n/GRAMMAR.md | 228 -- docs/pkg/i18n/README.md | 420 --- docs/pkg/log.md | 4 +- .../2026-02-20-authentik-traefik-plan.md | 1163 -------- docs/plans/2026-02-21-core-help-design.md | 155 - docs/plans/2026-02-21-core-help-plan.md | 642 ----- .../2026-02-05-mcp-integration-original.md | 851 ------ .../completed/2026-02-17-lem-chat-design.md | 82 - .../2026-02-20-go-api-design-original.md | 657 ----- .../2026-02-20-go-api-plan-original.md | 1503 ---------- ...-02-21-cli-meta-package-design-original.md | 128 - ...6-02-21-cli-sdk-expansion-plan-original.md | 1724 ----------- .../completed/2026-02-21-go-forge-design.md | 286 -- .../completed/2026-02-21-go-forge-plan.md | 2549 ----------------- docs/plans/completed/cli-meta-package.md | 30 - docs/plans/completed/cli-sdk-expansion.md | 39 - docs/plans/completed/go-api.md | 57 - docs/plans/completed/mcp-integration.md | 37 - docs/plans/completed/qk-bone-orientation.md | 62 - docs/skill/index.md | 35 - docs/static/assets/style.css | 0 docs/static/index.html | 0 docs/troubleshooting.md | 356 --- docs/user-guide.md | 100 - docs/workflows.md | 334 --- 139 files changed, 78 insertions(+), 22463 deletions(-) delete mode 100644 docs/cmd/ai/example.md delete mode 100644 docs/cmd/ai/index.md delete mode 100644 docs/cmd/build/example.md delete mode 100644 docs/cmd/build/index.md delete mode 100644 docs/cmd/build/sdk/example.md delete mode 100644 docs/cmd/build/sdk/index.md delete mode 100644 docs/cmd/ci/changelog/example.md delete mode 100644 docs/cmd/ci/changelog/index.md delete mode 100644 docs/cmd/ci/example.md delete mode 100644 docs/cmd/ci/index.md delete mode 100644 docs/cmd/ci/init/example.md delete mode 100644 docs/cmd/ci/init/index.md delete mode 100644 docs/cmd/ci/version/example.md delete mode 100644 docs/cmd/ci/version/index.md delete mode 100644 docs/cmd/dev/ci/index.md delete mode 100644 docs/cmd/dev/commit/index.md delete mode 100644 docs/cmd/dev/example.md delete mode 100644 docs/cmd/dev/health/index.md delete mode 100644 docs/cmd/dev/impact/index.md delete mode 100644 docs/cmd/dev/index.md delete mode 100644 docs/cmd/dev/issues/index.md delete mode 100644 docs/cmd/dev/pull/index.md delete mode 100644 docs/cmd/dev/push/index.md delete mode 100644 docs/cmd/dev/reviews/index.md delete mode 100644 docs/cmd/dev/work/example.md delete mode 100644 docs/cmd/dev/work/index.md delete mode 100644 docs/cmd/docs/example.md delete mode 100644 docs/cmd/docs/index.md delete mode 100644 docs/cmd/doctor/example.md delete mode 100644 docs/cmd/doctor/index.md delete mode 100644 docs/cmd/go/cov/example.md delete mode 100644 docs/cmd/go/cov/index.md delete mode 100644 docs/cmd/go/example.md delete mode 100644 docs/cmd/go/fmt/example.md delete mode 100644 docs/cmd/go/fmt/index.md delete mode 100644 docs/cmd/go/index.md delete mode 100644 docs/cmd/go/install/example.md delete mode 100644 docs/cmd/go/install/index.md delete mode 100644 docs/cmd/go/lint/example.md delete mode 100644 docs/cmd/go/lint/index.md delete mode 100644 docs/cmd/go/mod/download/index.md delete mode 100644 docs/cmd/go/mod/example.md delete mode 100644 docs/cmd/go/mod/graph/index.md delete mode 100644 docs/cmd/go/mod/index.md delete mode 100644 docs/cmd/go/mod/tidy/index.md delete mode 100644 docs/cmd/go/mod/verify/index.md delete mode 100644 docs/cmd/go/test/example.md delete mode 100644 docs/cmd/go/test/index.md delete mode 100644 docs/cmd/go/work/index.md delete mode 100644 docs/cmd/go/work/init/index.md delete mode 100644 docs/cmd/go/work/sync/index.md delete mode 100644 docs/cmd/go/work/use/index.md delete mode 100644 docs/cmd/index.md delete mode 100644 docs/cmd/php/example.md delete mode 100644 docs/cmd/php/index.md delete mode 100644 docs/cmd/pkg/example.md delete mode 100644 docs/cmd/pkg/index.md delete mode 100644 docs/cmd/pkg/search/example.md delete mode 100644 docs/cmd/pkg/search/index.md delete mode 100644 docs/cmd/sdk/example.md delete mode 100644 docs/cmd/sdk/index.md delete mode 100644 docs/cmd/setup/example.md delete mode 100644 docs/cmd/setup/index.md delete mode 100644 docs/cmd/test/example.md delete mode 100644 docs/cmd/test/index.md delete mode 100644 docs/cmd/vm/example.md delete mode 100644 docs/cmd/vm/index.md delete mode 100644 docs/cmd/vm/templates/example.md delete mode 100644 docs/cmd/vm/templates/index.md delete mode 100644 docs/configuration.md delete mode 100644 docs/ecosystem.md delete mode 100644 docs/examples/build-cpp.yaml delete mode 100644 docs/examples/build-docker-go.yaml delete mode 100644 docs/examples/build-docker.yaml delete mode 100644 docs/examples/build-full.yaml delete mode 100644 docs/examples/build-go-cli.yaml delete mode 100644 docs/examples/build-go-library.yaml delete mode 100644 docs/examples/build-go-wails.yaml delete mode 100644 docs/examples/build-linuxkit.yaml delete mode 100644 docs/examples/build-minimal.yaml delete mode 100644 docs/examples/build-multi-binary.yaml delete mode 100644 docs/examples/build-php-laravel.yaml delete mode 100644 docs/examples/linuxkit-docker.yml delete mode 100644 docs/examples/linuxkit-server.yml delete mode 100644 docs/examples/publish-all.yaml delete mode 100644 docs/examples/publish-aur.yaml delete mode 100644 docs/examples/publish-chocolatey.yaml delete mode 100644 docs/examples/publish-docker.yaml delete mode 100644 docs/examples/publish-github.yaml delete mode 100644 docs/examples/publish-homebrew.yaml delete mode 100644 docs/examples/publish-linuxkit.yaml delete mode 100644 docs/examples/publish-npm.yaml delete mode 100644 docs/examples/publish-scoop.yaml delete mode 100644 docs/examples/release-full.yaml delete mode 100644 docs/examples/release-go-cli.yaml delete mode 100644 docs/examples/release-go-wails.yaml delete mode 100644 docs/examples/release-minimal.yaml delete mode 100644 docs/examples/release-official-repos.yaml delete mode 100644 docs/examples/release-php-laravel.yaml delete mode 100644 docs/examples/sdk-full.yaml delete mode 100644 docs/faq.md delete mode 100644 docs/getting-started.md delete mode 100644 docs/glossary.md delete mode 100644 docs/mcp/angular-testing.md delete mode 100644 docs/migration.md delete mode 100644 docs/pkg-batch1-analysis.md delete mode 100644 docs/pkg-batch2-analysis.md delete mode 100644 docs/pkg-batch3-analysis.md delete mode 100644 docs/pkg-batch4-analysis.md delete mode 100644 docs/pkg-batch5-analysis.md delete mode 100644 docs/pkg-batch6-analysis.md delete mode 100644 docs/pkg/i18n/EXTENDING.md delete mode 100644 docs/pkg/i18n/GRAMMAR.md delete mode 100644 docs/pkg/i18n/README.md delete mode 100644 docs/plans/2026-02-20-authentik-traefik-plan.md delete mode 100644 docs/plans/2026-02-21-core-help-design.md delete mode 100644 docs/plans/2026-02-21-core-help-plan.md delete mode 100644 docs/plans/completed/2026-02-05-mcp-integration-original.md delete mode 100644 docs/plans/completed/2026-02-17-lem-chat-design.md delete mode 100644 docs/plans/completed/2026-02-20-go-api-design-original.md delete mode 100644 docs/plans/completed/2026-02-20-go-api-plan-original.md delete mode 100644 docs/plans/completed/2026-02-21-cli-meta-package-design-original.md delete mode 100644 docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md delete mode 100644 docs/plans/completed/2026-02-21-go-forge-design.md delete mode 100644 docs/plans/completed/2026-02-21-go-forge-plan.md delete mode 100644 docs/plans/completed/cli-meta-package.md delete mode 100644 docs/plans/completed/cli-sdk-expansion.md delete mode 100644 docs/plans/completed/go-api.md delete mode 100644 docs/plans/completed/mcp-integration.md delete mode 100644 docs/plans/completed/qk-bone-orientation.md delete mode 100644 docs/skill/index.md delete mode 100644 docs/static/assets/style.css delete mode 100644 docs/static/index.html delete mode 100644 docs/troubleshooting.md delete mode 100644 docs/user-guide.md delete mode 100644 docs/workflows.md diff --git a/docs/cmd/ai/example.md b/docs/cmd/ai/example.md deleted file mode 100644 index b115b09..0000000 --- a/docs/cmd/ai/example.md +++ /dev/null @@ -1,100 +0,0 @@ -# AI Examples - -## Workflow Example - -Complete task management workflow: - -```bash -# 1. List available tasks -core ai tasks --status pending - -# 2. Auto-select and claim a task -core ai task --auto --claim - -# 3. Work on the task... - -# 4. Update progress -core ai task:update abc123 --progress 75 - -# 5. Commit with task reference -core ai task:commit abc123 -m 'implement feature' - -# 6. Create PR -core ai task:pr abc123 - -# 7. Mark complete -core ai task:complete abc123 --output 'Feature implemented and PR created' -``` - -## Task Filtering - -```bash -# By status -core ai tasks --status pending -core ai tasks --status in_progress - -# By priority -core ai tasks --priority critical -core ai tasks --priority high - -# By labels -core ai tasks --labels bug,urgent - -# Combined filters -core ai tasks --status pending --priority high --labels bug -``` - -## Task Updates - -```bash -# Change status -core ai task:update abc123 --status in_progress -core ai task:update abc123 --status blocked - -# Update progress -core ai task:update abc123 --progress 25 -core ai task:update abc123 --progress 50 --notes 'Halfway done' -core ai task:update abc123 --progress 100 -``` - -## Git Integration - -```bash -# Commit with task reference -core ai task:commit abc123 -m 'add authentication' - -# With scope -core ai task:commit abc123 -m 'fix login' --scope auth - -# Commit and push -core ai task:commit abc123 -m 'complete feature' --push - -# Create PR -core ai task:pr abc123 - -# Draft PR -core ai task:pr abc123 --draft - -# PR with labels -core ai task:pr abc123 --labels 'enhancement,ready-for-review' - -# PR to different base -core ai task:pr abc123 --base develop -``` - -## Configuration - -### Environment Variables - -```env -AGENTIC_TOKEN=your-api-token -AGENTIC_BASE_URL=https://agentic.example.com -``` - -### ~/.core/agentic.yaml - -```yaml -token: your-api-token -base_url: https://agentic.example.com -default_project: my-project -``` diff --git a/docs/cmd/ai/index.md b/docs/cmd/ai/index.md deleted file mode 100644 index f6c49be..0000000 --- a/docs/cmd/ai/index.md +++ /dev/null @@ -1,262 +0,0 @@ -# core ai - -AI agent task management and Claude Code integration. - -## Task Management Commands - -| Command | Description | -|---------|-------------| -| `tasks` | List available tasks from core-agentic | -| `task` | View task details or auto-select | -| `task:update` | Update task status or progress | -| `task:complete` | Mark task as completed or failed | -| `task:commit` | Create git commit with task reference | -| `task:pr` | Create GitHub PR linked to task | - -## Claude Integration - -| Command | Description | -|---------|-------------| -| `claude run` | Run Claude Code in current directory | -| `claude config` | Manage Claude configuration | - ---- - -## Configuration - -Task commands load configuration from: -1. Environment variables (`AGENTIC_TOKEN`, `AGENTIC_BASE_URL`) -2. `.env` file in current directory -3. `~/.core/agentic.yaml` - ---- - -## ai tasks - -List available tasks from core-agentic. - -```bash -core ai tasks [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--status` | Filter by status (`pending`, `in_progress`, `completed`, `blocked`) | -| `--priority` | Filter by priority (`critical`, `high`, `medium`, `low`) | -| `--labels` | Filter by labels (comma-separated) | -| `--project` | Filter by project | -| `--limit` | Max number of tasks to return (default: 20) | - -### Examples - -```bash -# List all pending tasks -core ai tasks - -# Filter by status and priority -core ai tasks --status pending --priority high - -# Filter by labels -core ai tasks --labels bug,urgent -``` - ---- - -## ai task - -View task details or auto-select a task. - -```bash -core ai task [task-id] [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--auto` | Auto-select highest priority pending task | -| `--claim` | Claim the task after showing details | -| `--context` | Show gathered context for AI collaboration | - -### Examples - -```bash -# Show task details -core ai task abc123 - -# Show and claim -core ai task abc123 --claim - -# Show with context -core ai task abc123 --context - -# Auto-select highest priority pending task -core ai task --auto -``` - ---- - -## ai task:update - -Update a task's status, progress, or notes. - -```bash -core ai task:update [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--status` | New status (`pending`, `in_progress`, `completed`, `blocked`) | -| `--progress` | Progress percentage (0-100) | -| `--notes` | Notes about the update | - -### Examples - -```bash -# Set task to in progress -core ai task:update abc123 --status in_progress - -# Update progress with notes -core ai task:update abc123 --progress 50 --notes 'Halfway done' -``` - ---- - -## ai task:complete - -Mark a task as completed with optional output and artifacts. - -```bash -core ai task:complete [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--output` | Summary of the completed work | -| `--failed` | Mark the task as failed | -| `--error` | Error message if failed | - -### Examples - -```bash -# Complete successfully -core ai task:complete abc123 --output 'Feature implemented' - -# Mark as failed -core ai task:complete abc123 --failed --error 'Build failed' -``` - ---- - -## ai task:commit - -Create a git commit with a task reference and co-author attribution. - -```bash -core ai task:commit [flags] -``` - -Commit message format: -``` -feat(scope): description - -Task: #123 -Co-Authored-By: Claude -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `-m`, `--message` | Commit message (without task reference) | -| `--scope` | Scope for the commit type (e.g., `auth`, `api`, `ui`) | -| `--push` | Push changes after committing | - -### Examples - -```bash -# Commit with message -core ai task:commit abc123 --message 'add user authentication' - -# With scope -core ai task:commit abc123 -m 'fix login bug' --scope auth - -# Commit and push -core ai task:commit abc123 -m 'update docs' --push -``` - ---- - -## ai task:pr - -Create a GitHub pull request linked to a task. - -```bash -core ai task:pr [flags] -``` - -Requires the GitHub CLI (`gh`) to be installed and authenticated. - -### Flags - -| Flag | Description | -|------|-------------| -| `--title` | PR title (defaults to task title) | -| `--base` | Base branch (defaults to main) | -| `--draft` | Create as draft PR | -| `--labels` | Labels to add (comma-separated) | - -### Examples - -```bash -# Create PR with defaults -core ai task:pr abc123 - -# Custom title -core ai task:pr abc123 --title 'Add authentication feature' - -# Draft PR with labels -core ai task:pr abc123 --draft --labels 'enhancement,needs-review' - -# Target different base branch -core ai task:pr abc123 --base develop -``` - ---- - -## ai claude - -Claude Code integration commands. - -### ai claude run - -Run Claude Code in the current directory. - -```bash -core ai claude run -``` - -### ai claude config - -Manage Claude configuration. - -```bash -core ai claude config -``` - ---- - -## Workflow Example - -See [Workflow Example](example.md#workflow-example) for a complete task management workflow. - -## See Also - -- [dev](../dev/) - Multi-repo workflow commands -- [Claude Code documentation](https://claude.ai/code) diff --git a/docs/cmd/build/example.md b/docs/cmd/build/example.md deleted file mode 100644 index da2f3b4..0000000 --- a/docs/cmd/build/example.md +++ /dev/null @@ -1,83 +0,0 @@ -# Build Examples - -## Quick Start - -```bash -# Auto-detect and build -core build - -# Build for specific platforms -core build --targets linux/amd64,darwin/arm64 - -# CI mode -core build --ci -``` - -## Configuration - -`.core/build.yaml`: - -```yaml -version: 1 - -project: - name: myapp - binary: myapp - -build: - main: ./cmd/myapp - ldflags: - - -s -w - - -X main.version={{.Version}} - -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: arm64 -``` - -## Cross-Platform Build - -```bash -core build --targets linux/amd64,linux/arm64,darwin/arm64,windows/amd64 -``` - -Output: -``` -dist/ -├── myapp-linux-amd64.tar.gz -├── myapp-linux-arm64.tar.gz -├── myapp-darwin-arm64.tar.gz -├── myapp-windows-amd64.zip -└── CHECKSUMS.txt -``` - -## Code Signing - -```yaml -sign: - enabled: true - gpg: - key: $GPG_KEY_ID - macos: - identity: "Developer ID Application: Your Name (TEAM_ID)" - notarize: true - apple_id: $APPLE_ID - team_id: $APPLE_TEAM_ID - app_password: $APPLE_APP_PASSWORD -``` - -## Docker Build - -```bash -core build --type docker --image ghcr.io/myorg/myapp -``` - -## Wails Desktop App - -```bash -core build --type wails --targets darwin/arm64,windows/amd64 -``` diff --git a/docs/cmd/build/index.md b/docs/cmd/build/index.md deleted file mode 100644 index 6956e65..0000000 --- a/docs/cmd/build/index.md +++ /dev/null @@ -1,176 +0,0 @@ -# core build - -Build Go, Wails, Docker, and LinuxKit projects with automatic project detection. - -## Subcommands - -| Command | Description | -|---------|-------------| -| [sdk](sdk/) | Generate API SDKs from OpenAPI | -| `from-path` | Build from a local directory | -| `pwa` | Build from a live PWA URL | - -## Usage - -```bash -core build [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--type` | Project type: `go`, `wails`, `docker`, `linuxkit`, `taskfile` (auto-detected) | -| `--targets` | Build targets: `linux/amd64,darwin/arm64,windows/amd64` | -| `--output` | Output directory (default: `dist`) | -| `--ci` | CI mode - minimal output with JSON artifact list at the end | -| `--image` | Docker image name (for docker builds) | -| `--config` | Config file path (for linuxkit: YAML config, for docker: Dockerfile) | -| `--format` | Output format for linuxkit (iso-bios, qcow2-bios, raw, vmdk) | -| `--push` | Push Docker image after build (default: false) | -| `--archive` | Create archives (tar.gz for linux/darwin, zip for windows) - default: true | -| `--checksum` | Generate SHA256 checksums and CHECKSUMS.txt - default: true | -| `--no-sign` | Skip all code signing | -| `--notarize` | Enable macOS notarization (requires Apple credentials) | - -## Examples - -### Go Project - -```bash -# Auto-detect and build -core build - -# Build for specific platforms -core build --targets linux/amd64,linux/arm64,darwin/arm64 - -# CI mode -core build --ci -``` - -### Wails Project - -```bash -# Build Wails desktop app -core build --type wails - -# Build for all desktop platforms -core build --type wails --targets darwin/amd64,darwin/arm64,windows/amd64,linux/amd64 -``` - -### Docker Image - -```bash -# Build Docker image -core build --type docker - -# With custom image name -core build --type docker --image ghcr.io/myorg/myapp - -# Build and push to registry -core build --type docker --image ghcr.io/myorg/myapp --push -``` - -### LinuxKit Image - -```bash -# Build LinuxKit ISO -core build --type linuxkit - -# Build with specific format -core build --type linuxkit --config linuxkit.yml --format qcow2-bios -``` - -## Project Detection - -Core automatically detects project type based on files: - -| Files | Type | -|-------|------| -| `wails.json` | Wails | -| `go.mod` | Go | -| `Dockerfile` | Docker | -| `Taskfile.yml` | Taskfile | -| `composer.json` | PHP | -| `package.json` | Node | - -## Output - -Build artifacts are placed in `dist/` by default: - -``` -dist/ -├── myapp-linux-amd64.tar.gz -├── myapp-linux-arm64.tar.gz -├── myapp-darwin-amd64.tar.gz -├── myapp-darwin-arm64.tar.gz -├── myapp-windows-amd64.zip -└── CHECKSUMS.txt -``` - -## Configuration - -Optional `.core/build.yaml` - see [Configuration](example.md#configuration) for examples. - -## Code Signing - -Core supports GPG signing for checksums and native code signing for macOS. - -### GPG Signing - -Signs `CHECKSUMS.txt` with a detached ASCII signature (`.asc`): - -```bash -# Build with GPG signing (default if key configured) -core build - -# Skip signing -core build --no-sign -``` - -Users can verify: - -```bash -gpg --verify CHECKSUMS.txt.asc CHECKSUMS.txt -sha256sum -c CHECKSUMS.txt -``` - -### macOS Code Signing - -Signs Darwin binaries with your Developer ID and optionally notarizes with Apple: - -```bash -# Build with codesign (automatic if identity configured) -core build - -# Build with notarization (takes 1-5 minutes) -core build --notarize -``` - -### Environment Variables - -| Variable | Purpose | -|----------|---------| -| `GPG_KEY_ID` | GPG key ID or fingerprint | -| `CODESIGN_IDENTITY` | macOS Developer ID (fallback) | -| `APPLE_ID` | Apple account email | -| `APPLE_TEAM_ID` | Apple Developer Team ID | -| `APPLE_APP_PASSWORD` | App-specific password for notarization | - -## Building from PWAs and Static Sites - -### Build from Local Directory - -Build a desktop app from static web application files: - -```bash -core build from-path --path ./dist -``` - -### Build from Live PWA - -Build a desktop app from a live Progressive Web App URL: - -```bash -core build pwa --url https://example.com -``` diff --git a/docs/cmd/build/sdk/example.md b/docs/cmd/build/sdk/example.md deleted file mode 100644 index e832308..0000000 --- a/docs/cmd/build/sdk/example.md +++ /dev/null @@ -1,56 +0,0 @@ -# SDK Build Examples - -## Generate All SDKs - -```bash -core build sdk -``` - -## Specific Language - -```bash -core build sdk --lang typescript -core build sdk --lang php -core build sdk --lang go -``` - -## Custom Spec - -```bash -core build sdk --spec ./api/openapi.yaml -``` - -## With Version - -```bash -core build sdk --version v2.0.0 -``` - -## Preview - -```bash -core build sdk --dry-run -``` - -## Configuration - -`.core/sdk.yaml`: - -```yaml -version: 1 - -spec: ./api/openapi.yaml - -languages: - - name: typescript - output: sdk/typescript - package: "@myorg/api-client" - - - name: php - output: sdk/php - namespace: MyOrg\ApiClient - - - name: go - output: sdk/go - module: github.com/myorg/api-client-go -``` diff --git a/docs/cmd/build/sdk/index.md b/docs/cmd/build/sdk/index.md deleted file mode 100644 index 084c5ef..0000000 --- a/docs/cmd/build/sdk/index.md +++ /dev/null @@ -1,27 +0,0 @@ -# core build sdk - -Generate typed API clients from OpenAPI specifications. Supports TypeScript, Python, Go, and PHP. - -## Usage - -```bash -core build sdk [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--spec` | Path to OpenAPI spec file | -| `--lang` | Generate only this language (typescript, python, go, php) | -| `--version` | Version to embed in generated SDKs | -| `--dry-run` | Show what would be generated without writing files | - -## Examples - -```bash -core build sdk # Generate all -core build sdk --lang typescript # TypeScript only -core build sdk --spec ./api.yaml # Custom spec -core build sdk --dry-run # Preview -``` diff --git a/docs/cmd/ci/changelog/example.md b/docs/cmd/ci/changelog/example.md deleted file mode 100644 index 101cad7..0000000 --- a/docs/cmd/ci/changelog/example.md +++ /dev/null @@ -1,36 +0,0 @@ -# CI Changelog Examples - -```bash -core ci changelog -``` - -## Output - -```markdown -## v1.2.0 - -### Features -- Add user authentication (#123) -- Support dark mode (#124) - -### Bug Fixes -- Fix memory leak in worker (#125) - -### Performance -- Optimize database queries (#126) -``` - -## Configuration - -`.core/release.yaml`: - -```yaml -changelog: - include: - - feat - - fix - - perf - exclude: - - chore - - docs -``` diff --git a/docs/cmd/ci/changelog/index.md b/docs/cmd/ci/changelog/index.md deleted file mode 100644 index ffc0712..0000000 --- a/docs/cmd/ci/changelog/index.md +++ /dev/null @@ -1,28 +0,0 @@ -# core ci changelog - -Generate changelog from conventional commits. - -## Usage - -```bash -core ci changelog -``` - -## Output - -Generates markdown changelog from git commits since last tag: - -```markdown -## v1.2.0 - -### Features -- Add user authentication (#123) -- Support dark mode (#124) - -### Bug Fixes -- Fix memory leak in worker (#125) -``` - -## Configuration - -See [configuration.md](../../../configuration.md) for changelog configuration options. diff --git a/docs/cmd/ci/example.md b/docs/cmd/ci/example.md deleted file mode 100644 index faf4720..0000000 --- a/docs/cmd/ci/example.md +++ /dev/null @@ -1,90 +0,0 @@ -# CI Examples - -## Quick Start - -```bash -# Build first -core build - -# Preview release -core ci - -# Publish -core ci --we-are-go-for-launch -``` - -## Configuration - -`.core/release.yaml`: - -```yaml -version: 1 - -project: - name: myapp - repository: host-uk/myapp - -publishers: - - type: github -``` - -## Publisher Examples - -### GitHub + Docker - -```yaml -publishers: - - type: github - - - type: docker - registry: ghcr.io - image: host-uk/myapp - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" -``` - -### Full Stack (GitHub + npm + Homebrew) - -```yaml -publishers: - - type: github - - - type: npm - package: "@host-uk/myapp" - access: public - - - type: homebrew - tap: host-uk/homebrew-tap -``` - -### LinuxKit Image - -```yaml -publishers: - - type: linuxkit - config: .core/linuxkit/server.yml - formats: - - iso - - qcow2 - platforms: - - linux/amd64 - - linux/arm64 -``` - -## Changelog Configuration - -```yaml -changelog: - include: - - feat - - fix - - perf - exclude: - - chore - - docs - - test -``` diff --git a/docs/cmd/ci/index.md b/docs/cmd/ci/index.md deleted file mode 100644 index ee2c759..0000000 --- a/docs/cmd/ci/index.md +++ /dev/null @@ -1,79 +0,0 @@ -# core ci - -Publish releases to GitHub, Docker, npm, Homebrew, and more. - -**Safety:** Dry-run by default. Use `--we-are-go-for-launch` to actually publish. - -## Subcommands - -| Command | Description | -|---------|-------------| -| [init](init/) | Initialize release config | -| [changelog](changelog/) | Generate changelog | -| [version](version/) | Show determined version | - -## Usage - -```bash -core ci [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--we-are-go-for-launch` | Actually publish (default is dry-run) | -| `--version` | Override version | -| `--draft` | Create as draft release | -| `--prerelease` | Mark as prerelease | - -## Examples - -```bash -# Preview what would be published (safe) -core ci - -# Actually publish -core ci --we-are-go-for-launch - -# Publish as draft -core ci --we-are-go-for-launch --draft - -# Publish as prerelease -core ci --we-are-go-for-launch --prerelease -``` - -## Workflow - -Build and publish are **separated** to prevent accidents: - -```bash -# Step 1: Build artifacts -core build -core build sdk - -# Step 2: Preview (dry-run by default) -core ci - -# Step 3: Publish (explicit flag required) -core ci --we-are-go-for-launch -``` - -## Publishers - -See [Publisher Examples](example.md#publisher-examples) for configuration. - -| Type | Target | -|------|--------| -| `github` | GitHub Releases | -| `docker` | Container registries | -| `linuxkit` | LinuxKit images | -| `npm` | npm registry | -| `homebrew` | Homebrew tap | -| `scoop` | Scoop bucket | -| `aur` | Arch User Repository | -| `chocolatey` | Chocolatey | - -## Changelog - -Auto-generated from conventional commits. See [Changelog Configuration](example.md#changelog-configuration). diff --git a/docs/cmd/ci/init/example.md b/docs/cmd/ci/init/example.md deleted file mode 100644 index 8f76ab9..0000000 --- a/docs/cmd/ci/init/example.md +++ /dev/null @@ -1,17 +0,0 @@ -# CI Init Examples - -```bash -core ci init -``` - -Creates `.core/release.yaml`: - -```yaml -version: 1 - -project: - name: myapp - -publishers: - - type: github -``` diff --git a/docs/cmd/ci/init/index.md b/docs/cmd/ci/init/index.md deleted file mode 100644 index 23ba068..0000000 --- a/docs/cmd/ci/init/index.md +++ /dev/null @@ -1,11 +0,0 @@ -# core ci init - -Initialize release configuration. - -## Usage - -```bash -core ci init -``` - -Creates `.core/release.yaml` with default configuration. See [Configuration](../example.md#configuration) for output format. diff --git a/docs/cmd/ci/version/example.md b/docs/cmd/ci/version/example.md deleted file mode 100644 index e669d65..0000000 --- a/docs/cmd/ci/version/example.md +++ /dev/null @@ -1,18 +0,0 @@ -# CI Version Examples - -```bash -core ci version -``` - -## Output - -``` -v1.2.0 -``` - -## Version Resolution - -1. `--version` flag (if provided) -2. Git tag on HEAD -3. Latest git tag + increment -4. `v0.0.1` (no tags) diff --git a/docs/cmd/ci/version/index.md b/docs/cmd/ci/version/index.md deleted file mode 100644 index 7014a34..0000000 --- a/docs/cmd/ci/version/index.md +++ /dev/null @@ -1,21 +0,0 @@ -# core ci version - -Show the determined release version. - -## Usage - -```bash -core ci version -``` - -## Output - -``` -v1.2.0 -``` - -Version is determined from: -1. `--version` flag (if provided) -2. Git tag on HEAD -3. Latest git tag + increment -4. `v0.0.1` (if no tags exist) diff --git a/docs/cmd/dev/ci/index.md b/docs/cmd/dev/ci/index.md deleted file mode 100644 index 0cf8442..0000000 --- a/docs/cmd/dev/ci/index.md +++ /dev/null @@ -1,61 +0,0 @@ -# core dev ci - -Check CI status across all repositories. - -Fetches GitHub Actions workflow status for all repos. Shows latest run status for each repo. Requires the `gh` CLI to be installed and authenticated. - -## Usage - -```bash -core dev ci [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--branch` | Filter by branch (default: main) | -| `--failed` | Show only failed runs | - -## Examples - -```bash -# Check CI status for all repos -core dev ci - -# Check specific branch -core dev ci --branch develop - -# Show only failures -core dev ci --failed -``` - -## Output - -``` -core-php ✓ passing 2m ago -core-tenant ✓ passing 5m ago -core-admin ✗ failed 12m ago -core-api ⏳ running now -core-bio ✓ passing 1h ago -``` - -## Status Icons - -| Symbol | Meaning | -|--------|---------| -| `✓` | Passing | -| `✗` | Failed | -| `⏳` | Running | -| `-` | No runs | - -## Requirements - -- GitHub CLI (`gh`) must be installed -- Must be authenticated: `gh auth login` - -## See Also - -- [issues command](../issues/) - List open issues -- [reviews command](../reviews/) - List PRs needing review diff --git a/docs/cmd/dev/commit/index.md b/docs/cmd/dev/commit/index.md deleted file mode 100644 index 4258fb1..0000000 --- a/docs/cmd/dev/commit/index.md +++ /dev/null @@ -1,46 +0,0 @@ -# core dev commit - -Claude-assisted commits across repositories. - -Uses Claude to create commits for dirty repos. Shows uncommitted changes and invokes Claude to generate commit messages. - -## Usage - -```bash -core dev commit [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--all` | Commit all dirty repos without prompting | - -## Examples - -```bash -# Interactive commit (prompts for each repo) -core dev commit - -# Commit all dirty repos automatically -core dev commit --all - -# Use specific registry -core dev commit --registry ~/projects/repos.yaml -``` - -## How It Works - -1. Scans all repositories for uncommitted changes -2. For each dirty repo: - - Shows the diff - - Invokes Claude to generate a commit message - - Creates the commit with `Co-Authored-By: Claude` -3. Reports success/failure for each repo - -## See Also - -- [health command](../health/) - Check repo status -- [push command](../push/) - Push commits after committing -- [work command](../work/) - Full workflow (status + commit + push) diff --git a/docs/cmd/dev/example.md b/docs/cmd/dev/example.md deleted file mode 100644 index da75b5e..0000000 --- a/docs/cmd/dev/example.md +++ /dev/null @@ -1,203 +0,0 @@ -# Dev Examples - -## Multi-Repo Workflow - -```bash -# Quick status -core dev health - -# Detailed breakdown -core dev health --verbose - -# Full workflow -core dev work - -# Status only -core dev work --status - -# Commit and push -core dev work --commit - -# Commit dirty repos -core dev commit - -# Commit all without prompting -core dev commit --all - -# Push unpushed -core dev push - -# Push without confirmation -core dev push --force - -# Pull behind repos -core dev pull - -# Pull all repos -core dev pull --all -``` - -## GitHub Integration - -```bash -# Open issues -core dev issues - -# Filter by assignee -core dev issues --assignee @me - -# Limit results -core dev issues --limit 5 - -# PRs needing review -core dev reviews - -# All PRs including drafts -core dev reviews --all - -# Filter by author -core dev reviews --author username - -# CI status -core dev ci - -# Only failed runs -core dev ci --failed - -# Specific branch -core dev ci --branch develop -``` - -## Dependency Analysis - -```bash -# What depends on core-php? -core dev impact core-php -``` - -## Task Management - -```bash -# List tasks -core ai tasks - -# Filter by status and priority -core ai tasks --status pending --priority high - -# Filter by labels -core ai tasks --labels bug,urgent - -# Show task details -core ai task abc123 - -# Auto-select highest priority task -core ai task --auto - -# Claim a task -core ai task abc123 --claim - -# Update task status -core ai task:update abc123 --status in_progress - -# Add progress notes -core ai task:update abc123 --progress 50 --notes 'Halfway done' - -# Complete a task -core ai task:complete abc123 --output 'Feature implemented' - -# Mark as failed -core ai task:complete abc123 --failed --error 'Build failed' - -# Commit with task reference -core ai task:commit abc123 -m 'add user authentication' - -# Commit with scope and push -core ai task:commit abc123 -m 'fix login bug' --scope auth --push - -# Create PR for task -core ai task:pr abc123 - -# Create draft PR with labels -core ai task:pr abc123 --draft --labels 'enhancement,needs-review' -``` - -## Service API Management - -```bash -# Synchronize public service APIs -core dev sync - -# Or using the api command -core dev api sync -``` - -## Dev Environment - -```bash -# First time setup -core dev install -core dev boot - -# Open shell -core dev shell - -# Mount and serve -core dev serve - -# Run tests -core dev test - -# Sandboxed Claude -core dev claude -``` - -## Configuration - -### repos.yaml - -```yaml -org: host-uk -repos: - core-php: - type: package - description: Foundation framework - core-tenant: - type: package - depends: [core-php] -``` - -### ~/.core/config.yaml - -```yaml -version: 1 - -images: - source: auto # auto | github | registry | cdn - - cdn: - url: https://images.example.com/core-devops - - github: - repo: host-uk/core-images - - registry: - image: ghcr.io/host-uk/core-devops -``` - -### .core/test.yaml - -```yaml -version: 1 - -commands: - - name: unit - run: vendor/bin/pest --parallel - - name: types - run: vendor/bin/phpstan analyse - - name: lint - run: vendor/bin/pint --test - -env: - APP_ENV: testing - DB_CONNECTION: sqlite -``` diff --git a/docs/cmd/dev/health/index.md b/docs/cmd/dev/health/index.md deleted file mode 100644 index d104689..0000000 --- a/docs/cmd/dev/health/index.md +++ /dev/null @@ -1,52 +0,0 @@ -# core dev health - -Quick health check across all repositories. - -Shows a summary of repository health: total repos, dirty repos, unpushed commits, etc. - -## Usage - -```bash -core dev health [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--verbose` | Show detailed breakdown | - -## Examples - -```bash -# Quick health summary -core dev health - -# Detailed breakdown -core dev health --verbose - -# Use specific registry -core dev health --registry ~/projects/repos.yaml -``` - -## Output - -``` -18 repos │ 2 dirty │ 1 ahead │ all synced -``` - -With `--verbose`: - -``` -Repos: 18 -Dirty: 2 (core-php, core-admin) -Ahead: 1 (core-tenant) -Behind: 0 -Synced: ✓ -``` - -## See Also - -- [work command](../work/) - Full workflow (status + commit + push) -- [commit command](../commit/) - Claude-assisted commits diff --git a/docs/cmd/dev/impact/index.md b/docs/cmd/dev/impact/index.md deleted file mode 100644 index ac96e04..0000000 --- a/docs/cmd/dev/impact/index.md +++ /dev/null @@ -1,65 +0,0 @@ -# core dev impact - -Show impact of changing a repository. - -Analyses the dependency graph to show which repos would be affected by changes to the specified repo. - -## Usage - -```bash -core dev impact [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | - -## Examples - -```bash -# Show what depends on core-php -core dev impact core-php - -# Show what depends on core-tenant -core dev impact core-tenant -``` - -## Output - -``` -Impact of changes to core-php: - -Direct dependents (5): - core-tenant - core-admin - core-api - core-mcp - core-commerce - -Indirect dependents (12): - core-bio (via core-tenant) - core-social (via core-tenant) - core-analytics (via core-tenant) - core-notify (via core-tenant) - core-trust (via core-tenant) - core-support (via core-tenant) - core-content (via core-tenant) - core-developer (via core-tenant) - core-agentic (via core-mcp) - ... - -Total: 17 repos affected -``` - -## Use Cases - -- Before making breaking changes, see what needs updating -- Plan release order based on dependency graph -- Understand the ripple effect of changes - -## See Also - -- [health command](../health/) - Quick repo status -- [setup command](../../setup/) - Clone repos with dependencies diff --git a/docs/cmd/dev/index.md b/docs/cmd/dev/index.md deleted file mode 100644 index 56a5090..0000000 --- a/docs/cmd/dev/index.md +++ /dev/null @@ -1,388 +0,0 @@ -# core dev - -Multi-repo workflow and portable development environment. - -## Multi-Repo Commands - -| Command | Description | -|---------|-------------| -| [work](work/) | Full workflow: status + commit + push | -| `health` | Quick health check across repos | -| `commit` | Claude-assisted commits | -| `push` | Push repos with unpushed commits | -| `pull` | Pull repos that are behind | -| `issues` | List open issues | -| `reviews` | List PRs needing review | -| `ci` | Check CI status | -| `impact` | Show dependency impact | -| `api` | Tools for managing service APIs | -| `sync` | Synchronize public service APIs | - -## Task Management Commands - -> **Note:** Task management commands have moved to [`core ai`](../ai/). - -| Command | Description | -|---------|-------------| -| [`ai tasks`](../ai/) | List available tasks from core-agentic | -| [`ai task`](../ai/) | Show task details or auto-select a task | -| [`ai task:update`](../ai/) | Update task status or progress | -| [`ai task:complete`](../ai/) | Mark a task as completed | -| [`ai task:commit`](../ai/) | Auto-commit changes with task reference | -| [`ai task:pr`](../ai/) | Create a pull request for a task | - -## Dev Environment Commands - -| Command | Description | -|---------|-------------| -| `install` | Download the core-devops image | -| `boot` | Start the environment | -| `stop` | Stop the environment | -| `status` | Show status | -| `shell` | Open shell | -| `serve` | Start dev server | -| `test` | Run tests | -| `claude` | Sandboxed Claude | -| `update` | Update image | - ---- - -## Dev Environment Overview - -Core DevOps provides a sandboxed, immutable development environment based on LinuxKit with 100+ embedded tools. - -## Quick Start - -```bash -# First time setup -core dev install -core dev boot - -# Open shell -core dev shell - -# Or mount current project and serve -core dev serve -``` - -## dev install - -Download the core-devops image for your platform. - -```bash -core dev install -``` - -Downloads the platform-specific dev environment image including Go, PHP, Node.js, Python, Docker, and Claude CLI. Downloads are cached at `~/.core/images/`. - -### Examples - -```bash -# Download image (auto-detects platform) -core dev install -``` - -## dev boot - -Start the development environment. - -```bash -core dev boot [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--memory` | Memory allocation in MB (default: 4096) | -| `--cpus` | Number of CPUs (default: 2) | -| `--fresh` | Stop existing and start fresh | - -### Examples - -```bash -# Start with defaults -core dev boot - -# More resources -core dev boot --memory 8192 --cpus 4 - -# Fresh start -core dev boot --fresh -``` - -## dev shell - -Open a shell in the running environment. - -```bash -core dev shell [flags] [-- command] -``` - -Uses SSH by default, or serial console with `--console`. - -### Flags - -| Flag | Description | -|------|-------------| -| `--console` | Use serial console instead of SSH | - -### Examples - -```bash -# SSH into environment -core dev shell - -# Serial console (for debugging) -core dev shell --console - -# Run a command -core dev shell -- ls -la -``` - -## dev serve - -Mount current directory and start the appropriate dev server. - -```bash -core dev serve [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--port` | Port to expose (default: 8000) | -| `--path` | Subdirectory to serve | - -### Auto-Detection - -| Project | Server Command | -|---------|---------------| -| Laravel (`artisan`) | `php artisan octane:start` | -| Node (`package.json` with `dev` script) | `npm run dev` | -| PHP (`composer.json`) | `frankenphp php-server` | -| Other | `python -m http.server` | - -### Examples - -```bash -# Auto-detect and serve -core dev serve - -# Custom port -core dev serve --port 3000 -``` - -## dev test - -Run tests inside the environment. - -```bash -core dev test [flags] [-- custom command] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--name` | Run named test command from `.core/test.yaml` | - -### Test Detection - -Core auto-detects the test framework or uses `.core/test.yaml`: - -1. `.core/test.yaml` - Custom config -2. `composer.json` → `composer test` -3. `package.json` → `npm test` -4. `go.mod` → `go test ./...` -5. `pytest.ini` → `pytest` -6. `Taskfile.yaml` → `task test` - -### Examples - -```bash -# Auto-detect and run tests -core dev test - -# Run named test from config -core dev test --name integration - -# Custom command -core dev test -- go test -v ./pkg/... -``` - -### Test Configuration - -Create `.core/test.yaml` for custom test setup - see [Configuration](example.md#configuration) for examples. - -## dev claude - -Start a sandboxed Claude session with your project mounted. - -```bash -core dev claude [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--model` | Model to use (`opus`, `sonnet`) | -| `--no-auth` | Don't forward any auth credentials | -| `--auth` | Selective auth forwarding (`gh`, `anthropic`, `ssh`, `git`) | - -### What Gets Forwarded - -By default, these are forwarded to the sandbox: -- `~/.anthropic/` or `ANTHROPIC_API_KEY` -- `~/.config/gh/` (GitHub CLI auth) -- SSH agent -- Git config (name, email) - -### Examples - -```bash -# Full auth forwarding (default) -core dev claude - -# Use Opus model -core dev claude --model opus - -# Clean sandbox -core dev claude --no-auth - -# Only GitHub and Anthropic auth -core dev claude --auth gh,anthropic -``` - -### Why Use This? - -- **Immutable base** - Reset anytime with `core dev boot --fresh` -- **Safe experimentation** - Claude can install packages, make mistakes -- **Host system untouched** - All changes stay in the sandbox -- **Real credentials** - Can still push code, create PRs -- **Full tooling** - 100+ tools available in the image - -## dev status - -Show the current state of the development environment. - -```bash -core dev status -``` - -Output includes: -- Running/stopped state -- Resource usage (CPU, memory) -- Exposed ports -- Mounted directories - -## dev update - -Check for and apply updates. - -```bash -core dev update [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--apply` | Download and apply the update | - -### Examples - -```bash -# Check for updates -core dev update - -# Apply available update -core dev update --apply -``` - -## Embedded Tools - -The core-devops image includes 100+ tools: - -| Category | Tools | -|----------|-------| -| **AI/LLM** | claude, gemini, aider, ollama, llm | -| **VCS** | git, gh, glab, lazygit, delta, git-lfs | -| **Runtimes** | frankenphp, node, bun, deno, go, python3, rustc | -| **Package Mgrs** | composer, npm, pnpm, yarn, pip, uv, cargo | -| **Build** | task, make, just, nx, turbo | -| **Linting** | pint, phpstan, prettier, eslint, biome, golangci-lint, ruff | -| **Testing** | phpunit, pest, vitest, playwright, k6 | -| **Infra** | docker, kubectl, k9s, helm, terraform, ansible | -| **Databases** | sqlite3, mysql, psql, redis-cli, mongosh, usql | -| **HTTP/Net** | curl, httpie, xh, websocat, grpcurl, mkcert, ngrok | -| **Data** | jq, yq, fx, gron, miller, dasel | -| **Security** | age, sops, cosign, trivy, trufflehog, vault | -| **Files** | fd, rg, fzf, bat, eza, tree, zoxide, broot | -| **Editors** | nvim, helix, micro | - -## Configuration - -Global config in `~/.core/config.yaml` - see [Configuration](example.md#configuration) for examples. - -## Image Storage - -Images are stored in `~/.core/images/`: - -``` -~/.core/ -├── config.yaml -└── images/ - ├── core-devops-darwin-arm64.qcow2 - ├── core-devops-linux-amd64.qcow2 - └── manifest.json -``` - -## Multi-Repo Commands - -See the [work](work/) page for detailed documentation on multi-repo commands. - -### dev ci - -Check GitHub Actions workflow status across all repos. - -```bash -core dev ci [flags] -``` - -#### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--branch` | Filter by branch (default: main) | -| `--failed` | Show only failed runs | - -Requires the `gh` CLI to be installed and authenticated. - -### dev api - -Tools for managing service APIs. - -```bash -core dev api sync -``` - -Synchronizes the public service APIs with their internal implementations. - -### dev sync - -Alias for `core dev api sync`. Synchronizes the public service APIs with their internal implementations. - -```bash -core dev sync -``` - -This command scans the `pkg` directory for services and ensures that the top-level public API for each service is in sync with its internal implementation. It automatically generates the necessary Go files with type aliases. - -## See Also - -- [work](work/) - Multi-repo workflow commands (`core dev work`, `core dev health`, etc.) -- [ai](../ai/) - Task management commands (`core ai tasks`, `core ai task`, etc.) diff --git a/docs/cmd/dev/issues/index.md b/docs/cmd/dev/issues/index.md deleted file mode 100644 index 36091eb..0000000 --- a/docs/cmd/dev/issues/index.md +++ /dev/null @@ -1,57 +0,0 @@ -# core dev issues - -List open issues across all repositories. - -Fetches open issues from GitHub for all repos in the registry. Requires the `gh` CLI to be installed and authenticated. - -## Usage - -```bash -core dev issues [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--assignee` | Filter by assignee (use `@me` for yourself) | -| `--limit` | Max issues per repo (default 10) | - -## Examples - -```bash -# List all open issues -core dev issues - -# Show issues assigned to you -core dev issues --assignee @me - -# Limit to 5 issues per repo -core dev issues --limit 5 - -# Filter by specific assignee -core dev issues --assignee username -``` - -## Output - -``` -core-php (3 issues) - #42 Add retry logic to HTTP client bug - #38 Update documentation for v2 API docs - #35 Support custom serializers enhancement - -core-tenant (1 issue) - #12 Workspace isolation bug bug, critical -``` - -## Requirements - -- GitHub CLI (`gh`) must be installed -- Must be authenticated: `gh auth login` - -## See Also - -- [reviews command](../reviews/) - List PRs needing review -- [ci command](../ci/) - Check CI status diff --git a/docs/cmd/dev/pull/index.md b/docs/cmd/dev/pull/index.md deleted file mode 100644 index 1f6f3df..0000000 --- a/docs/cmd/dev/pull/index.md +++ /dev/null @@ -1,47 +0,0 @@ -# core dev pull - -Pull updates across all repositories. - -Pulls updates for all repos. By default only pulls repos that are behind. Use `--all` to pull all repos. - -## Usage - -```bash -core dev pull [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--all` | Pull all repos, not just those behind | - -## Examples - -```bash -# Pull only repos that are behind -core dev pull - -# Pull all repos -core dev pull --all - -# Use specific registry -core dev pull --registry ~/projects/repos.yaml -``` - -## Output - -``` -Pulling 2 repo(s) that are behind: - ✓ core-php (3 commits) - ✓ core-tenant (1 commit) - -Done: 2 pulled -``` - -## See Also - -- [push command](../push/) - Push local commits -- [health command](../health/) - Check sync status -- [work command](../work/) - Full workflow diff --git a/docs/cmd/dev/push/index.md b/docs/cmd/dev/push/index.md deleted file mode 100644 index 0c11195..0000000 --- a/docs/cmd/dev/push/index.md +++ /dev/null @@ -1,52 +0,0 @@ -# core dev push - -Push commits across all repositories. - -Pushes unpushed commits for all repos. Shows repos with commits to push and confirms before pushing. - -## Usage - -```bash -core dev push [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--force` | Skip confirmation prompt | - -## Examples - -```bash -# Push with confirmation -core dev push - -# Push without confirmation -core dev push --force - -# Use specific registry -core dev push --registry ~/projects/repos.yaml -``` - -## Output - -``` -3 repo(s) with unpushed commits: - core-php: 2 commit(s) - core-admin: 1 commit(s) - core-tenant: 1 commit(s) - -Push all? [y/N] y - - ✓ core-php - ✓ core-admin - ✓ core-tenant -``` - -## See Also - -- [commit command](../commit/) - Create commits before pushing -- [pull command](../pull/) - Pull updates from remote -- [work command](../work/) - Full workflow (status + commit + push) diff --git a/docs/cmd/dev/reviews/index.md b/docs/cmd/dev/reviews/index.md deleted file mode 100644 index 44c09ad..0000000 --- a/docs/cmd/dev/reviews/index.md +++ /dev/null @@ -1,61 +0,0 @@ -# core dev reviews - -List PRs needing review across all repositories. - -Fetches open PRs from GitHub for all repos in the registry. Shows review status (approved, changes requested, pending). Requires the `gh` CLI to be installed and authenticated. - -## Usage - -```bash -core dev reviews [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--all` | Show all PRs including drafts | -| `--author` | Filter by PR author | - -## Examples - -```bash -# List PRs needing review -core dev reviews - -# Include draft PRs -core dev reviews --all - -# Filter by author -core dev reviews --author username -``` - -## Output - -``` -core-php (2 PRs) - #45 feat: Add caching layer ✓ approved @alice - #43 fix: Memory leak in worker ⏳ pending @bob - -core-admin (1 PR) - #28 refactor: Extract components ✗ changes @charlie -``` - -## Review Status - -| Symbol | Meaning | -|--------|---------| -| `✓` | Approved | -| `⏳` | Pending review | -| `✗` | Changes requested | - -## Requirements - -- GitHub CLI (`gh`) must be installed -- Must be authenticated: `gh auth login` - -## See Also - -- [issues command](../issues/) - List open issues -- [ci command](../ci/) - Check CI status diff --git a/docs/cmd/dev/work/example.md b/docs/cmd/dev/work/example.md deleted file mode 100644 index 4d10470..0000000 --- a/docs/cmd/dev/work/example.md +++ /dev/null @@ -1,33 +0,0 @@ -# Dev Work Examples - -```bash -# Full workflow: status → commit → push -core dev work - -# Status only -core dev work --status -``` - -## Output - -``` -┌─────────────┬────────┬──────────┬─────────┐ -│ Repo │ Branch │ Status │ Behind │ -├─────────────┼────────┼──────────┼─────────┤ -│ core-php │ main │ clean │ 0 │ -│ core-tenant │ main │ 2 files │ 0 │ -│ core-admin │ dev │ clean │ 3 │ -└─────────────┴────────┴──────────┴─────────┘ -``` - -## Registry - -```yaml -repos: - - name: core - path: ./core - url: https://forge.lthn.ai/core/cli - - name: core-php - path: ./core-php - url: https://forge.lthn.ai/core/cli-php -``` diff --git a/docs/cmd/dev/work/index.md b/docs/cmd/dev/work/index.md deleted file mode 100644 index 454fe22..0000000 --- a/docs/cmd/dev/work/index.md +++ /dev/null @@ -1,293 +0,0 @@ -# core dev work - -Multi-repo git operations for managing the host-uk organization. - -## Overview - -The `core dev work` command and related subcommands help manage multiple repositories in the host-uk ecosystem simultaneously. - -## Commands - -| Command | Description | -|---------|-------------| -| `core dev work` | Full workflow: status + commit + push | -| `core dev work --status` | Status table only | -| `core dev work --commit` | Use Claude to commit dirty repos | -| `core dev health` | Quick health check across all repos | -| `core dev commit` | Claude-assisted commits across repos | -| `core dev push` | Push commits across all repos | -| `core dev pull` | Pull updates across all repos | -| `core dev issues` | List open issues across all repos | -| `core dev reviews` | List PRs needing review | -| `core dev ci` | Check CI status across all repos | -| `core dev impact` | Show impact of changing a repo | - -## core dev work - -Manage git status, commits, and pushes across multiple repositories. - -```bash -core dev work [flags] -``` - -Reads `repos.yaml` to discover repositories and their relationships. Shows status, optionally commits with Claude, and pushes changes. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--status` | Show status only, don't push | -| `--commit` | Use Claude to commit dirty repos before pushing | - -### Examples - -```bash -# Full workflow -core dev work - -# Status only -core dev work --status - -# Commit and push -core dev work --commit -``` - -## core dev health - -Quick health check showing summary of repository health across all repos. - -```bash -core dev health [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--verbose` | Show detailed breakdown | - -Output shows: -- Total repos -- Dirty repos -- Unpushed commits -- Repos behind remote - -### Examples - -```bash -# Quick summary -core dev health - -# Detailed breakdown -core dev health --verbose -``` - -## core dev issues - -List open issues across all repositories. - -```bash -core dev issues [flags] -``` - -Fetches open issues from GitHub for all repos in the registry. Requires the `gh` CLI to be installed and authenticated. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--assignee` | Filter by assignee (use `@me` for yourself) | -| `--limit` | Max issues per repo (default: 10) | - -### Examples - -```bash -# List all open issues -core dev issues - -# Filter by assignee -core dev issues --assignee @me - -# Limit results -core dev issues --limit 5 -``` - -## core dev reviews - -List pull requests needing review across all repos. - -```bash -core dev reviews [flags] -``` - -Fetches open PRs from GitHub for all repos in the registry. Shows review status (approved, changes requested, pending). Requires the `gh` CLI to be installed and authenticated. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--all` | Show all PRs including drafts | -| `--author` | Filter by PR author | - -### Examples - -```bash -# List PRs needing review -core dev reviews - -# Show all PRs including drafts -core dev reviews --all - -# Filter by author -core dev reviews --author username -``` - -## core dev commit - -Create commits across repos with Claude assistance. - -```bash -core dev commit [flags] -``` - -Uses Claude to create commits for dirty repos. Shows uncommitted changes and invokes Claude to generate commit messages. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--all` | Commit all dirty repos without prompting | - -### Examples - -```bash -# Commit with prompts -core dev commit - -# Commit all automatically -core dev commit --all -``` - -## core dev push - -Push commits across all repos. - -```bash -core dev push [flags] -``` - -Pushes unpushed commits for all repos. Shows repos with commits to push and confirms before pushing. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--force` | Skip confirmation prompt | - -### Examples - -```bash -# Push with confirmation -core dev push - -# Skip confirmation -core dev push --force -``` - -## core dev pull - -Pull updates across all repos. - -```bash -core dev pull [flags] -``` - -Pulls updates for all repos. By default only pulls repos that are behind. Use `--all` to pull all repos. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--all` | Pull all repos, not just those behind | - -### Examples - -```bash -# Pull repos that are behind -core dev pull - -# Pull all repos -core dev pull --all -``` - -## core dev ci - -Check GitHub Actions workflow status across all repos. - -```bash -core dev ci [flags] -``` - -Fetches GitHub Actions workflow status for all repos. Shows latest run status for each repo. Requires the `gh` CLI to be installed and authenticated. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | -| `--branch` | Filter by branch (default: main) | -| `--failed` | Show only failed runs | - -### Examples - -```bash -# Show CI status for all repos -core dev ci - -# Show only failed runs -core dev ci --failed - -# Check specific branch -core dev ci --branch develop -``` - -## core dev impact - -Show the impact of changing a repository. - -```bash -core dev impact [flags] -``` - -Analyzes the dependency graph to show which repos would be affected by changes to the specified repo. - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to `repos.yaml` (auto-detected if not specified) | - -### Examples - -```bash -# Show impact of changing core-php -core dev impact core-php -``` - -## Registry - -These commands use `repos.yaml` to know which repos to manage. See [repos.yaml](../../../configuration.md#reposyaml) for format. - -Use `core setup` to clone all repos from the registry. - -## See Also - -- [setup command](../../setup/) - Clone repos from registry -- [search command](../../pkg/search/) - Find and install repos diff --git a/docs/cmd/docs/example.md b/docs/cmd/docs/example.md deleted file mode 100644 index 7729970..0000000 --- a/docs/cmd/docs/example.md +++ /dev/null @@ -1,14 +0,0 @@ -# Docs Examples - -## List - -```bash -core docs list -``` - -## Sync - -```bash -core docs sync -core docs sync --output ./docs -``` diff --git a/docs/cmd/docs/index.md b/docs/cmd/docs/index.md deleted file mode 100644 index d73ebf0..0000000 --- a/docs/cmd/docs/index.md +++ /dev/null @@ -1,110 +0,0 @@ -# core docs - -Documentation management across repositories. - -## Usage - -```bash -core docs [flags] -``` - -## Commands - -| Command | Description | -|---------|-------------| -| `list` | List documentation across repos | -| `sync` | Sync documentation to output directory | - -## docs list - -Show documentation coverage across all repos. - -```bash -core docs list [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml | - -### Output - -``` -Repo README CLAUDE CHANGELOG docs/ -────────────────────────────────────────────────────────────────────── -core ✓ ✓ — 12 files -core-php ✓ ✓ ✓ 8 files -core-images ✓ — — — - -Coverage: 3 with docs, 0 without -``` - -## docs sync - -Sync documentation from all repos to an output directory. - -```bash -core docs sync [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml | -| `--output` | Output directory (default: ./docs-build) | -| `--dry-run` | Show what would be synced | - -### Output Structure - -``` -docs-build/ -└── packages/ - ├── core/ - │ ├── index.md # from README.md - │ ├── claude.md # from CLAUDE.md - │ ├── changelog.md # from CHANGELOG.md - │ ├── build.md # from docs/build.md - │ └── ... - └── core-php/ - ├── index.md - └── ... -``` - -### Example - -```bash -# Preview what will be synced -core docs sync --dry-run - -# Sync to default output -core docs sync - -# Sync to custom directory -core docs sync --output ./site/content -``` - -## What Gets Synced - -For each repo, the following files are collected: - -| Source | Destination | -|--------|-------------| -| `README.md` | `index.md` | -| `CLAUDE.md` | `claude.md` | -| `CHANGELOG.md` | `changelog.md` | -| `docs/*.md` | `*.md` | - -## Integration with core.help - -The synced docs are used to build https://core.help: - -1. Run `core docs sync --output ../core-php/docs/packages` -2. VitePress builds the combined documentation -3. Deploy to core.help - -## See Also - -- [Configuration](../../configuration.md) - Project configuration diff --git a/docs/cmd/doctor/example.md b/docs/cmd/doctor/example.md deleted file mode 100644 index ba94d71..0000000 --- a/docs/cmd/doctor/example.md +++ /dev/null @@ -1,20 +0,0 @@ -# Doctor Examples - -```bash -core doctor -``` - -## Output - -``` -✓ go 1.25.0 -✓ git 2.43.0 -✓ gh 2.40.0 -✓ docker 24.0.7 -✓ task 3.30.0 -✓ golangci-lint 1.55.0 -✗ wails (not installed) -✓ php 8.3.0 -✓ composer 2.6.0 -✓ node 20.10.0 -``` diff --git a/docs/cmd/doctor/index.md b/docs/cmd/doctor/index.md deleted file mode 100644 index 02cc44d..0000000 --- a/docs/cmd/doctor/index.md +++ /dev/null @@ -1,81 +0,0 @@ -# core doctor - -Check your development environment for required tools and configuration. - -## Usage - -```bash -core doctor [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--verbose` | Show detailed version information | - -## What It Checks - -### Required Tools - -| Tool | Purpose | -|------|---------| -| `git` | Version control | -| `go` | Go compiler | -| `gh` | GitHub CLI | - -### Optional Tools - -| Tool | Purpose | -|------|---------| -| `node` | Node.js runtime | -| `docker` | Container runtime | -| `wails` | Desktop app framework | -| `qemu` | VM runtime for LinuxKit | -| `gpg` | Code signing | -| `codesign` | macOS signing (macOS only) | - -### Configuration - -- Git user name and email -- GitHub CLI authentication -- Go workspace setup - -## Output - -``` -Core Doctor -=========== - -Required: - [OK] git 2.43.0 - [OK] go 1.23.0 - [OK] gh 2.40.0 - -Optional: - [OK] node 20.10.0 - [OK] docker 24.0.7 - [--] wails (not installed) - [OK] qemu 8.2.0 - [OK] gpg 2.4.3 - [OK] codesign (available) - -Configuration: - [OK] git user.name: Your Name - [OK] git user.email: you@example.com - [OK] gh auth status: Logged in - -All checks passed! -``` - -## Exit Codes - -| Code | Meaning | -|------|---------| -| 0 | All required checks passed | -| 1 | One or more required checks failed | - -## See Also - -- [setup command](../setup/) - Clone repos from registry -- [dev](../dev/) - Development environment diff --git a/docs/cmd/go/cov/example.md b/docs/cmd/go/cov/example.md deleted file mode 100644 index 4fdc6c2..0000000 --- a/docs/cmd/go/cov/example.md +++ /dev/null @@ -1,18 +0,0 @@ -# Go Coverage Examples - -```bash -# Summary -core go cov - -# HTML report -core go cov --html - -# Open in browser -core go cov --open - -# Fail if below threshold -core go cov --threshold 80 - -# Specific package -core go cov --pkg ./pkg/release -``` diff --git a/docs/cmd/go/cov/index.md b/docs/cmd/go/cov/index.md deleted file mode 100644 index 3adeca3..0000000 --- a/docs/cmd/go/cov/index.md +++ /dev/null @@ -1,28 +0,0 @@ -# core go cov - -Generate coverage report with thresholds. - -## Usage - -```bash -core go cov [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--pkg` | Package to test (default: `./...`) | -| `--html` | Generate HTML coverage report | -| `--open` | Generate and open HTML report in browser | -| `--threshold` | Minimum coverage percentage (exit 1 if below) | - -## Examples - -```bash -core go cov # Summary -core go cov --html # HTML report -core go cov --open # Open in browser -core go cov --threshold 80 # Fail if < 80% -core go cov --pkg ./pkg/release # Specific package -``` diff --git a/docs/cmd/go/example.md b/docs/cmd/go/example.md deleted file mode 100644 index 51ad71a..0000000 --- a/docs/cmd/go/example.md +++ /dev/null @@ -1,89 +0,0 @@ -# Go Examples - -## Testing - -```bash -# Run all tests -core go test - -# Specific package -core go test --pkg ./pkg/core - -# Specific test -core go test --run TestHash - -# With coverage -core go test --coverage - -# Race detection -core go test --race -``` - -## Coverage - -```bash -# Summary -core go cov - -# HTML report -core go cov --html - -# Open in browser -core go cov --open - -# Fail if below threshold -core go cov --threshold 80 -``` - -## Formatting - -```bash -# Check -core go fmt - -# Fix -core go fmt --fix - -# Show diff -core go fmt --diff -``` - -## Linting - -```bash -# Check -core go lint - -# Auto-fix -core go lint --fix -``` - -## Installing - -```bash -# Auto-detect cmd/ -core go install - -# Specific path -core go install ./cmd/myapp - -# Pure Go (no CGO) -core go install --no-cgo -``` - -## Module Management - -```bash -core go mod tidy -core go mod download -core go mod verify -core go mod graph -``` - -## Workspace - -```bash -core go work sync -core go work init -core go work use ./pkg/mymodule -``` diff --git a/docs/cmd/go/fmt/example.md b/docs/cmd/go/fmt/example.md deleted file mode 100644 index 40233e0..0000000 --- a/docs/cmd/go/fmt/example.md +++ /dev/null @@ -1,12 +0,0 @@ -# Go Format Examples - -```bash -# Check only -core go fmt - -# Apply fixes -core go fmt --fix - -# Show diff -core go fmt --diff -``` diff --git a/docs/cmd/go/fmt/index.md b/docs/cmd/go/fmt/index.md deleted file mode 100644 index fe6113e..0000000 --- a/docs/cmd/go/fmt/index.md +++ /dev/null @@ -1,25 +0,0 @@ -# core go fmt - -Format Go code using goimports or gofmt. - -## Usage - -```bash -core go fmt [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--fix` | Fix formatting in place | -| `--diff` | Show diff of changes | -| `--check` | Check only, exit 1 if not formatted | - -## Examples - -```bash -core go fmt # Check formatting -core go fmt --fix # Fix formatting -core go fmt --diff # Show diff -``` diff --git a/docs/cmd/go/index.md b/docs/cmd/go/index.md deleted file mode 100644 index 981953c..0000000 --- a/docs/cmd/go/index.md +++ /dev/null @@ -1,15 +0,0 @@ -# core go - -Go development tools with enhanced output and environment setup. - -## Subcommands - -| Command | Description | -|---------|-------------| -| [test](test/) | Run tests with coverage | -| [cov](cov/) | Run tests with coverage report | -| [fmt](fmt/) | Format Go code | -| [lint](lint/) | Run golangci-lint | -| [install](install/) | Install Go binary | -| [mod](mod/) | Module management | -| [work](work/) | Workspace management | diff --git a/docs/cmd/go/install/example.md b/docs/cmd/go/install/example.md deleted file mode 100644 index bba88cd..0000000 --- a/docs/cmd/go/install/example.md +++ /dev/null @@ -1,15 +0,0 @@ -# Go Install Examples - -```bash -# Auto-detect cmd/ -core go install - -# Specific path -core go install ./cmd/myapp - -# Pure Go (no CGO) -core go install --no-cgo - -# Verbose -core go install -v -``` diff --git a/docs/cmd/go/install/index.md b/docs/cmd/go/install/index.md deleted file mode 100644 index e7bd109..0000000 --- a/docs/cmd/go/install/index.md +++ /dev/null @@ -1,25 +0,0 @@ -# core go install - -Install Go binary with auto-detection. - -## Usage - -```bash -core go install [path] [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--no-cgo` | Disable CGO | -| `-v` | Verbose | - -## Examples - -```bash -core go install # Install current module -core go install ./cmd/core # Install specific path -core go install --no-cgo # Pure Go (no C dependencies) -core go install -v # Verbose output -``` diff --git a/docs/cmd/go/lint/example.md b/docs/cmd/go/lint/example.md deleted file mode 100644 index 56b46d4..0000000 --- a/docs/cmd/go/lint/example.md +++ /dev/null @@ -1,22 +0,0 @@ -# Go Lint Examples - -```bash -# Check -core go lint - -# Auto-fix -core go lint --fix -``` - -## Configuration - -`.golangci.yml`: - -```yaml -linters: - enable: - - gofmt - - govet - - errcheck - - staticcheck -``` diff --git a/docs/cmd/go/lint/index.md b/docs/cmd/go/lint/index.md deleted file mode 100644 index 5f9e804..0000000 --- a/docs/cmd/go/lint/index.md +++ /dev/null @@ -1,22 +0,0 @@ -# core go lint - -Run golangci-lint. - -## Usage - -```bash -core go lint [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--fix` | Fix issues automatically | - -## Examples - -```bash -core go lint # Check -core go lint --fix # Auto-fix -``` diff --git a/docs/cmd/go/mod/download/index.md b/docs/cmd/go/mod/download/index.md deleted file mode 100644 index 240ef6d..0000000 --- a/docs/cmd/go/mod/download/index.md +++ /dev/null @@ -1,29 +0,0 @@ -# core go mod download - -Download modules to local cache. - -Wrapper around `go mod download`. Downloads all dependencies to the module cache. - -## Usage - -```bash -core go mod download -``` - -## What It Does - -- Downloads all modules in go.mod to `$GOPATH/pkg/mod` -- Useful for pre-populating cache for offline builds -- Validates checksums against go.sum - -## Examples - -```bash -# Download all dependencies -core go mod download -``` - -## See Also - -- [tidy](../tidy/) - Clean up go.mod -- [verify](../verify/) - Verify checksums diff --git a/docs/cmd/go/mod/example.md b/docs/cmd/go/mod/example.md deleted file mode 100644 index 57d2e66..0000000 --- a/docs/cmd/go/mod/example.md +++ /dev/null @@ -1,15 +0,0 @@ -# Go Module Examples - -```bash -# Tidy -core go mod tidy - -# Download -core go mod download - -# Verify -core go mod verify - -# Graph -core go mod graph -``` diff --git a/docs/cmd/go/mod/graph/index.md b/docs/cmd/go/mod/graph/index.md deleted file mode 100644 index 160c553..0000000 --- a/docs/cmd/go/mod/graph/index.md +++ /dev/null @@ -1,44 +0,0 @@ -# core go mod graph - -Print module dependency graph. - -Wrapper around `go mod graph`. Outputs the module dependency graph in text form. - -## Usage - -```bash -core go mod graph -``` - -## What It Does - -- Prints module dependencies as pairs -- Each line shows: `module@version dependency@version` -- Useful for understanding dependency relationships - -## Examples - -```bash -# Print dependency graph -core go mod graph - -# Find who depends on a specific module -core go mod graph | grep "some/module" - -# Visualise with graphviz -core go mod graph | dot -Tpng -o deps.png -``` - -## Output - -``` -forge.lthn.ai/core/cli github.com/stretchr/testify@v1.11.1 -github.com/stretchr/testify@v1.11.1 github.com/davecgh/go-spew@v1.1.2 -github.com/stretchr/testify@v1.11.1 github.com/pmezard/go-difflib@v1.0.1 -... -``` - -## See Also - -- [tidy](../tidy/) - Clean up go.mod -- [dev impact](../../../dev/impact/) - Show repo dependency impact diff --git a/docs/cmd/go/mod/index.md b/docs/cmd/go/mod/index.md deleted file mode 100644 index ee8e46e..0000000 --- a/docs/cmd/go/mod/index.md +++ /dev/null @@ -1,21 +0,0 @@ -# core go mod - -Module management. - -## Subcommands - -| Command | Description | -|---------|-------------| -| `tidy` | Add missing and remove unused modules | -| `download` | Download modules to local cache | -| `verify` | Verify dependencies | -| `graph` | Print module dependency graph | - -## Examples - -```bash -core go mod tidy -core go mod download -core go mod verify -core go mod graph -``` diff --git a/docs/cmd/go/mod/tidy/index.md b/docs/cmd/go/mod/tidy/index.md deleted file mode 100644 index 684b07e..0000000 --- a/docs/cmd/go/mod/tidy/index.md +++ /dev/null @@ -1,29 +0,0 @@ -# core go mod tidy - -Add missing and remove unused modules. - -Wrapper around `go mod tidy`. Ensures go.mod and go.sum are in sync with the source code. - -## Usage - -```bash -core go mod tidy -``` - -## What It Does - -- Adds missing module requirements -- Removes unused module requirements -- Updates go.sum with checksums - -## Examples - -```bash -# Tidy the current module -core go mod tidy -``` - -## See Also - -- [download](../download/) - Download modules -- [verify](../verify/) - Verify dependencies diff --git a/docs/cmd/go/mod/verify/index.md b/docs/cmd/go/mod/verify/index.md deleted file mode 100644 index e01dc2a..0000000 --- a/docs/cmd/go/mod/verify/index.md +++ /dev/null @@ -1,41 +0,0 @@ -# core go mod verify - -Verify dependencies have not been modified. - -Wrapper around `go mod verify`. Checks that dependencies in the module cache match their checksums in go.sum. - -## Usage - -```bash -core go mod verify -``` - -## What It Does - -- Verifies each module in the cache -- Compares against go.sum checksums -- Reports any tampering or corruption - -## Examples - -```bash -# Verify all dependencies -core go mod verify -``` - -## Output - -``` -all modules verified -``` - -Or if verification fails: - -``` -github.com/example/pkg v1.2.3: dir has been modified -``` - -## See Also - -- [download](../download/) - Download modules -- [tidy](../tidy/) - Clean up go.mod diff --git a/docs/cmd/go/test/example.md b/docs/cmd/go/test/example.md deleted file mode 100644 index 85ff1b5..0000000 --- a/docs/cmd/go/test/example.md +++ /dev/null @@ -1,27 +0,0 @@ -# Go Test Examples - -```bash -# All tests -core go test - -# Specific package -core go test --pkg ./pkg/core - -# Specific test -core go test --run TestHash - -# With coverage -core go test --coverage - -# Race detection -core go test --race - -# Short tests only -core go test --short - -# Verbose -core go test -v - -# JSON output (CI) -core go test --json -``` diff --git a/docs/cmd/go/test/index.md b/docs/cmd/go/test/index.md deleted file mode 100644 index 8b54524..0000000 --- a/docs/cmd/go/test/index.md +++ /dev/null @@ -1,31 +0,0 @@ -# core go test - -Run Go tests with coverage and filtered output. - -## Usage - -```bash -core go test [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--pkg` | Package to test (default: `./...`) | -| `--run` | Run only tests matching regexp | -| `--short` | Run only short tests | -| `--race` | Enable race detector | -| `--coverage` | Show detailed per-package coverage | -| `--json` | Output JSON results | -| `-v` | Verbose output | - -## Examples - -```bash -core go test # All tests -core go test --pkg ./pkg/core # Specific package -core go test --run TestHash # Specific test -core go test --coverage # With coverage -core go test --race # Race detection -``` diff --git a/docs/cmd/go/work/index.md b/docs/cmd/go/work/index.md deleted file mode 100644 index 4022507..0000000 --- a/docs/cmd/go/work/index.md +++ /dev/null @@ -1,19 +0,0 @@ -# core go work - -Go workspace management commands. - -## Subcommands - -| Command | Description | -|---------|-------------| -| `sync` | Sync go.work with modules | -| `init` | Initialize go.work | -| `use` | Add module to workspace | - -## Examples - -```bash -core go work sync # Sync workspace -core go work init # Initialize workspace -core go work use ./pkg/mymodule # Add module to workspace -``` diff --git a/docs/cmd/go/work/init/index.md b/docs/cmd/go/work/init/index.md deleted file mode 100644 index 6527324..0000000 --- a/docs/cmd/go/work/init/index.md +++ /dev/null @@ -1,40 +0,0 @@ -# core go work init - -Initialize a Go workspace. - -Wrapper around `go work init`. Creates a new go.work file in the current directory. - -## Usage - -```bash -core go work init -``` - -## What It Does - -- Creates a go.work file -- Automatically adds current module if go.mod exists -- Enables multi-module development - -## Examples - -```bash -# Initialize workspace -core go work init - -# Then add more modules -core go work use ./pkg/mymodule -``` - -## Generated File - -```go -go 1.25 - -use . -``` - -## See Also - -- [use](../use/) - Add module to workspace -- [sync](../sync/) - Sync workspace diff --git a/docs/cmd/go/work/sync/index.md b/docs/cmd/go/work/sync/index.md deleted file mode 100644 index 38caed1..0000000 --- a/docs/cmd/go/work/sync/index.md +++ /dev/null @@ -1,35 +0,0 @@ -# core go work sync - -Sync go.work with modules. - -Wrapper around `go work sync`. Synchronises the workspace's build list back to the workspace modules. - -## Usage - -```bash -core go work sync -``` - -## What It Does - -- Updates each module's go.mod to match the workspace build list -- Ensures all modules use compatible dependency versions -- Run after adding new modules or updating dependencies - -## Examples - -```bash -# Sync workspace -core go work sync -``` - -## When To Use - -- After running `go get` to update a dependency -- After adding a new module with `core go work use` -- When modules have conflicting dependency versions - -## See Also - -- [init](../init/) - Initialize workspace -- [use](../use/) - Add module to workspace diff --git a/docs/cmd/go/work/use/index.md b/docs/cmd/go/work/use/index.md deleted file mode 100644 index 25e0cab..0000000 --- a/docs/cmd/go/work/use/index.md +++ /dev/null @@ -1,46 +0,0 @@ -# core go work use - -Add module to workspace. - -Wrapper around `go work use`. Adds one or more modules to the go.work file. - -## Usage - -```bash -core go work use [paths...] -``` - -## What It Does - -- Adds specified module paths to go.work -- Auto-discovers modules if no paths given -- Enables developing multiple modules together - -## Examples - -```bash -# Add a specific module -core go work use ./pkg/mymodule - -# Add multiple modules -core go work use ./pkg/one ./pkg/two - -# Auto-discover and add all modules -core go work use -``` - -## Auto-Discovery - -When called without arguments, scans for go.mod files and adds all found modules: - -```bash -core go work use -# Added ./pkg/build -# Added ./pkg/repos -# Added ./cmd/core -``` - -## See Also - -- [init](../init/) - Initialize workspace -- [sync](../sync/) - Sync workspace diff --git a/docs/cmd/index.md b/docs/cmd/index.md deleted file mode 100644 index de2c061..0000000 --- a/docs/cmd/index.md +++ /dev/null @@ -1,31 +0,0 @@ -# Core CLI - -Unified interface for Go/PHP development, multi-repo management, and deployment. - -## Commands - -| Command | Description | -|---------|-------------| -| [ai](ai/) | AI agent task management and Claude integration | -| [go](go/) | Go development tools | -| [php](php/) | Laravel/PHP development tools | -| [build](build/) | Build projects | -| [ci](ci/) | Publish releases | -| [sdk](sdk/) | SDK validation and compatibility | -| [dev](dev/) | Multi-repo workflow + dev environment | -| [pkg](pkg/) | Package management | -| [vm](vm/) | LinuxKit VM management | -| [docs](docs/) | Documentation management | -| [setup](setup/) | Clone repos from registry | -| [doctor](doctor/) | Check environment | -| [test](test/) | Run Go tests with coverage | - -## Installation - -```bash -go install forge.lthn.ai/core/cli/cmd/core@latest -``` - -Verify: `core doctor` - -See [Getting Started](../getting-started.md) for all installation options. diff --git a/docs/cmd/php/example.md b/docs/cmd/php/example.md deleted file mode 100644 index 96e1600..0000000 --- a/docs/cmd/php/example.md +++ /dev/null @@ -1,111 +0,0 @@ -# PHP Examples - -## Development - -```bash -# Start all services -core php dev - -# With HTTPS -core php dev --https - -# Skip services -core php dev --no-vite --no-horizon -``` - -## Testing - -```bash -# Run all -core php test - -# Parallel -core php test --parallel - -# With coverage -core php test --coverage - -# Filter -core php test --filter UserTest -``` - -## Code Quality - -```bash -# Format -core php fmt --fix - -# Static analysis -core php analyse --level 9 -``` - -## Deployment - -```bash -# Production -core php deploy - -# Staging -core php deploy --staging - -# Wait for completion -core php deploy --wait - -# Check status -core php deploy:status - -# Rollback -core php deploy:rollback -``` - -## Configuration - -### .env - -```env -COOLIFY_URL=https://coolify.example.com -COOLIFY_TOKEN=your-api-token -COOLIFY_APP_ID=production-app-id -COOLIFY_STAGING_APP_ID=staging-app-id -``` - -### .core/php.yaml - -```yaml -version: 1 - -dev: - domain: myapp.test - ssl: true - services: - - frankenphp - - vite - - horizon - - reverb - - redis - -deploy: - coolify: - server: https://coolify.example.com - project: my-project -``` - -## Package Linking - -```bash -# Link local packages -core php packages link ../my-package - -# Update linked -core php packages update - -# Unlink -core php packages unlink my-package -``` - -## SSL Setup - -```bash -core php ssl -core php ssl --domain myapp.test -``` diff --git a/docs/cmd/php/index.md b/docs/cmd/php/index.md deleted file mode 100644 index 83ad596..0000000 --- a/docs/cmd/php/index.md +++ /dev/null @@ -1,413 +0,0 @@ -# core php - -Laravel/PHP development tools with FrankenPHP. - -## Commands - -### Development - -| Command | Description | -|---------|-------------| -| [`dev`](#php-dev) | Start development environment | -| [`logs`](#php-logs) | View service logs | -| [`stop`](#php-stop) | Stop all services | -| [`status`](#php-status) | Show service status | -| [`ssl`](#php-ssl) | Setup SSL certificates with mkcert | - -### Build & Production - -| Command | Description | -|---------|-------------| -| [`build`](#php-build) | Build Docker or LinuxKit image | -| [`serve`](#php-serve) | Run production container | -| [`shell`](#php-shell) | Open shell in running container | - -### Code Quality - -| Command | Description | -|---------|-------------| -| [`test`](#php-test) | Run PHP tests (PHPUnit/Pest) | -| [`fmt`](#php-fmt) | Format code with Laravel Pint | -| [`analyse`](#php-analyse) | Run PHPStan static analysis | - -### Package Management - -| Command | Description | -|---------|-------------| -| [`packages link`](#php-packages-link) | Link local packages by path | -| [`packages unlink`](#php-packages-unlink) | Unlink packages by name | -| [`packages update`](#php-packages-update) | Update linked packages | -| [`packages list`](#php-packages-list) | List linked packages | - -### Deployment (Coolify) - -| Command | Description | -|---------|-------------| -| [`deploy`](#php-deploy) | Deploy to Coolify | -| [`deploy:status`](#php-deploystatus) | Show deployment status | -| [`deploy:rollback`](#php-deployrollback) | Rollback to previous deployment | -| [`deploy:list`](#php-deploylist) | List recent deployments | - ---- - -## php dev - -Start the Laravel development environment with all detected services. - -```bash -core php dev [flags] -``` - -### Services Orchestrated - -- **FrankenPHP/Octane** - HTTP server (port 8000, HTTPS on 443) -- **Vite** - Frontend dev server (port 5173) -- **Laravel Horizon** - Queue workers -- **Laravel Reverb** - WebSocket server (port 8080) -- **Redis** - Cache and queue backend (port 6379) - -### Flags - -| Flag | Description | -|------|-------------| -| `--no-vite` | Skip Vite dev server | -| `--no-horizon` | Skip Laravel Horizon | -| `--no-reverb` | Skip Laravel Reverb | -| `--no-redis` | Skip Redis server | -| `--https` | Enable HTTPS with mkcert | -| `--domain` | Domain for SSL certificate (default: from APP_URL) | -| `--port` | FrankenPHP port (default: 8000) | - -### Examples - -```bash -# Start all detected services -core php dev - -# With HTTPS -core php dev --https - -# Skip optional services -core php dev --no-horizon --no-reverb -``` - ---- - -## php logs - -Stream unified logs from all running services. - -```bash -core php logs [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--follow` | Follow log output | -| `--service` | Specific service (frankenphp, vite, horizon, reverb, redis) | - ---- - -## php stop - -Stop all running Laravel services. - -```bash -core php stop -``` - ---- - -## php status - -Show the status of all Laravel services and project configuration. - -```bash -core php status -``` - ---- - -## php ssl - -Setup local SSL certificates using mkcert. - -```bash -core php ssl [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--domain` | Domain for certificate (default: from APP_URL or localhost) | - ---- - -## php build - -Build a production-ready container image. - -```bash -core php build [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--type` | Build type: `docker` (default) or `linuxkit` | -| `--name` | Image name (default: project directory name) | -| `--tag` | Image tag (default: latest) | -| `--platform` | Target platform (e.g., linux/amd64, linux/arm64) | -| `--dockerfile` | Path to custom Dockerfile | -| `--output` | Output path for LinuxKit image | -| `--format` | LinuxKit format: qcow2 (default), iso, raw, vmdk | -| `--template` | LinuxKit template name (default: server-php) | -| `--no-cache` | Build without cache | - -### Examples - -```bash -# Build Docker image -core php build - -# With custom name and tag -core php build --name myapp --tag v1.0 - -# Build LinuxKit image -core php build --type linuxkit -``` - ---- - -## php serve - -Run a production container. - -```bash -core php serve [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--name` | Docker image name (required) | -| `--tag` | Image tag (default: latest) | -| `--container` | Container name | -| `--port` | HTTP port (default: 80) | -| `--https-port` | HTTPS port (default: 443) | -| `-d` | Run in detached mode | -| `--env-file` | Path to environment file | - -### Examples - -```bash -core php serve --name myapp -core php serve --name myapp -d -core php serve --name myapp --port 8080 -``` - ---- - -## php shell - -Open an interactive shell in a running container. - -```bash -core php shell -``` - ---- - -## php test - -Run PHP tests using PHPUnit or Pest. - -```bash -core php test [flags] -``` - -Auto-detects Pest if `tests/Pest.php` exists. - -### Flags - -| Flag | Description | -|------|-------------| -| `--parallel` | Run tests in parallel | -| `--coverage` | Generate code coverage | -| `--filter` | Filter tests by name pattern | -| `--group` | Run only tests in specified group | - -### Examples - -```bash -core php test -core php test --parallel --coverage -core php test --filter UserTest -``` - ---- - -## php fmt - -Format PHP code using Laravel Pint. - -```bash -core php fmt [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--fix` | Auto-fix formatting issues | -| `--diff` | Show diff of changes | - ---- - -## php analyse - -Run PHPStan or Larastan static analysis. - -```bash -core php analyse [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--level` | PHPStan analysis level (0-9) | -| `--memory` | Memory limit (e.g., 2G) | - ---- - -## php packages link - -Link local PHP packages for development. - -```bash -core php packages link [...] -``` - -Adds path repositories to composer.json with symlink enabled. - ---- - -## php packages unlink - -Remove linked packages from composer.json. - -```bash -core php packages unlink [...] -``` - ---- - -## php packages update - -Update linked packages via Composer. - -```bash -core php packages update [...] -``` - ---- - -## php packages list - -List all locally linked packages. - -```bash -core php packages list -``` - ---- - -## php deploy - -Deploy the PHP application to Coolify. - -```bash -core php deploy [flags] -``` - -### Configuration - -Requires environment variables in `.env`: -``` -COOLIFY_URL=https://coolify.example.com -COOLIFY_TOKEN=your-api-token -COOLIFY_APP_ID=production-app-id -COOLIFY_STAGING_APP_ID=staging-app-id -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--staging` | Deploy to staging environment | -| `--force` | Force deployment even if no changes detected | -| `--wait` | Wait for deployment to complete | - ---- - -## php deploy:status - -Show the status of a deployment. - -```bash -core php deploy:status [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--staging` | Check staging environment | -| `--id` | Specific deployment ID | - ---- - -## php deploy:rollback - -Rollback to a previous deployment. - -```bash -core php deploy:rollback [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--staging` | Rollback staging environment | -| `--id` | Specific deployment ID to rollback to | -| `--wait` | Wait for rollback to complete | - ---- - -## php deploy:list - -List recent deployments. - -```bash -core php deploy:list [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--staging` | List staging deployments | -| `--limit` | Number of deployments (default: 10) | - ---- - -## Configuration - -Optional `.core/php.yaml` - see [Configuration](example.md#configuration) for examples. diff --git a/docs/cmd/pkg/example.md b/docs/cmd/pkg/example.md deleted file mode 100644 index 7904aae..0000000 --- a/docs/cmd/pkg/example.md +++ /dev/null @@ -1,36 +0,0 @@ -# Package Examples - -## Search - -```bash -core pkg search core- -core pkg search api -core pkg search --org myorg -``` - -## Install - -```bash -core pkg install core-api -core pkg install host-uk/core-api -``` - -## List - -```bash -core pkg list -core pkg list --format json -``` - -## Update - -```bash -core pkg update -core pkg update core-api -``` - -## Outdated - -```bash -core pkg outdated -``` diff --git a/docs/cmd/pkg/index.md b/docs/cmd/pkg/index.md deleted file mode 100644 index fcc218b..0000000 --- a/docs/cmd/pkg/index.md +++ /dev/null @@ -1,144 +0,0 @@ -# core pkg - -Package management for host-uk repositories. - -## Usage - -```bash -core pkg [flags] -``` - -## Commands - -| Command | Description | -|---------|-------------| -| [`search`](#pkg-search) | Search GitHub for packages | -| [`install`](#pkg-install) | Clone a package from GitHub | -| [`list`](#pkg-list) | List installed packages | -| [`update`](#pkg-update) | Update installed packages | -| [`outdated`](#pkg-outdated) | Check for outdated packages | - ---- - -## pkg search - -Search GitHub for host-uk packages. - -```bash -core pkg search [flags] -``` - -Results are cached for 1 hour in `.core/cache/`. - -### Flags - -| Flag | Description | -|------|-------------| -| `--org` | GitHub organisation (default: host-uk) | -| `--pattern` | Repo name pattern (* for wildcard) | -| `--type` | Filter by type in name (mod, services, plug, website) | -| `--limit` | Max results (default: 50) | -| `--refresh` | Bypass cache and fetch fresh data | - -### Examples - -```bash -# List all repos in org -core pkg search - -# Search for core-* repos -core pkg search --pattern 'core-*' - -# Search different org -core pkg search --org mycompany - -# Bypass cache -core pkg search --refresh -``` - ---- - -## pkg install - -Clone a package from GitHub. - -```bash -core pkg install [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--dir` | Target directory (default: ./packages or current dir) | -| `--add` | Add to repos.yaml registry | - -### Examples - -```bash -# Clone to packages/ -core pkg install host-uk/core-php - -# Clone to custom directory -core pkg install host-uk/core-tenant --dir ./packages - -# Clone and add to registry -core pkg install host-uk/core-admin --add -``` - ---- - -## pkg list - -List installed packages from repos.yaml. - -```bash -core pkg list -``` - -Shows installed status (✓) and description for each package. - ---- - -## pkg update - -Pull latest changes for installed packages. - -```bash -core pkg update [...] [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--all` | Update all packages | - -### Examples - -```bash -# Update specific package -core pkg update core-php - -# Update all packages -core pkg update --all -``` - ---- - -## pkg outdated - -Check which packages have unpulled commits. - -```bash -core pkg outdated -``` - -Fetches from remote and shows packages that are behind. - ---- - -## See Also - -- [setup](../setup/) - Clone all repos from registry -- [dev work](../dev/work/) - Multi-repo workflow diff --git a/docs/cmd/pkg/search/example.md b/docs/cmd/pkg/search/example.md deleted file mode 100644 index fbcaa6f..0000000 --- a/docs/cmd/pkg/search/example.md +++ /dev/null @@ -1,23 +0,0 @@ -# Package Search Examples - -```bash -# Find all core-* packages -core pkg search core- - -# Search term -core pkg search api - -# Different org -core pkg search --org myorg query -``` - -## Output - -``` -┌──────────────┬─────────────────────────────┐ -│ Package │ Description │ -├──────────────┼─────────────────────────────┤ -│ core-api │ REST API framework │ -│ core-auth │ Authentication utilities │ -└──────────────┴─────────────────────────────┘ -``` diff --git a/docs/cmd/pkg/search/index.md b/docs/cmd/pkg/search/index.md deleted file mode 100644 index 57fea91..0000000 --- a/docs/cmd/pkg/search/index.md +++ /dev/null @@ -1,75 +0,0 @@ -# core pkg search - -Search GitHub for repositories matching a pattern. - -Uses `gh` CLI for authenticated search. Results are cached for 1 hour. - -## Usage - -```bash -core pkg search [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--pattern` | Repo name pattern (* for wildcard) | -| `--org` | GitHub organization (default: host-uk) | -| `--type` | Filter by type in name (mod, services, plug, website) | -| `--limit` | Max results (default: 50) | -| `--refresh` | Bypass cache and fetch fresh data | - -## Examples - -```bash -# List all host-uk repos -core pkg search - -# Search for core-* repos -core pkg search --pattern "core-*" - -# Search different org -core pkg search --org mycompany - -# Filter by type -core pkg search --type services - -# Bypass cache -core pkg search --refresh - -# Combine filters -core pkg search --pattern "core-*" --type mod --limit 20 -``` - -## Output - -``` -Found 5 repositories: - - host-uk/core - Go CLI for the host-uk ecosystem - ★ 42 Go Updated 2 hours ago - - host-uk/core-php - PHP/Laravel packages for Core - ★ 18 PHP Updated 1 day ago - - host-uk/core-images - Docker and LinuxKit images - ★ 8 Dockerfile Updated 3 days ago -``` - -## Authentication - -Uses GitHub CLI (`gh`) authentication. Ensure you're logged in: - -```bash -gh auth status -gh auth login # if not authenticated -``` - -## See Also - -- [pkg install](../) - Clone a package from GitHub -- [setup command](../../setup/) - Clone all repos from registry diff --git a/docs/cmd/sdk/example.md b/docs/cmd/sdk/example.md deleted file mode 100644 index 2fada8c..0000000 --- a/docs/cmd/sdk/example.md +++ /dev/null @@ -1,35 +0,0 @@ -# SDK Examples - -## Validate - -```bash -core sdk validate -core sdk validate --spec ./api.yaml -``` - -## Diff - -```bash -# Compare with tag -core sdk diff --base v1.0.0 - -# Compare files -core sdk diff --base ./old-api.yaml --spec ./new-api.yaml -``` - -## Output - -``` -Breaking changes detected: - -- DELETE /users/{id}/profile - Endpoint removed - -- PATCH /users/{id} - Required field 'email' added - -Non-breaking changes: - -+ POST /users/{id}/avatar - New endpoint added -``` diff --git a/docs/cmd/sdk/index.md b/docs/cmd/sdk/index.md deleted file mode 100644 index bd6828c..0000000 --- a/docs/cmd/sdk/index.md +++ /dev/null @@ -1,106 +0,0 @@ -# core sdk - -SDK validation and API compatibility tools. - -To generate SDKs, use: `core build sdk` - -## Usage - -```bash -core sdk [flags] -``` - -## Commands - -| Command | Description | -|---------|-------------| -| `diff` | Check for breaking API changes | -| `validate` | Validate OpenAPI spec | - -## sdk validate - -Validate an OpenAPI specification file. - -```bash -core sdk validate [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--spec` | Path to OpenAPI spec file (auto-detected) | - -### Examples - -```bash -# Validate detected spec -core sdk validate - -# Validate specific file -core sdk validate --spec api/openapi.yaml -``` - -## sdk diff - -Check for breaking changes between API versions. - -```bash -core sdk diff [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `--base` | Base spec version (git tag or file path) | -| `--spec` | Current spec file (auto-detected) | - -### Examples - -```bash -# Compare against previous release -core sdk diff --base v1.0.0 - -# Compare two files -core sdk diff --base old-api.yaml --spec new-api.yaml -``` - -### Breaking Changes Detected - -- Removed endpoints -- Changed parameter types -- Removed required fields -- Changed response types - -## SDK Generation - -SDK generation is handled by `core build sdk`, not this command. - -```bash -# Generate SDKs -core build sdk - -# Generate specific language -core build sdk --lang typescript - -# Preview without writing -core build sdk --dry-run -``` - -See [build sdk](../build/sdk/) for generation details. - -## Spec Auto-Detection - -Core looks for OpenAPI specs in this order: - -1. Path specified in config (`sdk.spec`) -2. `openapi.yaml` / `openapi.json` -3. `api/openapi.yaml` / `api/openapi.json` -4. `docs/openapi.yaml` / `docs/openapi.json` -5. Laravel Scramble endpoint (`/docs/api.json`) - -## See Also - -- [build sdk](../build/sdk/) - Generate SDKs from OpenAPI -- [ci command](../ci/) - Release workflow diff --git a/docs/cmd/setup/example.md b/docs/cmd/setup/example.md deleted file mode 100644 index 23f2410..0000000 --- a/docs/cmd/setup/example.md +++ /dev/null @@ -1,293 +0,0 @@ -# Setup Examples - -## Clone from Registry - -```bash -# Clone all repos defined in repos.yaml -core setup - -# Preview what would be cloned -core setup --dry-run - -# Only foundation packages -core setup --only foundation - -# Multiple types -core setup --only foundation,module - -# Use specific registry file -core setup --registry ~/projects/repos.yaml -``` - -## Bootstrap New Workspace - -```bash -# In an empty directory - bootstraps in place -mkdir my-workspace && cd my-workspace -core setup - -# Shows interactive wizard to select packages: -# ┌─────────────────────────────────────────────┐ -# │ Select packages to clone │ -# │ Use space to select, enter to confirm │ -# │ │ -# │ ── Foundation (core framework) ── │ -# │ ☑ core-php Foundation framework │ -# │ ☑ core-tenant Multi-tenancy module │ -# │ │ -# │ ── Products (applications) ── │ -# │ ☐ core-bio Link-in-bio product │ -# │ ☐ core-social Social scheduling │ -# └─────────────────────────────────────────────┘ - -# Non-interactive: clone all packages -core setup --all - -# Create workspace in subdirectory -cd ~/Code -core setup --name my-project - -# CI mode: fully non-interactive -core setup --all --name ci-test -``` - -## Setup Single Repository - -```bash -# In a git repo without .core/ configuration -cd ~/Code/my-go-project -core setup - -# Shows choice dialog: -# ┌─────────────────────────────────────────────┐ -# │ Setup options │ -# │ You're in a git repository. What would you │ -# │ like to do? │ -# │ │ -# │ ● Setup this repo (create .core/ config) │ -# │ ○ Create a new workspace (clone repos) │ -# └─────────────────────────────────────────────┘ - -# Preview generated configuration -core setup --dry-run - -# Output: -# → Setting up repository configuration -# -# ✓ Detected project type: go -# → Also found: (none) -# -# → Would create: -# /Users/you/Code/my-go-project/.core/build.yaml -# -# Configuration preview: -# version: 1 -# project: -# name: my-go-project -# description: Go application -# main: ./cmd/my-go-project -# binary: my-go-project -# ... -``` - -## Configuration Files - -### repos.yaml (Workspace Registry) - -```yaml -org: host-uk -base_path: . -defaults: - ci: github - license: EUPL-1.2 - branch: main -repos: - core-php: - type: foundation - description: Foundation framework - core-tenant: - type: module - depends_on: [core-php] - description: Multi-tenancy module - core-admin: - type: module - depends_on: [core-php, core-tenant] - description: Admin panel - core-bio: - type: product - depends_on: [core-php, core-tenant] - description: Link-in-bio product - domain: bio.host.uk.com - core-devops: - type: foundation - clone: false # Already exists, skip cloning -``` - -### .core/build.yaml (Repository Config) - -Generated for Go projects: - -```yaml -version: 1 -project: - name: my-project - description: Go application - main: ./cmd/my-project - binary: my-project -build: - cgo: false - flags: - - -trimpath - ldflags: - - -s - - -w - env: [] -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 -sign: - enabled: false -``` - -Generated for Wails projects: - -```yaml -version: 1 -project: - name: my-app - description: Wails desktop application - main: . - binary: my-app -targets: - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - - os: linux - arch: amd64 -``` - -### .core/release.yaml (Release Config) - -Generated for Go projects: - -```yaml -version: 1 -project: - name: my-project - repository: owner/my-project - -changelog: - include: - - feat - - fix - - perf - - refactor - exclude: - - chore - - docs - - style - - test - -publishers: - - type: github - draft: false - prerelease: false -``` - -### .core/test.yaml (Test Config) - -Generated for Go projects: - -```yaml -version: 1 - -commands: - - name: unit - run: go test ./... - - name: coverage - run: go test -coverprofile=coverage.out ./... - - name: race - run: go test -race ./... - -env: - CGO_ENABLED: "0" -``` - -Generated for PHP projects: - -```yaml -version: 1 - -commands: - - name: unit - run: vendor/bin/pest --parallel - - name: types - run: vendor/bin/phpstan analyse - - name: lint - run: vendor/bin/pint --test - -env: - APP_ENV: testing - DB_CONNECTION: sqlite -``` - -Generated for Node.js projects: - -```yaml -version: 1 - -commands: - - name: unit - run: npm test - - name: lint - run: npm run lint - - name: typecheck - run: npm run typecheck - -env: - NODE_ENV: test -``` - -## Workflow Examples - -### New Developer Setup - -```bash -# Clone the workspace -mkdir host-uk && cd host-uk -core setup - -# Select packages in wizard, then: -core health # Check all repos are healthy -core doctor # Verify environment -``` - -### CI Pipeline Setup - -```bash -# Non-interactive full clone -core setup --all --name workspace - -# Or with specific packages -core setup --only foundation,module --name workspace -``` - -### Adding Build Config to Existing Repo - -```bash -cd my-existing-project -core setup # Choose "Setup this repo" -# Edit .core/build.yaml as needed -core build # Build the project -``` diff --git a/docs/cmd/setup/index.md b/docs/cmd/setup/index.md deleted file mode 100644 index d07121f..0000000 --- a/docs/cmd/setup/index.md +++ /dev/null @@ -1,213 +0,0 @@ -# core setup - -Clone repositories from registry or bootstrap a new workspace. - -## Overview - -The `setup` command operates in three modes: - -1. **Registry mode** - When `repos.yaml` exists nearby, clones repositories into packages/ -2. **Bootstrap mode** - When no registry exists, clones `core-devops` first, then presents an interactive wizard to select packages -3. **Repo setup mode** - When run in a git repo root, offers to create `.core/build.yaml` configuration - -## Usage - -```bash -core setup [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--registry` | Path to repos.yaml (auto-detected if not specified) | -| `--dry-run` | Show what would be cloned without cloning | -| `--only` | Only clone repos of these types (comma-separated: foundation,module,product) | -| `--all` | Skip wizard, clone all packages (non-interactive) | -| `--name` | Project directory name for bootstrap mode | -| `--build` | Run build after cloning | - ---- - -## Registry Mode - -When `repos.yaml` is found nearby (current directory or parents), setup clones all defined repositories: - -```bash -# In a directory with repos.yaml -core setup - -# Preview what would be cloned -core setup --dry-run - -# Only clone foundation packages -core setup --only foundation - -# Multiple types -core setup --only foundation,module -``` - -In registry mode with a TTY, an interactive wizard allows you to select which packages to clone. Use `--all` to skip the wizard and clone everything. - ---- - -## Bootstrap Mode - -When no `repos.yaml` exists, setup enters bootstrap mode: - -```bash -# In an empty directory - bootstraps workspace in place -mkdir my-project && cd my-project -core setup - -# In a non-empty directory - creates subdirectory -cd ~/Code -core setup --name my-workspace - -# Non-interactive: clone all packages -core setup --all --name ci-test -``` - -Bootstrap mode: -1. Detects if current directory is empty -2. If not empty, prompts for project name (or uses `--name`) -3. Clones `core-devops` (contains `repos.yaml`) -4. Loads the registry from core-devops -5. Shows interactive package selection wizard (unless `--all`) -6. Clones selected packages -7. Optionally runs build (with `--build`) - ---- - -## Repo Setup Mode - -When run in a git repository root (without `repos.yaml`), setup offers two choices: - -1. **Setup Working Directory** - Creates `.core/build.yaml` based on detected project type -2. **Create Package** - Creates a subdirectory and clones packages there - -```bash -cd ~/Code/my-go-project -core setup - -# Output: -# >> This directory is a git repository -# > Setup Working Directory -# Create Package (clone repos into subdirectory) -``` - -Choosing "Setup Working Directory" detects the project type and generates configuration: - -| Detected File | Project Type | -|---------------|--------------| -| `wails.json` | Wails | -| `go.mod` | Go | -| `composer.json` | PHP | -| `package.json` | Node.js | - -Creates three config files in `.core/`: - -| File | Purpose | -|------|---------| -| `build.yaml` | Build targets, flags, output settings | -| `release.yaml` | Changelog format, GitHub release config | -| `test.yaml` | Test commands, environment variables | - -Also auto-detects GitHub repo from git remote for release config. - -See [Configuration Files](example.md#configuration-files) for generated config examples. - ---- - -## Interactive Wizard - -When running in a terminal (TTY), the setup command presents an interactive multi-select wizard: - -- Packages are grouped by type (foundation, module, product, template) -- Use arrow keys to navigate -- Press space to select/deselect packages -- Type to filter the list -- Press enter to confirm selection - -The wizard is skipped when: -- `--all` flag is specified -- Not running in a TTY (e.g., CI pipelines) -- `--dry-run` is specified - ---- - -## Examples - -### Clone from Registry - -```bash -# Clone all repos (interactive wizard) -core setup - -# Clone all repos (non-interactive) -core setup --all - -# Preview without cloning -core setup --dry-run - -# Only foundation packages -core setup --only foundation -``` - -### Bootstrap New Workspace - -```bash -# Interactive bootstrap in empty directory -mkdir workspace && cd workspace -core setup - -# Non-interactive with all packages -core setup --all --name my-project - -# Bootstrap and run build -core setup --all --name my-project --build -``` - ---- - -## Registry Format - -The registry file (`repos.yaml`) defines repositories. See [Configuration Files](example.md#configuration-files) for format. - ---- - -## Finding Registry - -Core looks for `repos.yaml` in: - -1. Current directory -2. Parent directories (walking up to root) -3. `~/Code/host-uk/repos.yaml` -4. `~/.config/core/repos.yaml` - ---- - -## After Setup - -```bash -# Check workspace health -core dev health - -# Full workflow (status + commit + push) -core dev work - -# Build the project -core build - -# Run tests -core go test # Go projects -core php test # PHP projects -``` - ---- - -## See Also - -- [dev work](../dev/work/) - Multi-repo operations -- [build](../build/) - Build projects -- [doctor](../doctor/) - Check environment diff --git a/docs/cmd/test/example.md b/docs/cmd/test/example.md deleted file mode 100644 index 9e2a4a7..0000000 --- a/docs/cmd/test/example.md +++ /dev/null @@ -1,8 +0,0 @@ -# Test Examples - -**Note:** Prefer `core go test` or `core php test` instead. - -```bash -core test -core test --coverage -``` diff --git a/docs/cmd/test/index.md b/docs/cmd/test/index.md deleted file mode 100644 index 920baea..0000000 --- a/docs/cmd/test/index.md +++ /dev/null @@ -1,74 +0,0 @@ -# core test - -Run Go tests with coverage reporting. - -Sets `MACOSX_DEPLOYMENT_TARGET=26.0` to suppress linker warnings on macOS. - -## Usage - -```bash -core test [flags] -``` - -## Flags - -| Flag | Description | -|------|-------------| -| `--coverage` | Show detailed per-package coverage | -| `--json` | Output JSON for CI/agents | -| `--pkg` | Package pattern to test (default: ./...) | -| `--race` | Enable race detector | -| `--run` | Run only tests matching this regex | -| `--short` | Skip long-running tests | -| `--verbose` | Show test output as it runs | - -## Examples - -```bash -# Run all tests with coverage summary -core test - -# Show test output as it runs -core test --verbose - -# Detailed per-package coverage -core test --coverage - -# Test specific packages -core test --pkg ./pkg/... - -# Run specific test by name -core test --run TestName - -# Run tests matching pattern -core test --run "Test.*Good" - -# Skip long-running tests -core test --short - -# Enable race detector -core test --race - -# Output JSON for CI/agents -core test --json -``` - -## JSON Output - -With `--json`, outputs structured results: - -```json -{ - "passed": 14, - "failed": 0, - "skipped": 0, - "coverage": 75.1, - "exit_code": 0, - "failed_packages": [] -} -``` - -## See Also - -- [go test](../go/test/) - Go-specific test options -- [go cov](../go/cov/) - Coverage reports diff --git a/docs/cmd/vm/example.md b/docs/cmd/vm/example.md deleted file mode 100644 index f31f97e..0000000 --- a/docs/cmd/vm/example.md +++ /dev/null @@ -1,52 +0,0 @@ -# VM Examples - -## Running VMs - -```bash -# Run image -core vm run server.iso - -# Detached with resources -core vm run -d --memory 4096 --cpus 4 server.iso - -# From template -core vm run --template core-dev --var SSH_KEY="ssh-rsa AAAA..." -``` - -## Management - -```bash -# List running -core vm ps - -# Include stopped -core vm ps -a - -# Stop -core vm stop abc123 - -# View logs -core vm logs abc123 - -# Follow logs -core vm logs -f abc123 - -# Execute command -core vm exec abc123 ls -la - -# Shell -core vm exec abc123 /bin/sh -``` - -## Templates - -```bash -# List -core vm templates - -# Show content -core vm templates show core-dev - -# Show variables -core vm templates vars core-dev -``` diff --git a/docs/cmd/vm/index.md b/docs/cmd/vm/index.md deleted file mode 100644 index ec0be0f..0000000 --- a/docs/cmd/vm/index.md +++ /dev/null @@ -1,163 +0,0 @@ -# core vm - -LinuxKit VM management. - -LinuxKit VMs are lightweight, immutable VMs built from YAML templates. -They run using qemu or hyperkit depending on your system. - -## Usage - -```bash -core vm [flags] -``` - -## Commands - -| Command | Description | -|---------|-------------| -| [`run`](#vm-run) | Run a LinuxKit image or template | -| [`ps`](#vm-ps) | List running VMs | -| [`stop`](#vm-stop) | Stop a VM | -| [`logs`](#vm-logs) | View VM logs | -| [`exec`](#vm-exec) | Execute command in VM | -| [templates](templates/) | Manage LinuxKit templates | - ---- - -## vm run - -Run a LinuxKit image or build from a template. - -```bash -core vm run [flags] -core vm run --template [flags] -``` - -Supported image formats: `.iso`, `.qcow2`, `.vmdk`, `.raw` - -### Flags - -| Flag | Description | -|------|-------------| -| `--template` | Run from a LinuxKit template (build + run) | -| `--var` | Template variable in KEY=VALUE format (repeatable) | -| `--name` | Name for the container | -| `--memory` | Memory in MB (default: 1024) | -| `--cpus` | CPU count (default: 1) | -| `--ssh-port` | SSH port for exec commands (default: 2222) | -| `-d` | Run in detached mode (background) | - -### Examples - -```bash -# Run from image file -core vm run image.iso - -# Run detached with more resources -core vm run -d image.qcow2 --memory 2048 --cpus 4 - -# Run from template -core vm run --template core-dev --var SSH_KEY="ssh-rsa AAAA..." - -# Multiple template variables -core vm run --template server-php --var SSH_KEY="..." --var DOMAIN=example.com -``` - ---- - -## vm ps - -List running VMs. - -```bash -core vm ps [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `-a` | Show all (including stopped) | - -### Output - -``` -ID NAME IMAGE STATUS STARTED PID -abc12345 myvm ...core-dev.qcow2 running 5m 12345 -``` - ---- - -## vm stop - -Stop a running VM by ID or name. - -```bash -core vm stop -``` - -Supports partial ID matching. - -### Examples - -```bash -# Full ID -core vm stop abc12345678 - -# Partial ID -core vm stop abc1 -``` - ---- - -## vm logs - -View VM logs. - -```bash -core vm logs [flags] -``` - -### Flags - -| Flag | Description | -|------|-------------| -| `-f` | Follow log output | - -### Examples - -```bash -# View logs -core vm logs abc12345 - -# Follow logs -core vm logs -f abc1 -``` - ---- - -## vm exec - -Execute a command in a running VM via SSH. - -```bash -core vm exec -``` - -### Examples - -```bash -# List files -core vm exec abc12345 ls -la - -# Open shell -core vm exec abc1 /bin/sh -``` - ---- - -## See Also - -- [templates](templates/) - Manage LinuxKit templates -- [build](../build/) - Build LinuxKit images -- [dev](../dev/) - Dev environment management diff --git a/docs/cmd/vm/templates/example.md b/docs/cmd/vm/templates/example.md deleted file mode 100644 index c1f8b35..0000000 --- a/docs/cmd/vm/templates/example.md +++ /dev/null @@ -1,53 +0,0 @@ -# VM Templates Examples - -## List - -```bash -core vm templates -``` - -## Show - -```bash -core vm templates show core-dev -``` - -## Variables - -```bash -core vm templates vars core-dev -``` - -## Output - -``` -Variables for core-dev: - SSH_KEY (required) SSH public key - MEMORY (optional) Memory in MB (default: 4096) - CPUS (optional) CPU count (default: 4) -``` - -## Using Templates - -```bash -core vm run --template core-dev --var SSH_KEY="ssh-rsa AAAA..." -``` - -## Template Format - -`.core/linuxkit/myserver.yml`: - -```yaml -kernel: - image: linuxkit/kernel:5.15 - cmdline: "console=tty0" - -init: - - linuxkit/init:v1.0.0 - -services: - - name: sshd - image: linuxkit/sshd:v1.0.0 - - name: myapp - image: ghcr.io/myorg/myapp:latest -``` diff --git a/docs/cmd/vm/templates/index.md b/docs/cmd/vm/templates/index.md deleted file mode 100644 index 7ca3700..0000000 --- a/docs/cmd/vm/templates/index.md +++ /dev/null @@ -1,124 +0,0 @@ -# core vm templates - -Manage LinuxKit templates for container images. - -## Usage - -```bash -core vm templates [command] -``` - -## Commands - -| Command | Description | -|---------|-------------| -| `list` | List available templates | -| `show` | Show template details | -| `vars` | Show template variables | - -## templates list - -List all available LinuxKit templates. - -```bash -core vm templates list -``` - -### Output - -``` -Available Templates: - - core-dev - Full development environment with 100+ tools - Platforms: linux/amd64, linux/arm64 - - server-php - FrankenPHP production server - Platforms: linux/amd64, linux/arm64 - - edge-node - Minimal edge deployment - Platforms: linux/amd64, linux/arm64 -``` - -## templates show - -Show details of a specific template. - -```bash -core vm templates show -``` - -### Example - -```bash -core vm templates show core-dev -``` - -Output: -``` -Template: core-dev - -Description: Full development environment with 100+ tools - -Platforms: - - linux/amd64 - - linux/arm64 - -Formats: - - iso - - qcow2 - -Services: - - sshd - - docker - - frankenphp - -Size: ~1.8GB -``` - -## templates vars - -Show variables defined by a template. - -```bash -core vm templates vars -``` - -### Example - -```bash -core vm templates vars core-dev -``` - -Output: -``` -Variables for core-dev: - SSH_KEY (required) SSH public key - MEMORY (optional) Memory in MB (default: 4096) - CPUS (optional) CPU count (default: 4) -``` - -## Template Locations - -Templates are searched in order: - -1. `.core/linuxkit/` - Project-specific -2. `~/.core/templates/` - User templates -3. Built-in templates - -## Creating Templates - -Create a LinuxKit YAML in `.core/linuxkit/`. See [Template Format](example.md#template-format) for examples. - -Run with: - -```bash -core vm run --template myserver -``` - -## See Also - -- [vm command](../) - Run LinuxKit images -- [build command](../../build/) - Build LinuxKit images diff --git a/docs/configuration.md b/docs/configuration.md deleted file mode 100644 index 568e259..0000000 --- a/docs/configuration.md +++ /dev/null @@ -1,380 +0,0 @@ -# Configuration - -Core uses `.core/` directory for project configuration. - -## Directory Structure - -``` -.core/ -├── release.yaml # Release configuration -├── build.yaml # Build configuration (optional) -├── php.yaml # PHP configuration (optional) -└── linuxkit/ # LinuxKit templates - ├── server.yml - └── dev.yml -``` - -## release.yaml - -Full release configuration reference: - -```yaml -version: 1 - -project: - name: myapp - repository: myorg/myapp - -build: - targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - -publishers: - # GitHub Releases (required - others reference these artifacts) - - type: github - prerelease: false - draft: false - - # npm binary wrapper - - type: npm - package: "@myorg/myapp" - access: public # or "restricted" - - # Homebrew formula - - type: homebrew - tap: myorg/homebrew-tap - formula: myapp - official: - enabled: false - output: dist/homebrew - - # Scoop manifest (Windows) - - type: scoop - bucket: myorg/scoop-bucket - official: - enabled: false - output: dist/scoop - - # AUR (Arch Linux) - - type: aur - maintainer: "Name " - - # Chocolatey (Windows) - - type: chocolatey - push: false # true to publish - - # Docker multi-arch - - type: docker - registry: ghcr.io - image: myorg/myapp - dockerfile: Dockerfile - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - build_args: - VERSION: "{{.Version}}" - - # LinuxKit images - - type: linuxkit - config: .core/linuxkit/server.yml - formats: - - iso - - qcow2 - - docker - platforms: - - linux/amd64 - - linux/arm64 - -changelog: - include: - - feat - - fix - - perf - - refactor - exclude: - - chore - - docs - - style - - test - - ci -``` - -## build.yaml - -Optional build configuration: - -```yaml -version: 1 - -project: - name: myapp - binary: myapp - -build: - main: ./cmd/myapp - env: - CGO_ENABLED: "0" - flags: - - -trimpath - ldflags: - - -s -w - - -X main.version={{.Version}} - - -X main.commit={{.Commit}} - -targets: - - os: linux - arch: amd64 - - os: darwin - arch: arm64 -``` - -## php.yaml - -PHP/Laravel configuration: - -```yaml -version: 1 - -dev: - domain: myapp.test - ssl: true - port: 8000 - services: - - frankenphp - - vite - - horizon - - reverb - - redis - -test: - parallel: true - coverage: true - thresholds: - statements: 40 - branches: 35 - -deploy: - coolify: - server: https://coolify.example.com - project: my-project - environment: production -``` - -## LinuxKit Templates - -LinuxKit YAML configuration: - -```yaml -kernel: - image: linuxkit/kernel:6.6 - cmdline: "console=tty0 console=ttyS0" - -init: - - linuxkit/init:latest - - linuxkit/runc:latest - - linuxkit/containerd:latest - - linuxkit/ca-certificates:latest - -onboot: - - name: sysctl - image: linuxkit/sysctl:latest - -services: - - name: dhcpcd - image: linuxkit/dhcpcd:latest - - name: sshd - image: linuxkit/sshd:latest - - name: myapp - image: myorg/myapp:latest - capabilities: - - CAP_NET_BIND_SERVICE - -files: - - path: /etc/myapp/config.yaml - contents: | - server: - port: 8080 -``` - -## repos.yaml - -Package registry for multi-repo workspaces: - -```yaml -# Organisation name (used for GitHub URLs) -org: host-uk - -# Base path for cloning (default: current directory) -base_path: . - -# Default settings for all repos -defaults: - ci: github - license: EUPL-1.2 - branch: main - -# Repository definitions -repos: - # Foundation packages (no dependencies) - core-php: - type: foundation - description: Foundation framework - - core-devops: - type: foundation - description: Development environment - clone: false # Skip during setup (already exists) - - # Module packages (depend on foundation) - core-tenant: - type: module - depends_on: [core-php] - description: Multi-tenancy module - - core-admin: - type: module - depends_on: [core-php, core-tenant] - description: Admin panel - - core-api: - type: module - depends_on: [core-php] - description: REST API framework - - # Product packages (user-facing applications) - core-bio: - type: product - depends_on: [core-php, core-tenant] - description: Link-in-bio product - domain: bio.host.uk.com - - core-social: - type: product - depends_on: [core-php, core-tenant] - description: Social scheduling - domain: social.host.uk.com - - # Templates - core-template: - type: template - description: Starter template for new projects -``` - -### repos.yaml Fields - -| Field | Required | Description | -|-------|----------|-------------| -| `org` | Yes | GitHub organisation name | -| `base_path` | No | Directory for cloning (default: `.`) | -| `defaults` | No | Default settings applied to all repos | -| `repos` | Yes | Map of repository definitions | - -### Repository Fields - -| Field | Required | Description | -|-------|----------|-------------| -| `type` | Yes | `foundation`, `module`, `product`, or `template` | -| `description` | No | Human-readable description | -| `depends_on` | No | List of package dependencies | -| `clone` | No | Set `false` to skip during setup | -| `domain` | No | Production domain (for products) | -| `branch` | No | Override default branch | - -### Package Types - -| Type | Description | Dependencies | -|------|-------------|--------------| -| `foundation` | Core framework packages | None | -| `module` | Reusable modules | Foundation packages | -| `product` | User-facing applications | Foundation + modules | -| `template` | Starter templates | Any | - ---- - -## Environment Variables - -Complete reference of environment variables used by Core CLI. - -### Authentication - -| Variable | Used By | Description | -|----------|---------|-------------| -| `GITHUB_TOKEN` | `core ci`, `core dev` | GitHub API authentication | -| `ANTHROPIC_API_KEY` | `core ai`, `core dev claude` | Claude API key | -| `AGENTIC_TOKEN` | `core ai task*` | Agentic API authentication | -| `AGENTIC_BASE_URL` | `core ai task*` | Agentic API endpoint | - -### Publishing - -| Variable | Used By | Description | -|----------|---------|-------------| -| `NPM_TOKEN` | `core ci` (npm publisher) | npm registry auth token | -| `CHOCOLATEY_API_KEY` | `core ci` (chocolatey publisher) | Chocolatey API key | -| `DOCKER_USERNAME` | `core ci` (docker publisher) | Docker registry username | -| `DOCKER_PASSWORD` | `core ci` (docker publisher) | Docker registry password | - -### Deployment - -| Variable | Used By | Description | -|----------|---------|-------------| -| `COOLIFY_URL` | `core php deploy` | Coolify server URL | -| `COOLIFY_TOKEN` | `core php deploy` | Coolify API token | -| `COOLIFY_APP_ID` | `core php deploy` | Production application ID | -| `COOLIFY_STAGING_APP_ID` | `core php deploy --staging` | Staging application ID | - -### Build - -| Variable | Used By | Description | -|----------|---------|-------------| -| `CGO_ENABLED` | `core build`, `core go *` | Enable/disable CGO (default: 0) | -| `GOOS` | `core build` | Target operating system | -| `GOARCH` | `core build` | Target architecture | - -### Configuration Paths - -| Variable | Description | -|----------|-------------| -| `CORE_CONFIG` | Override config directory (default: `~/.core/`) | -| `CORE_REGISTRY` | Override repos.yaml path | - ---- - -## Defaults - -If no configuration exists, sensible defaults are used: - -- **Targets**: linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, windows/amd64 -- **Publishers**: GitHub only -- **Changelog**: feat, fix, perf, refactor included - -## Logging - -Logging can be configured to rotate and retain logs automatically. - -Default retention policy: -- **Max Size**: 100 MB -- **Max Backups**: 5 -- **Max Age**: 28 days - -Example configuration: - -```yaml -level: info -rotation: - filename: "app.log" - max_size: 100 # megabytes - max_backups: 5 # number of old log files to retain - max_age: 28 # days to keep old log files -``` diff --git a/docs/ecosystem.md b/docs/ecosystem.md deleted file mode 100644 index 551f7e6..0000000 --- a/docs/ecosystem.md +++ /dev/null @@ -1,457 +0,0 @@ -# Core Go Ecosystem - -The Core Go ecosystem is a set of 19 standalone Go modules that form the infrastructure backbone for the host-uk platform and the Lethean network. All modules are hosted under the `forge.lthn.ai/core/` organisation. Each module has its own repository, independent versioning, and a `docs/` directory. - -The CLI framework documented in the rest of this site (`forge.lthn.ai/core/cli`) is one node in this graph. The satellite packages listed here are separate repositories that the CLI imports or that stand alone as libraries. - ---- - -## Module Index - -| Package | Module Path | Managed By | -|---------|-------------|-----------| -| [go-inference](#go-inference) | `forge.lthn.ai/core/go-inference` | Virgil | -| [go-mlx](#go-mlx) | `forge.lthn.ai/core/go-mlx` | Virgil | -| [go-rocm](#go-rocm) | `forge.lthn.ai/core/go-rocm` | Charon | -| [go-ml](#go-ml) | `forge.lthn.ai/core/go-ml` | Virgil | -| [go-ai](#go-ai) | `forge.lthn.ai/core/go-ai` | Virgil | -| [go-agentic](#go-agentic) | `forge.lthn.ai/core/go-agentic` | Charon | -| [go-rag](#go-rag) | `forge.lthn.ai/core/go-rag` | Charon | -| [go-i18n](#go-i18n) | `forge.lthn.ai/core/go-i18n` | Virgil | -| [go-html](#go-html) | `forge.lthn.ai/core/go-html` | Charon | -| [go-crypt](#go-crypt) | `forge.lthn.ai/core/go-crypt` | Virgil | -| [go-scm](#go-scm) | `forge.lthn.ai/core/go-scm` | Charon | -| [go-p2p](#go-p2p) | `forge.lthn.ai/core/go-p2p` | Charon | -| [go-devops](#go-devops) | `forge.lthn.ai/core/go-devops` | Virgil | -| [go-help](#go-help) | `forge.lthn.ai/core/go-help` | Charon | -| [go-ratelimit](#go-ratelimit) | `forge.lthn.ai/core/go-ratelimit` | Charon | -| [go-session](#go-session) | `forge.lthn.ai/core/go-session` | Charon | -| [go-store](#go-store) | `forge.lthn.ai/core/go-store` | Charon | -| [go-ws](#go-ws) | `forge.lthn.ai/core/go-ws` | Charon | -| [go-webview](#go-webview) | `forge.lthn.ai/core/go-webview` | Charon | - ---- - -## Dependency Graph - -The graph below shows import relationships. An arrow `A → B` means A imports B. - -``` -go-inference (no dependencies — foundation contract) - ↑ - ├── go-mlx (CGO, Apple Silicon Metal GPU) - ├── go-rocm (AMD ROCm, llama-server subprocess) - └── go-ml (scoring engine, backends, orchestrator) - ↑ - └── go-ai (MCP hub, 49 tools) - ↑ - └── go-agentic (service lifecycle, allowances) - -go-rag (Qdrant + Ollama, standalone) - ↑ - └── go-ai - -go-i18n (grammar engine, standalone; Phase 2a imports go-mlx) - -go-crypt (standalone) - ↑ - ├── go-p2p (UEPS wire protocol) - └── go-scm (AgentCI dispatch) - -go-store (SQLite KV, standalone) - ↑ - ├── go-ratelimit (sliding window limiter) - ├── go-session (transcript parser) - └── go-agentic - -go-ws (WebSocket hub, standalone) - ↑ - └── go-ai - -go-webview (CDP client, standalone) - ↑ - └── go-ai - -go-html (DOM compositor, standalone) - -go-help (help catalogue, standalone) - -go-devops (Ansible, build, infrastructure — imports go-scm) -``` - -The CLI framework (`forge.lthn.ai/core/cli`) has internal equivalents of several of these packages (`pkg/rag`, `pkg/ws`, `pkg/webview`, `pkg/i18n`) that were developed in parallel. The satellite packages are the canonical standalone versions intended for use outside the CLI binary. - ---- - -## Package Descriptions - -### go-inference - -**Module:** `forge.lthn.ai/core/go-inference` - -Zero-dependency interface package that defines the common contract for all inference backends in the ecosystem: - -- `TextModel` — the top-level model interface (`Generate`, `Stream`, `Close`) -- `Backend` — hardware/runtime abstraction (Metal, ROCm, CPU, remote) -- `Token` — streaming token type with metadata - -No concrete implementations live here. Any package that needs to call inference without depending on a specific hardware library imports `go-inference` and receives an implementation at runtime. - ---- - -### go-mlx - -**Module:** `forge.lthn.ai/core/go-mlx` - -Native Metal GPU inference for Apple Silicon using CGO bindings to `mlx-c` (the C API for Apple's MLX framework). Implements the `go-inference` interfaces. - -Build requirements: -- macOS 13+ (Ventura) on Apple Silicon -- `mlx-c` installed (`brew install mlx`) -- CGO enabled: `CGO_CFLAGS` and `CGO_LDFLAGS` must reference the mlx-c headers and library - -Features: -- Loads GGUF and MLX-format models -- Streaming token generation directly on GPU -- Quantised model support (Q4, Q8) -- Phase 4 backend abstraction in progress — will allow hot-swapping backends at runtime - -Local path: `/Users/snider/Code/go-mlx` - ---- - -### go-rocm - -**Module:** `forge.lthn.ai/core/go-rocm` - -AMD ROCm GPU inference for Linux. Rather than using CGO, this package manages a `llama-server` subprocess (from llama.cpp) compiled with ROCm support and communicates over its HTTP API. - -Features: -- Subprocess lifecycle management (start, health-check, restart on crash) -- OpenAI-compatible HTTP client wrapping llama-server's API -- Implements `go-inference` interfaces -- Targeted at the homelab RX 7800 XT running Ubuntu 24.04 - -Managed by Charon (Linux homelab). - ---- - -### go-ml - -**Module:** `forge.lthn.ai/core/go-ml` - -Scoring engine, backend registry, and agent orchestration layer. The hub that connects models from `go-mlx`, `go-rocm`, and future backends into a unified interface. - -Features: -- Backend registry: register multiple inference backends, select by capability -- Scoring pipeline: evaluate model outputs against rubrics -- Agent orchestrator: coordinate multi-step inference tasks -- ~3.5K LOC - ---- - -### go-ai - -**Module:** `forge.lthn.ai/core/go-ai` - -MCP (Model Context Protocol) server hub with 49 registered tools. Acts as the primary facade for AI capabilities in the ecosystem. - -Features: -- 49 MCP tools covering file operations, RAG, metrics, process management, WebSocket, and CDP/webview -- Imports `go-ml`, `go-rag`, `go-mlx` -- Can run as stdio MCP server or TCP MCP server -- AI usage metrics recorded to JSONL - -Run the MCP server: - -```bash -# stdio (for Claude Desktop / Claude Code) -core mcp serve - -# TCP -MCP_ADDR=:9000 core mcp serve -``` - ---- - -### go-agentic - -**Module:** `forge.lthn.ai/core/go-agentic` - -Service lifecycle and allowance management for autonomous agents. Handles: - -- Agent session tracking and state persistence -- Allowance system: budget constraints on tool calls, token usage, and wall-clock time -- Integration with `go-store` for persistence -- REST client for the PHP `core-agentic` backend - -Managed by Charon. - ---- - -### go-rag - -**Module:** `forge.lthn.ai/core/go-rag` - -Retrieval-Augmented Generation pipeline using Qdrant for vector storage and Ollama for embeddings. - -Features: -- `ChunkMarkdown`: semantic splitting by H2 headers and paragraphs with overlap -- `Ingest`: crawl a directory of Markdown files, embed, and store in Qdrant -- `Query`: semantic search returning ranked `QueryResult` slices -- `FormatResultsContext`: formats results as XML tags for LLM prompt injection -- Clients: `QdrantClient` and `OllamaClient` wrapping their respective Go SDKs - -Managed by Charon. - ---- - -### go-i18n - -**Module:** `forge.lthn.ai/core/go-i18n` - -Grammar engine for natural-language generation. Goes beyond key-value lookup tables to handle pluralisation, verb conjugation, past tense, gerunds, and semantic sentence construction ("Subject verbed object"). - -Features: -- `T(key, args...)` — main translation function -- `S(noun, value)` — semantic subject with grammatical context -- Language rules defined in JSON; algorithmic fallbacks for irregular verbs -- **GrammarImprint**: a linguistic hash (reversal of the grammar engine) used as a semantic fingerprint — part of the Lethean identity verification stack -- Phase 2a (imports `go-mlx` for language model-assisted reversal) currently blocked on `go-mlx` Phase 4 - -Local path: `/Users/snider/Code/go-i18n` - ---- - -### go-html - -**Module:** `forge.lthn.ai/core/go-html` - -HLCRF DOM compositor — a programmatic HTML/DOM construction library targeting both server-side rendering and WASM (browser). - -HLCRF stands for Header, Left, Content, Right, Footer — the region layout model used throughout the CLI's terminal UI and web rendering layer. - -Features: -- Composable region-based layout (mirrors the terminal `Composite` in `pkg/cli`) -- WASM build target: runs in the browser without JavaScript -- Used by the LEM Chat UI and web SDK generation - -Managed by Charon. - ---- - -### go-crypt - -**Module:** `forge.lthn.ai/core/go-crypt` - -Cryptographic primitives, authentication, and trust policy enforcement. - -Features: -- Password hashing (Argon2id with tuned parameters) -- Symmetric encryption (ChaCha20-Poly1305, AES-GCM) -- Key derivation (HKDF, Scrypt) -- OpenPGP challenge-response authentication -- Trust policies: define and evaluate access rules -- Foundation for the UEPS (User-controlled Encryption Policy System) wire protocol in `go-p2p` - ---- - -### go-scm - -**Module:** `forge.lthn.ai/core/go-scm` - -Source control management and CI integration, including the AgentCI dispatch system. - -Features: -- Forgejo and Gitea API clients (typed wrappers) -- GitHub integration via the `gh` CLI -- `AgentCI`: dispatches AI work items to agent runners over SSH using Charm stack libraries (`soft-serve`, `keygen`, `melt`, `wishlist`) -- PR lifecycle management: create, review, merge, label -- JSONL job journal for audit trails - -Managed by Charon. - ---- - -### go-p2p - -**Module:** `forge.lthn.ai/core/go-p2p` - -Peer-to-peer mesh networking implementing the UEPS (User-controlled Encryption Policy System) wire protocol. - -Features: -- UEPS: consent-gated TLV frames with Ed25519 consent tokens and an Intent-Broker -- Peer discovery and mesh routing -- Encrypted relay transport -- Integration with `go-crypt` for all cryptographic operations - -This is a core component of the Lethean Web3 network layer. - -Managed by Charon (Linux homelab). - ---- - -### go-devops - -**Module:** `forge.lthn.ai/core/go-devops` - -Infrastructure automation, build tooling, and release pipeline utilities, intended as a standalone library form of what the Core CLI provides as commands. - -Features: -- Ansible-lite engine (native Go SSH playbook execution) -- LinuxKit image building and VM lifecycle -- Multi-target binary build and release -- Integration with `go-scm` for repository operations - ---- - -### go-help - -**Module:** `forge.lthn.ai/core/go-help` - -Embedded documentation catalogue with full-text search and an optional HTTP server for serving help content. - -Features: -- YAML-frontmatter Markdown topic parsing -- In-memory reverse index with title/heading/body scoring -- Snippet extraction with keyword highlighting -- `HTTP server` mode: serve the catalogue as a documentation site -- Used by the `core pkg search` command and the `pkg/help` package inside the CLI - -Managed by Charon. - ---- - -### go-ratelimit - -**Module:** `forge.lthn.ai/core/go-ratelimit` - -Sliding-window rate limiter with a SQLite persistence backend. - -Features: -- Token bucket and sliding-window algorithms -- SQLite backend via `go-store` for durable rate state across restarts -- HTTP middleware helper -- Used by `go-ai` and `go-agentic` to enforce per-agent API quotas - -Managed by Charon. - ---- - -### go-session - -**Module:** `forge.lthn.ai/core/go-session` - -Claude Code JSONL transcript parser and visualisation toolkit (standalone version of `pkg/session` inside the CLI). - -Features: -- `ParseTranscript(path)`: reads `.jsonl` session files and reconstructs tool use timelines -- `ListSessions(dir)`: scans a Claude projects directory for session files -- `Search(dir, query)`: full-text search across sessions -- `RenderHTML(sess, path)`: single-file HTML visualisation -- `RenderMP4(sess, path)`: terminal video replay via VHS - -Managed by Charon. - ---- - -### go-store - -**Module:** `forge.lthn.ai/core/go-store` - -SQLite-backed key-value store with reactive change notification. - -Features: -- `Get`, `Set`, `Delete`, `List` over typed keys -- `Watch(key, handler)`: register a callback that fires on change -- `OnChange(handler)`: subscribe to all changes -- Used by `go-ratelimit`, `go-session`, and `go-agentic` for lightweight persistence - -Managed by Charon. - ---- - -### go-ws - -**Module:** `forge.lthn.ai/core/go-ws` - -WebSocket hub with channel-based subscriptions and an optional Redis pub/sub bridge for multi-instance deployments. - -Features: -- Hub pattern: central registry of connected clients -- Channel routing: `SendToChannel(topic, msg)` delivers only to subscribers -- Redis bridge: publish messages from one instance, receive on all -- HTTP handler: `hub.Handler()` for embedding in any Go HTTP server -- `SendProcessOutput(id, line)`: convenience method for streaming process logs - -Managed by Charon. - ---- - -### go-webview - -**Module:** `forge.lthn.ai/core/go-webview` - -Chrome DevTools Protocol (CDP) client for browser automation, testing, and AI-driven web interaction (standalone version of `pkg/webview` inside the CLI). - -Features: -- Navigation, click, type, screenshot -- `Evaluate(script)`: arbitrary JavaScript execution with result capture -- Console capture and filtering -- Angular-aware helpers: `WaitForAngular()`, `GetNgModel(selector)` -- `ActionSequence`: chain interactions into a single call -- Used by `go-ai` to expose browser tools to MCP agents - -Managed by Charon. - ---- - -## Forge Repository Paths - -All repositories are hosted at `forge.lthn.ai` (Forgejo). SSH access uses port 2223: - -``` -ssh://git@forge.lthn.ai:2223/core/go-inference.git -ssh://git@forge.lthn.ai:2223/core/go-mlx.git -ssh://git@forge.lthn.ai:2223/core/go-rocm.git -ssh://git@forge.lthn.ai:2223/core/go-ml.git -ssh://git@forge.lthn.ai:2223/core/go-ai.git -ssh://git@forge.lthn.ai:2223/core/go-agentic.git -ssh://git@forge.lthn.ai:2223/core/go-rag.git -ssh://git@forge.lthn.ai:2223/core/go-i18n.git -ssh://git@forge.lthn.ai:2223/core/go-html.git -ssh://git@forge.lthn.ai:2223/core/go-crypt.git -ssh://git@forge.lthn.ai:2223/core/go-scm.git -ssh://git@forge.lthn.ai:2223/core/go-p2p.git -ssh://git@forge.lthn.ai:2223/core/go-devops.git -ssh://git@forge.lthn.ai:2223/core/go-help.git -ssh://git@forge.lthn.ai:2223/core/go-ratelimit.git -ssh://git@forge.lthn.ai:2223/core/go-session.git -ssh://git@forge.lthn.ai:2223/core/go-store.git -ssh://git@forge.lthn.ai:2223/core/go-ws.git -ssh://git@forge.lthn.ai:2223/core/go-webview.git -``` - -HTTPS authentication is not available on Forge. Always use SSH remotes. - ---- - -## Go Workspace Setup - -The satellite packages can be used together in a Go workspace. After cloning the repositories you need: - -```bash -go work init -go work use ./go-inference ./go-mlx ./go-rag ./go-ai # add as needed -go work sync -``` - -The CLI repository already uses a Go workspace that includes `cmd/core-gui`, `cmd/bugseti`, and `cmd/examples/*`. - ---- - -## See Also - -- [index.md](index.md) — Main documentation hub -- [getting-started.md](getting-started.md) — CLI installation -- [configuration.md](configuration.md) — `repos.yaml` registry format diff --git a/docs/examples/build-cpp.yaml b/docs/examples/build-cpp.yaml deleted file mode 100644 index 3cee856..0000000 --- a/docs/examples/build-cpp.yaml +++ /dev/null @@ -1,83 +0,0 @@ -# Example: C++ Build Configuration -# CMake + Conan 2 project using host-uk/build system - -version: 1 - -project: - name: my-cpp-project - type: cpp - description: "A C++ application" - -cpp: - standard: 17 - build_type: Release - static: false - - # Conan package manager - conan: - version: "2.21.0" - requires: - - zlib/1.3.1 - - boost/1.85.0 - - openssl/3.2.0 - tool_requires: - - cmake/3.31.9 - options: - boost/*:without_test: true - registry: - url: http://forge.snider.dev:4000/api/packages/host-uk/conan - remote: conan_build - - # CMake settings - cmake: - minimum_version: "3.16" - variables: - USE_CCACHE: "ON" - presets: - - conan-release - - conan-debug - - # Optional project-specific build options - options: - testnet: false - -# Cross-compilation targets -targets: - - os: linux - arch: x86_64 - profile: gcc-linux-x86_64 - - os: linux - arch: arm64 - profile: gcc-linux-armv8 - - os: darwin - arch: arm64 - profile: apple-clang-armv8 - - os: darwin - arch: x86_64 - profile: apple-clang-x86_64 - - os: windows - arch: x86_64 - profile: msvc-194-x86_64 - -# Packaging -package: - generators: - - TGZ - - ZIP - vendor: host-uk - contact: developers@lethean.io - website: https://lt.hn - -# Docker output -docker: - dockerfile: .core/build/docker/Dockerfile - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - build_args: - BUILD_THREADS: auto - BUILD_STATIC: "0" - BUILD_TYPE: Release diff --git a/docs/examples/build-docker-go.yaml b/docs/examples/build-docker-go.yaml deleted file mode 100644 index 4542d7a..0000000 --- a/docs/examples/build-docker-go.yaml +++ /dev/null @@ -1,42 +0,0 @@ -# Example: Go + Docker Build Configuration -# Build Go binary then containerize - -version: 1 - -project: - name: myservice - binary: myservice - -# First: build Go binary -build: - main: ./cmd/myservice - env: - CGO_ENABLED: "0" - GOOS: linux - ldflags: - - -s -w - - -X main.version={{.Version}} - -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - -# Then: build Docker image with the binary -docker: - dockerfile: Dockerfile - registry: ghcr.io - image: myorg/myservice - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - -# Dockerfile should COPY the built binary: -# -# FROM alpine:latest -# COPY myservice /usr/local/bin/myservice -# ENTRYPOINT ["/usr/local/bin/myservice"] diff --git a/docs/examples/build-docker.yaml b/docs/examples/build-docker.yaml deleted file mode 100644 index 3cd641d..0000000 --- a/docs/examples/build-docker.yaml +++ /dev/null @@ -1,40 +0,0 @@ -# Example: Docker Build Configuration -# Multi-arch container image - -version: 1 - -project: - name: myservice - type: docker - -docker: - dockerfile: Dockerfile - context: . - registry: ghcr.io - image: myorg/myservice - - platforms: - - linux/amd64 - - linux/arm64 - - tags: - - latest - - "{{.Version}}" - - "{{.Version}}-alpine" - - build_args: - APP_VERSION: "{{.Version}}" - BUILD_DATE: "{{.Date}}" - - labels: - org.opencontainers.image.source: https://github.com/myorg/myservice - org.opencontainers.image.description: My Service - org.opencontainers.image.licenses: MIT - - # Optional: build stage target - target: production - - # Optional: cache settings - cache: - from: type=gha - to: type=gha,mode=max diff --git a/docs/examples/build-full.yaml b/docs/examples/build-full.yaml deleted file mode 100644 index bd4f35c..0000000 --- a/docs/examples/build-full.yaml +++ /dev/null @@ -1,121 +0,0 @@ -# Example: Full Build Configuration -# All available options - -version: 1 - -project: - name: myapp - binary: myapp - type: auto # auto, go, wails, docker, linuxkit, php - -build: - # Go build settings - main: ./cmd/myapp - - # Environment variables - env: - CGO_ENABLED: "0" - GOFLAGS: "-mod=readonly" - - # Build flags - flags: - - -trimpath - - -v - - # Linker flags - ldflags: - - -s -w - - -X main.version={{.Version}} - - -X main.commit={{.Commit}} - - -X main.date={{.Date}} - - -X main.builtBy=core - - # Build tags - tags: - - production - - netgo - -# Build targets -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: linux - arch: "386" - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - - os: windows - arch: arm64 - - os: freebsd - arch: amd64 - -# Wails configuration (if type: wails) -wails: - frontend: ./frontend - install_cmd: install - build_cmd: build - dev_cmd: dev - -# Docker configuration (if type: docker or docker output enabled) -docker: - dockerfile: Dockerfile - context: . - registry: ghcr.io - image: myorg/myapp - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - build_args: - VERSION: "{{.Version}}" - labels: - org.opencontainers.image.source: https://github.com/myorg/myapp - target: production - cache: - from: type=gha - to: type=gha,mode=max - -# LinuxKit configuration (if type: linuxkit) -linuxkit: - config: .core/linuxkit/server.yml - formats: - - iso - - qcow2 - - docker - platforms: - - linux/amd64 - - linux/arm64 - -# Archive settings -archive: - format: tar.gz - format_windows: zip - name: "{{.Project}}-{{.Version}}-{{.OS}}-{{.Arch}}" - files: - - LICENSE - - README.md - - CHANGELOG.md - strip_parent: true - -# Checksum settings -checksum: - algorithm: sha256 - file: checksums.txt - -# Hooks -hooks: - pre_build: - - go generate ./... - - go mod tidy - post_build: - - echo "Build complete" - -# Output directory -output: dist diff --git a/docs/examples/build-go-cli.yaml b/docs/examples/build-go-cli.yaml deleted file mode 100644 index 22b21ca..0000000 --- a/docs/examples/build-go-cli.yaml +++ /dev/null @@ -1,39 +0,0 @@ -# Example: Go CLI Build Configuration -# Cross-platform CLI tool - -version: 1 - -project: - name: mycli - binary: mycli - -build: - main: ./cmd/mycli - env: - CGO_ENABLED: "0" - flags: - - -trimpath - ldflags: - - -s -w - - -X main.version={{.Version}} - - -X main.commit={{.Commit}} - - -X main.date={{.Date}} - -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - -archive: - format: tar.gz - format_windows: zip - files: - - LICENSE - - README.md diff --git a/docs/examples/build-go-library.yaml b/docs/examples/build-go-library.yaml deleted file mode 100644 index 63fe5fb..0000000 --- a/docs/examples/build-go-library.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# Example: Go Library Build Configuration -# No binary output, just validation and testing - -version: 1 - -project: - name: mylib - type: library # No binary build - -build: - # Library-specific settings - env: - CGO_ENABLED: "0" - -# Test configuration -test: - race: true - cover: true - packages: - - ./... - -# No targets needed for library -# targets: [] diff --git a/docs/examples/build-go-wails.yaml b/docs/examples/build-go-wails.yaml deleted file mode 100644 index 8a952bc..0000000 --- a/docs/examples/build-go-wails.yaml +++ /dev/null @@ -1,46 +0,0 @@ -# Example: Wails Desktop App Build Configuration -# Cross-platform desktop application with web frontend - -version: 1 - -project: - name: myapp - binary: myapp - -build: - main: . - env: - CGO_ENABLED: "1" # Required for Wails - ldflags: - - -s -w - - -X main.version={{.Version}} - -# Wails-specific configuration -wails: - frontend: ./frontend - # Auto-detects: npm, pnpm, yarn, bun - install_cmd: install - build_cmd: build - -targets: - # Desktop platforms only - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - - os: linux - arch: amd64 - -# Platform-specific packaging -package: - darwin: - - dmg - - app - windows: - - nsis - - zip - linux: - - tar.gz - - appimage diff --git a/docs/examples/build-linuxkit.yaml b/docs/examples/build-linuxkit.yaml deleted file mode 100644 index 75ebb19..0000000 --- a/docs/examples/build-linuxkit.yaml +++ /dev/null @@ -1,33 +0,0 @@ -# Example: LinuxKit Build Configuration -# Immutable Linux images - -version: 1 - -project: - name: myserver - type: linuxkit - -linuxkit: - config: .core/linuxkit/server.yml - - formats: - - iso # Bootable ISO (BIOS/EFI) - - qcow2 # QEMU/KVM/Proxmox - - raw # Raw disk image - - vmdk # VMware - - docker # Docker-loadable tarball - - platforms: - - linux/amd64 - - linux/arm64 - - # Output naming - name: "{{.Project}}-{{.Version}}" - -# The linuxkit config file (.core/linuxkit/server.yml) defines: -# - kernel version -# - init system -# - services to run -# - files to include -# -# See linuxkit-server.yml example diff --git a/docs/examples/build-minimal.yaml b/docs/examples/build-minimal.yaml deleted file mode 100644 index 9801947..0000000 --- a/docs/examples/build-minimal.yaml +++ /dev/null @@ -1,7 +0,0 @@ -# Example: Minimal Build Configuration -# Auto-detects everything from project structure - -version: 1 - -project: - name: myapp diff --git a/docs/examples/build-multi-binary.yaml b/docs/examples/build-multi-binary.yaml deleted file mode 100644 index 563a357..0000000 --- a/docs/examples/build-multi-binary.yaml +++ /dev/null @@ -1,51 +0,0 @@ -# Example: Multi-Binary Build Configuration -# Multiple binaries from one repository - -version: 1 - -project: - name: mytools - -# Multiple build targets -builds: - - name: cli - binary: mytool - main: ./cmd/mytool - ldflags: - - -s -w - - -X main.version={{.Version}} - - - name: server - binary: myserver - main: ./cmd/server - ldflags: - - -s -w - - -X main.version={{.Version}} - - - name: worker - binary: myworker - main: ./cmd/worker - ldflags: - - -s -w - -# Shared settings -build: - env: - CGO_ENABLED: "0" - flags: - - -trimpath - -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: arm64 - -# Archive includes all binaries -archive: - format: tar.gz - files: - - LICENSE - - README.md diff --git a/docs/examples/build-php-laravel.yaml b/docs/examples/build-php-laravel.yaml deleted file mode 100644 index ae23cad..0000000 --- a/docs/examples/build-php-laravel.yaml +++ /dev/null @@ -1,50 +0,0 @@ -# Example: PHP/Laravel Build Configuration -# FrankenPHP container with Laravel app - -version: 1 - -project: - name: mylaravel - type: php - -php: - version: "8.4" - - # Composer settings - composer: - install_args: - - --no-dev - - --optimize-autoloader - - --no-interaction - - # Frontend build - frontend: - enabled: true - build_cmd: "npm run build" - - # Octane configuration - octane: - server: frankenphp - workers: auto - max_requests: 500 - -# Docker output -docker: - dockerfile: Dockerfile - registry: ghcr.io - image: myorg/mylaravel - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - build_args: - PHP_VERSION: "8.4" - -# Optional: LinuxKit for immutable deployment -linuxkit: - config: .core/linuxkit/server-php.yml - formats: - - qcow2 - - iso diff --git a/docs/examples/linuxkit-docker.yml b/docs/examples/linuxkit-docker.yml deleted file mode 100644 index 416ede1..0000000 --- a/docs/examples/linuxkit-docker.yml +++ /dev/null @@ -1,29 +0,0 @@ -# Example: LinuxKit Docker Format -# Build immutable container that loads with `docker load` - -kernel: - image: linuxkit/kernel:6.6 - cmdline: "console=tty0" - -init: - - linuxkit/init:latest - - linuxkit/runc:latest - - linuxkit/containerd:latest - -services: - - name: myservice - image: ghcr.io/myorg/myservice:latest - -# Use in release.yaml: -# -# publishers: -# - type: linuxkit -# config: .core/linuxkit/docker-format.yml -# formats: -# - docker # Outputs .docker.tar -# platforms: -# - linux/amd64 -# - linux/arm64 -# -# Load the image: -# docker load < linuxkit-v1.0.0-amd64.docker.tar diff --git a/docs/examples/linuxkit-server.yml b/docs/examples/linuxkit-server.yml deleted file mode 100644 index 7727ca6..0000000 --- a/docs/examples/linuxkit-server.yml +++ /dev/null @@ -1,51 +0,0 @@ -# Example: LinuxKit Server Configuration -# Minimal immutable Linux server with your application - -kernel: - image: linuxkit/kernel:6.6 - cmdline: "console=tty0 console=ttyS0" - -init: - - linuxkit/init:latest - - linuxkit/runc:latest - - linuxkit/containerd:latest - - linuxkit/ca-certificates:latest - -onboot: - - name: sysctl - image: linuxkit/sysctl:latest - - name: dhcpcd - image: linuxkit/dhcpcd:latest - -services: - # SSH for management - - name: sshd - image: linuxkit/sshd:latest - binds: - - /etc/ssh/authorized_keys:/root/.ssh/authorized_keys - - # Your application - - name: myapp - image: ghcr.io/myorg/myapp:latest - capabilities: - - CAP_NET_BIND_SERVICE - binds: - - /etc/myapp:/etc/myapp:ro - -files: - # SSH authorized keys - - path: /etc/ssh/authorized_keys - mode: "0600" - contents: | - ssh-ed25519 AAAA... your-key - - # Application config - - path: /etc/myapp/config.yaml - mode: "0644" - contents: | - server: - host: 0.0.0.0 - port: 8080 - database: - host: ${DB_HOST:-localhost} - port: ${DB_PORT:-5432} diff --git a/docs/examples/publish-all.yaml b/docs/examples/publish-all.yaml deleted file mode 100644 index a8f6b51..0000000 --- a/docs/examples/publish-all.yaml +++ /dev/null @@ -1,68 +0,0 @@ -# Example: All Publishers Combined -# Use in .core/release.yaml publishers array - -publishers: - # 1. GitHub - always first (others reference these artifacts) - - type: github - prerelease: false - draft: false - - # 2. npm - JavaScript ecosystem - - type: npm - package: "@myorg/mycli" - access: public - - # 3. Homebrew - macOS/Linux - - type: homebrew - tap: myorg/homebrew-tap - official: - enabled: true - output: dist/homebrew - - # 4. Scoop - Windows - - type: scoop - bucket: myorg/scoop-bucket - official: - enabled: true - output: dist/scoop - - # 5. AUR - Arch Linux - - type: aur - maintainer: "Your Name " - - # 6. Chocolatey - Windows enterprise - - type: chocolatey - push: false - - # 7. Docker - Containers - - type: docker - registry: ghcr.io - image: myorg/mycli - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - - # 8. LinuxKit - Immutable infrastructure - - type: linuxkit - config: .core/linuxkit/server.yml - formats: - - iso - - qcow2 - - docker - platforms: - - linux/amd64 - - linux/arm64 - -# Required environment variables: -# GITHUB_TOKEN - via gh CLI auth -# NPM_TOKEN - npm publish -# CHOCOLATEY_API_KEY - if push: true -# -# Required tools: -# gh - GitHub CLI -# npm - Node package manager -# docker - Docker with buildx -# linuxkit - LinuxKit CLI diff --git a/docs/examples/publish-aur.yaml b/docs/examples/publish-aur.yaml deleted file mode 100644 index 711c291..0000000 --- a/docs/examples/publish-aur.yaml +++ /dev/null @@ -1,28 +0,0 @@ -# Example: AUR Publisher (Arch Linux) -# PKGBUILD generation and AUR push - -type: aur - -# Package name (will be suffixed with -bin) -package: mycli - -# Maintainer info (required by AUR) -maintainer: "Your Name " - -# Generate files only (don't push to AUR) -official: - enabled: true - output: dist/aur - -# Environment: SSH key for aur.archlinux.org -# -# Usage after publish: -# yay -S mycli-bin -# # or -# paru -S mycli-bin -# -# Generated files: -# - PKGBUILD -# - .SRCINFO -# -# Supports both x86_64 and aarch64 diff --git a/docs/examples/publish-chocolatey.yaml b/docs/examples/publish-chocolatey.yaml deleted file mode 100644 index 358ab39..0000000 --- a/docs/examples/publish-chocolatey.yaml +++ /dev/null @@ -1,29 +0,0 @@ -# Example: Chocolatey Publisher (Windows) -# NuSpec package for Windows enterprise - -type: chocolatey - -# Package name -package: mycli - -# Push to Chocolatey community repo -push: false # Set true to auto-publish - -# Generate files only -official: - enabled: true - output: dist/chocolatey - -# Environment: CHOCOLATEY_API_KEY required if push: true -# -# Usage after publish: -# choco install mycli -# -# Generated files: -# - mycli.nuspec -# - tools/chocolateyinstall.ps1 -# -# Manual publish: -# cd dist/chocolatey -# choco pack -# choco push mycli.1.0.0.nupkg --source https://push.chocolatey.org/ diff --git a/docs/examples/publish-docker.yaml b/docs/examples/publish-docker.yaml deleted file mode 100644 index d3d9063..0000000 --- a/docs/examples/publish-docker.yaml +++ /dev/null @@ -1,38 +0,0 @@ -# Example: Docker Publisher -# Multi-arch container images - -type: docker - -# Registry (default: ghcr.io) -registry: ghcr.io - -# Image name -image: myorg/myapp - -# Dockerfile path (default: Dockerfile) -dockerfile: Dockerfile - -# Target platforms -platforms: - - linux/amd64 - - linux/arm64 - -# Image tags -tags: - - latest - - "{{.Version}}" - - "{{.Version}}-alpine" - -# Build arguments -build_args: - VERSION: "{{.Version}}" - BUILD_DATE: "{{.Date}}" - -# Environment: Docker login to registry -# -# For ghcr.io: -# echo $GITHUB_TOKEN | docker login ghcr.io -u USERNAME --password-stdin -# -# Usage after publish: -# docker pull ghcr.io/myorg/myapp:latest -# docker run ghcr.io/myorg/myapp:v1.0.0 diff --git a/docs/examples/publish-github.yaml b/docs/examples/publish-github.yaml deleted file mode 100644 index 0fa37b3..0000000 --- a/docs/examples/publish-github.yaml +++ /dev/null @@ -1,14 +0,0 @@ -# Example: GitHub Releases Publisher -# Foundation publisher - others reference these artifacts - -type: github - -# Release settings -prerelease: false -draft: false - -# Auto-detect from git tag, or override -# version: v1.0.0 - -# Auto-detect from git remote, or specify -# repository: myorg/myapp diff --git a/docs/examples/publish-homebrew.yaml b/docs/examples/publish-homebrew.yaml deleted file mode 100644 index c186068..0000000 --- a/docs/examples/publish-homebrew.yaml +++ /dev/null @@ -1,29 +0,0 @@ -# Example: Homebrew Publisher -# Formula generation and tap management - -type: homebrew - -# Your tap repository -tap: myorg/homebrew-tap - -# Formula name (defaults to project name) -formula: mycli - -# Generate files for official homebrew-core PR -official: - enabled: true - output: dist/homebrew - -# Environment: Uses gh CLI authentication -# -# Usage after publish: -# brew tap myorg/tap -# brew install mycli -# -# Or directly: -# brew install myorg/tap/mycli -# -# Generated formula includes: -# - Multi-platform support (macOS Intel/ARM, Linux) -# - SHA256 checksums from GitHub release -# - Version from git tag diff --git a/docs/examples/publish-linuxkit.yaml b/docs/examples/publish-linuxkit.yaml deleted file mode 100644 index c83fb2a..0000000 --- a/docs/examples/publish-linuxkit.yaml +++ /dev/null @@ -1,36 +0,0 @@ -# Example: LinuxKit Publisher -# Immutable Linux images uploaded to GitHub release - -type: linuxkit - -# LinuxKit YAML configuration -config: .core/linuxkit/server.yml - -# Output formats -formats: - - iso # Bootable ISO (bare metal, VMs) - - qcow2 # QEMU/KVM/Proxmox - - raw # Raw disk image - - vmdk # VMware - - docker # Docker-loadable tarball - -# Target platforms -platforms: - - linux/amd64 - - linux/arm64 - -# Environment: linuxkit CLI installed -# -# Artifacts uploaded to GitHub release: -# - myapp-v1.0.0-amd64.iso -# - myapp-v1.0.0-amd64.qcow2 -# - myapp-v1.0.0-amd64.docker.tar -# - myapp-v1.0.0-arm64.iso -# - ... -# -# Usage: -# # Boot ISO -# qemu-system-x86_64 -cdrom myapp-v1.0.0-amd64.iso -m 1024 -# -# # Load Docker image -# docker load < myapp-v1.0.0-amd64.docker.tar diff --git a/docs/examples/publish-npm.yaml b/docs/examples/publish-npm.yaml deleted file mode 100644 index a34a912..0000000 --- a/docs/examples/publish-npm.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Example: npm Publisher -# Binary wrapper pattern - downloads correct platform binary on install - -type: npm - -# Package name (scoped recommended) -package: "@myorg/mycli" - -# Access level -access: public # or "restricted" for private - -# Environment: NPM_TOKEN required -# -# Usage after publish: -# npm install -g @myorg/mycli -# npx @myorg/mycli --help -# -# The published package contains: -# - package.json -# - install.js (postinstall downloads binary) -# - bin/run.js (wrapper that executes binary) diff --git a/docs/examples/publish-scoop.yaml b/docs/examples/publish-scoop.yaml deleted file mode 100644 index a5c975a..0000000 --- a/docs/examples/publish-scoop.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# Example: Scoop Publisher (Windows) -# JSON manifest for Windows package manager - -type: scoop - -# Your bucket repository -bucket: myorg/scoop-bucket - -# Generate files for official scoop-main PR -official: - enabled: true - output: dist/scoop - -# Environment: Uses gh CLI authentication -# -# Usage after publish: -# scoop bucket add myorg https://github.com/myorg/scoop-bucket -# scoop install mycli -# -# Generated manifest includes: -# - 64-bit and ARM64 Windows support -# - SHA256 checksums -# - Auto-update configuration diff --git a/docs/examples/release-full.yaml b/docs/examples/release-full.yaml deleted file mode 100644 index facc635..0000000 --- a/docs/examples/release-full.yaml +++ /dev/null @@ -1,98 +0,0 @@ -# Example: Full Release Configuration -# Complete configuration with all publishers - -version: 1 - -project: - name: core - repository: host-uk/core - -build: - targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - - os: windows - arch: arm64 - -publishers: - # 1. GitHub Releases - always first, others reference these - - type: github - prerelease: false - draft: false - - # 2. npm - JavaScript ecosystem - - type: npm - package: "@host-uk/core" - access: public - - # 3. Homebrew - macOS/Linux - - type: homebrew - tap: host-uk/homebrew-tap - formula: core - # Generate files for official homebrew-core PR - official: - enabled: true - output: dist/homebrew - - # 4. Scoop - Windows - - type: scoop - bucket: host-uk/scoop-bucket - # Generate files for official scoop-main PR - official: - enabled: true - output: dist/scoop - - # 5. AUR - Arch Linux - - type: aur - maintainer: "Host UK " - - # 6. Chocolatey - Windows enterprise - - type: chocolatey - push: false # Manual review before push - - # 7. Docker - Container deployment - - type: docker - registry: ghcr.io - image: host-uk/core - dockerfile: Dockerfile - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - - "{{.Version}}-alpine" - - # 8. LinuxKit - Immutable infrastructure - - type: linuxkit - config: .core/linuxkit/core-server.yml - formats: - - iso # Bootable ISO for bare metal - - qcow2 # QEMU/KVM/Proxmox - - docker # Immutable container - platforms: - - linux/amd64 - - linux/arm64 - -changelog: - include: - - feat # New features - - fix # Bug fixes - - perf # Performance improvements - - refactor # Code refactoring - - security # Security fixes - exclude: - - chore - - docs - - style - - test - - ci - - build diff --git a/docs/examples/release-go-cli.yaml b/docs/examples/release-go-cli.yaml deleted file mode 100644 index 24fcec2..0000000 --- a/docs/examples/release-go-cli.yaml +++ /dev/null @@ -1,59 +0,0 @@ -# Example: Go CLI Release Configuration -# Publishes to GitHub, npm, Homebrew, Scoop, AUR, and Chocolatey - -version: 1 - -project: - name: mycli - repository: myorg/mycli - -build: - targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - -publishers: - # GitHub Releases - foundation for all other publishers - - type: github - prerelease: false - draft: false - - # npm - binary wrapper pattern - # Users install via: npm install -g @myorg/mycli - - type: npm - package: "@myorg/mycli" - access: public - - # Homebrew - tap repository - # Users install via: brew install myorg/tap/mycli - - type: homebrew - tap: myorg/homebrew-tap - - # Scoop - Windows package manager - # Users install via: scoop bucket add myorg https://github.com/myorg/scoop-bucket && scoop install mycli - - type: scoop - bucket: myorg/scoop-bucket - - # AUR - Arch Linux User Repository - # Users install via: yay -S mycli-bin - - type: aur - maintainer: "Your Name " - - # Chocolatey - Windows enterprise - # Users install via: choco install mycli - - type: chocolatey - push: false # Set true to auto-publish - -changelog: - include: - - feat - - fix - - perf diff --git a/docs/examples/release-go-wails.yaml b/docs/examples/release-go-wails.yaml deleted file mode 100644 index c4d5eaf..0000000 --- a/docs/examples/release-go-wails.yaml +++ /dev/null @@ -1,36 +0,0 @@ -# Example: Wails Desktop App Release Configuration -# Builds cross-platform desktop app and publishes to GitHub - -version: 1 - -project: - name: myapp - repository: myorg/myapp - -build: - targets: - - os: darwin - arch: amd64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 - - os: linux - arch: amd64 - -publishers: - - type: github - prerelease: false - draft: true # Review before publishing - - # Homebrew cask for macOS - - type: homebrew - tap: myorg/homebrew-tap - formula: myapp - -changelog: - include: - - feat - - fix - - perf - - ui # Custom type for UI changes diff --git a/docs/examples/release-minimal.yaml b/docs/examples/release-minimal.yaml deleted file mode 100644 index 1f328d4..0000000 --- a/docs/examples/release-minimal.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# Example: Minimal Release Configuration -# Just GitHub releases with defaults - -version: 1 - -project: - name: myapp - repository: myorg/myapp - -publishers: - - type: github diff --git a/docs/examples/release-official-repos.yaml b/docs/examples/release-official-repos.yaml deleted file mode 100644 index 1c36d7a..0000000 --- a/docs/examples/release-official-repos.yaml +++ /dev/null @@ -1,51 +0,0 @@ -# Example: Generate Files for Official Repository PRs -# Creates files for PRs to homebrew-core, scoop-main, etc. - -version: 1 - -project: - name: myapp - repository: myorg/myapp - -publishers: - - type: github - - # Generate formula for homebrew-core PR - # Output: dist/homebrew/myapp.rb - - type: homebrew - tap: myorg/homebrew-tap # Also push to own tap - official: - enabled: true - output: dist/homebrew - - # Generate manifest for scoop-main PR - # Output: dist/scoop/myapp.json - - type: scoop - bucket: myorg/scoop-bucket # Also push to own bucket - official: - enabled: true - output: dist/scoop - - # Generate files for AUR - # Output: dist/aur/PKGBUILD, dist/aur/.SRCINFO - - type: aur - maintainer: "Your Name " - official: - enabled: true - output: dist/aur - -# After release, submit PRs: -# -# Homebrew: -# cd homebrew-core -# cp ../myapp/dist/homebrew/myapp.rb Formula/m/myapp.rb -# git checkout -b myapp-1.0.0 -# git add . && git commit -m "myapp 1.0.0 (new formula)" -# gh pr create -# -# Scoop: -# cd Main -# cp ../myapp/dist/scoop/myapp.json bucket/myapp.json -# git checkout -b myapp-1.0.0 -# git add . && git commit -m "myapp: Add version 1.0.0" -# gh pr create diff --git a/docs/examples/release-php-laravel.yaml b/docs/examples/release-php-laravel.yaml deleted file mode 100644 index 8ebdbae..0000000 --- a/docs/examples/release-php-laravel.yaml +++ /dev/null @@ -1,42 +0,0 @@ -# Example: PHP/Laravel Release Configuration -# Builds Docker container and LinuxKit image - -version: 1 - -project: - name: mylaravel - repository: myorg/mylaravel - -publishers: - - type: github - prerelease: false - - # Docker container for deployment - - type: docker - registry: ghcr.io - image: myorg/mylaravel - dockerfile: Dockerfile - platforms: - - linux/amd64 - - linux/arm64 - tags: - - latest - - "{{.Version}}" - build_args: - PHP_VERSION: "8.4" - APP_ENV: production - - # LinuxKit for immutable server deployment - - type: linuxkit - config: .core/linuxkit/server-php.yml - formats: - - iso - - qcow2 - platforms: - - linux/amd64 - -changelog: - include: - - feat - - fix - - security diff --git a/docs/examples/sdk-full.yaml b/docs/examples/sdk-full.yaml deleted file mode 100644 index 52e1f5a..0000000 --- a/docs/examples/sdk-full.yaml +++ /dev/null @@ -1,43 +0,0 @@ -# Example: Full SDK Configuration -# Generate typed API clients from OpenAPI specs - -sdk: - # OpenAPI spec source (auto-detected if omitted) - spec: api/openapi.yaml - - # Languages to generate - languages: - - typescript - - python - - go - - php - - # Output directory (default: sdk/) - output: sdk/ - - # Package naming - package: - name: myapi - version: "{{.Version}}" - - # Breaking change detection - diff: - enabled: true - fail_on_breaking: true # CI fails on breaking changes - - # Optional: publish to monorepo - publish: - repo: myorg/sdks - path: packages/myapi - -# Required tools (install one per language): -# TypeScript: npm i -g openapi-typescript-codegen (or Docker) -# Python: pip install openapi-python-client (or Docker) -# Go: go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@latest -# PHP: Docker required -# -# Usage: -# core sdk generate # Generate all configured languages -# core sdk generate --lang go # Generate single language -# core sdk diff --base v1.0.0 # Check for breaking changes -# core sdk validate # Validate spec diff --git a/docs/faq.md b/docs/faq.md deleted file mode 100644 index 2c3b6e4..0000000 --- a/docs/faq.md +++ /dev/null @@ -1,97 +0,0 @@ -# Frequently Asked Questions (FAQ) - -Common questions and answers about the Core CLI and Framework. - -## General - -### What is Core? - -Core is a unified CLI and framework for building and managing Go, PHP, and Wails applications. It provides an opinionated set of tools for development, testing, building, and releasing projects within the host-uk ecosystem. - -### Is Core a CLI or a Framework? - -It is both. The Core Framework (`pkg/core`) is a library for building Go desktop applications with Wails. The Core CLI (`cmd/core`) is the tool you use to manage projects, run tests, build binaries, and handle multi-repository workspaces. - ---- - -## Installation - -### How do I install the Core CLI? - -The recommended way is via Go: - -```bash -go install forge.lthn.ai/core/cli/cmd/core@latest -``` - -Ensure your Go bin directory is in your PATH. See [Getting Started](getting-started.md) for more options. - -### I get "command not found: core" after installation. - -This usually means your Go bin directory is not in your system's PATH. Add it by adding this to your shell profile (`.bashrc`, `.zshrc`, etc.): - -```bash -export PATH="$PATH:$(go env GOPATH)/bin" -``` - ---- - -## Usage - -### Why does `core ci` not publish anything by default? - -Core is designed to be **safe by default**. `core ci` runs in dry-run mode to show you what would be published. To actually publish a release, you must use the `--we-are-go-for-launch` flag: - -```bash -core ci --we-are-go-for-launch -``` - -### How do I run tests for only one package? - -You can pass standard Go test flags to `core go test`: - -```bash -core go test ./pkg/my-package -``` - -### What is `core doctor` for? - -`core doctor` checks your development environment to ensure all required tools (Go, Git, Docker, etc.) are installed and correctly configured. It's the first thing you should run if something isn't working. - ---- - -## Configuration - -### Where is Core's configuration stored? - -- **Project-specific**: In the `.core/` directory within your project root. -- **Global**: In `~/.core/` or as defined by `CORE_CONFIG`. -- **Registry**: The `repos.yaml` file defines the multi-repo workspace. - -### How do I change the build targets? - -You can specify targets in `.core/release.yaml` or use the `--targets` flag with the `core build` command: - -```bash -core build --targets linux/amd64,darwin/arm64 -``` - ---- - -## Workspaces and Registry - -### What is a "workspace" in Core? - -In the context of the CLI, a workspace is a directory containing multiple repositories defined in a `repos.yaml` file. The `core dev` commands allow you to manage status, commits, and synchronization across all repositories in the workspace at once. - -### What is `repos.yaml`? - -`repos.yaml` is the "registry" for your workspace. It lists the repositories, their types (foundation, module, product), and their dependencies. Core uses this file to know which repositories to clone during `core setup`. - ---- - -## See Also - -- [Getting Started](getting-started.md) - Installation and first steps -- [User Guide](user-guide.md) - Detailed usage information -- [Troubleshooting](troubleshooting.md) - Solving common issues diff --git a/docs/getting-started.md b/docs/getting-started.md deleted file mode 100644 index 486dbfa..0000000 --- a/docs/getting-started.md +++ /dev/null @@ -1,191 +0,0 @@ -# Getting Started - -This guide walks you through installing Core and running your first build. - -## Prerequisites - -Before installing Core, ensure you have: - -| Tool | Minimum Version | Check Command | -|------|-----------------|---------------| -| Go | 1.23+ | `go version` | -| Git | 2.30+ | `git --version` | - -Optional (for specific features): - -| Tool | Required For | Install | -|------|--------------|---------| -| `gh` | GitHub integration (`core dev issues`, `core dev reviews`) | [cli.github.com](https://cli.github.com) | -| Docker | Container builds | [docker.com](https://docker.com) | -| `task` | Task automation | `go install github.com/go-task/task/v3/cmd/task@latest` | - -## Installation - -### Option 1: Go Install (Recommended) - -```bash -# Install latest release -go install forge.lthn.ai/core/cli/cmd/core@latest - -# Verify installation -core doctor -``` - -If `core: command not found`, add Go's bin directory to your PATH: - -```bash -export PATH="$PATH:$(go env GOPATH)/bin" -``` - -### Option 2: Download Binary - -Download pre-built binaries from [GitHub Releases](https://forge.lthn.ai/core/cli/releases): - -```bash -# macOS (Apple Silicon) -curl -Lo core https://forge.lthn.ai/core/cli/releases/latest/download/core-darwin-arm64 -chmod +x core -sudo mv core /usr/local/bin/ - -# macOS (Intel) -curl -Lo core https://forge.lthn.ai/core/cli/releases/latest/download/core-darwin-amd64 -chmod +x core -sudo mv core /usr/local/bin/ - -# Linux (x86_64) -curl -Lo core https://forge.lthn.ai/core/cli/releases/latest/download/core-linux-amd64 -chmod +x core -sudo mv core /usr/local/bin/ -``` - -### Option 3: Build from Source - -```bash -# Clone repository -git clone https://forge.lthn.ai/core/cli.git -cd core - -# Build with Task (recommended) -task cli:build -# Binary at ./bin/core - -# Or build with Go directly -CGO_ENABLED=0 go build -o core ./cmd/core/ -sudo mv core /usr/local/bin/ -``` - -## Your First Build - -### 1. Navigate to a Go Project - -```bash -cd ~/Code/my-go-project -``` - -### 2. Initialise Configuration - -```bash -core setup -``` - -This detects your project type and creates configuration files in `.core/`: -- `build.yaml` - Build settings -- `release.yaml` - Release configuration -- `test.yaml` - Test commands - -### 3. Build - -```bash -core build -``` - -Output appears in `dist/`: - -``` -dist/ -├── my-project-darwin-arm64.tar.gz -├── my-project-linux-amd64.tar.gz -└── CHECKSUMS.txt -``` - -### 4. Cross-Compile (Optional) - -```bash -core build --targets linux/amd64,linux/arm64,darwin/arm64,windows/amd64 -``` - -## Your First Release - -Releases are **safe by default** - Core runs in dry-run mode unless you explicitly confirm. - -### 1. Preview - -```bash -core ci -``` - -This shows what would be published without actually publishing. - -### 2. Publish - -```bash -core ci --we-are-go-for-launch -``` - -This creates a GitHub release with your built artifacts. - -## Multi-Repo Workflow - -If you work with multiple repositories (like the host-uk ecosystem): - -### 1. Clone All Repositories - -```bash -mkdir host-uk && cd host-uk -core setup -``` - -Select packages in the interactive wizard. - -### 2. Check Status - -```bash -core dev health -# Output: "18 repos │ clean │ synced" -``` - -### 3. Work Across Repos - -```bash -core dev work --status # See status table -core dev work # Commit and push all dirty repos -``` - -## Next Steps - -| Task | Command | Documentation | -|------|---------|---------------| -| Run tests | `core go test` | [go/test](cmd/go/test/) | -| Format code | `core go fmt --fix` | [go/fmt](cmd/go/fmt/) | -| Lint code | `core go lint` | [go/lint](cmd/go/lint/) | -| PHP development | `core php dev` | [php](cmd/php/) | -| View all commands | `core --help` | [cmd](cmd/) | - -## Getting Help - -```bash -# Check environment -core doctor - -# Command help -core --help - -# Full documentation -https://forge.lthn.ai/core/cli/tree/main/docs -``` - -## See Also - -- [Configuration](configuration.md) - All config options -- [Workflows](workflows.md) - Common task sequences -- [Troubleshooting](troubleshooting.md) - When things go wrong diff --git a/docs/glossary.md b/docs/glossary.md deleted file mode 100644 index ea9d280..0000000 --- a/docs/glossary.md +++ /dev/null @@ -1,112 +0,0 @@ -# Glossary - -Definitions of terms used throughout Core CLI documentation. - -## A - -### Artifact -A file produced by a build, typically a binary, archive, or checksum file. Artifacts are stored in the `dist/` directory and published during releases. - -## C - -### CGO -Go's mechanism for calling C code. Core disables CGO by default (`CGO_ENABLED=0`) to produce statically-linked binaries that don't depend on system libraries. - -### Changelog -Automatically generated list of changes between releases, created from conventional commit messages. Configure in `.core/release.yaml`. - -### Conventional Commits -A commit message format: `type(scope): description`. Types include `feat`, `fix`, `docs`, `chore`. Core uses this to generate changelogs. - -## D - -### Dry-run -A mode where commands show what they would do without actually doing it. `core ci` runs in dry-run mode by default for safety. - -## F - -### Foundation Package -A core package with no dependencies on other packages. Examples: `core-php`, `core-devops`. These form the base of the dependency tree. - -### FrankenPHP -A modern PHP application server used by `core php dev`. Combines PHP with Caddy for high-performance serving. - -## G - -### `gh` -The GitHub CLI tool. Required for commands that interact with GitHub: `core dev issues`, `core dev reviews`, `core dev ci`. - -## L - -### LinuxKit -A toolkit for building lightweight, immutable Linux distributions. Core can build LinuxKit images via `core build --type linuxkit`. - -## M - -### Module (Go) -A collection of Go packages with a `go.mod` file. Core's Go commands operate on modules. - -### Module (Package) -A host-uk package that depends on foundation packages. Examples: `core-tenant`, `core-admin`. Compare with **Foundation Package** and **Product**. - -## P - -### Package -An individual repository in the host-uk ecosystem. Packages are defined in `repos.yaml` and managed with `core pkg` commands. - -### Package Index -The `repos.yaml` file that lists all packages in a workspace. Contains metadata like dependencies, type, and description. - -### Product -A user-facing application package. Examples: `core-bio`, `core-social`. Products depend on foundation and module packages. - -### Publisher -A release target configured in `.core/release.yaml`. Types include `github`, `docker`, `npm`, `homebrew`, `linuxkit`. - -## R - -### Registry (Docker/npm) -A remote repository for container images or npm packages. Core can publish to registries during releases. - -### `repos.yaml` -The package index file defining all repositories in a workspace. Used by multi-repo commands like `core dev work`. - -## S - -### SDK -Software Development Kit. Core can generate API client SDKs from OpenAPI specs via `core build sdk`. - -## T - -### Target -A build target specified as `os/arch`, e.g., `linux/amd64`, `darwin/arm64`. Use `--targets` flag to specify. - -## W - -### Wails -A framework for building desktop applications with Go backends and web frontends. Core detects Wails projects and uses appropriate build commands. - -### Workspace (Go) -A Go 1.18+ feature for working with multiple modules simultaneously. Managed via `core go work` commands. - -### Workspace (Multi-repo) -A directory containing multiple packages from `repos.yaml`. Created via `core setup` and managed with `core dev` commands. - -## Symbols - -### `.core/` -Directory containing project configuration files: -- `build.yaml` - Build settings -- `release.yaml` - Release targets -- `test.yaml` - Test configuration -- `linuxkit/` - LinuxKit templates - -### `--we-are-go-for-launch` -Flag to disable dry-run mode and actually publish a release. Named as a deliberate friction to prevent accidental releases. - ---- - -## See Also - -- [Configuration](configuration.md) - Config file reference -- [Getting Started](getting-started.md) - First-time setup diff --git a/docs/index.md b/docs/index.md index 6f9c30a..064ef6f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,207 +1,70 @@ # Core Go Framework — Documentation -Core is a Go framework and unified CLI for the host-uk ecosystem. It provides two complementary things: a **dependency injection container** for building Go services and Wails v3 desktop applications, and a **command-line tool** for managing the full development lifecycle across Go, PHP, and container workloads. +Core (`forge.lthn.ai/core/go`) is a **dependency injection and service lifecycle framework** for Go. It provides a typed service registry, lifecycle hooks, and a message-passing bus for decoupled communication between services. -The `core` binary is the single entry point for all development tasks: testing, building, releasing, multi-repo management, MCP servers, and AI-assisted workflows. - ---- - -## Getting Started - -| Document | Description | -|----------|-------------| -| [Getting Started](getting-started.md) | Install the CLI, run your first build, and set up a multi-repo workspace | -| [User Guide](user-guide.md) | Key concepts and daily workflow patterns | -| [Workflows](workflows.md) | End-to-end task sequences for common scenarios | -| [FAQ](faq.md) | Answers to common questions | - ---- - -## Architecture - -| Document | Description | -|----------|-------------| -| [Package Standards](pkg/PACKAGE_STANDARDS.md) | Canonical patterns for creating packages: Service struct, factory, IPC, thread safety | -| [pkg/i18n — Grammar](pkg/i18n/GRAMMAR.md) | Grammar engine internals and language rule format | -| [pkg/i18n — Extending](pkg/i18n/EXTENDING.md) | How to add new locales and translation files | - -### Framework Architecture Summary - -The Core framework (`pkg/framework`) is a dependency injection container built around three ideas: - -**Service registry.** Services are registered via factory functions and retrieved with type-safe generics: - -```go -core, _ := framework.New( - framework.WithService(mypackage.NewService(opts)), -) -svc, _ := framework.ServiceFor[*mypackage.Service](core, "mypackage") -``` - -**Lifecycle.** Services implementing `Startable` or `Stoppable` are called automatically during boot and shutdown. - -**ACTION bus.** Services communicate by broadcasting typed messages via `core.ACTION(msg)` and registering handlers via `core.RegisterAction()`. This decouples packages without requiring direct imports between them. - ---- - -## Command Reference - -The `core` CLI is documented command-by-command in `docs/cmd/`: - -| Command | Description | -|---------|-------------| -| [cmd/](cmd/) | Full command index | -| [cmd/go/](cmd/go/) | Go development: test, fmt, lint, coverage, mod, work | -| [cmd/php/](cmd/php/) | Laravel/PHP development: dev server, test, deploy | -| [cmd/build/](cmd/build/) | Build Go, Wails, Docker, LinuxKit projects | -| [cmd/ci/](cmd/ci/) | Publish releases to GitHub, Docker, npm, Homebrew | -| [cmd/sdk/](cmd/sdk/) | SDK generation and OpenAPI validation | -| [cmd/dev/](cmd/dev/) | Multi-repo workflow and sandboxed dev environment | -| [cmd/ai/](cmd/ai/) | AI task management and Claude integration | -| [cmd/pkg/](cmd/pkg/) | Package search and install | -| [cmd/vm/](cmd/vm/) | LinuxKit VM management | -| [cmd/docs/](cmd/docs/) | Documentation sync and management | -| [cmd/setup/](cmd/setup/) | Clone repositories from a registry | -| [cmd/doctor/](cmd/doctor/) | Verify development environment | -| [cmd/test/](cmd/test/) | Run Go tests with coverage reporting | +This is the foundation layer — it has no CLI, no GUI, and minimal dependencies (`go-io`, `go-log`, `testify`). --- ## Packages -The Core repository contains the following internal packages. Full API analysis for each is available in the batch analysis documents listed under [Reference](#reference). - -### Foundation - | Package | Description | |---------|-------------| -| `pkg/framework` | Dependency injection container; re-exports `pkg/framework/core` | -| `pkg/log` | Structured logger with `Err` error type, operation chains, and log rotation | -| `pkg/config` | 12-factor config management layered over Viper; accepts `io.Medium` | -| `pkg/io` | Filesystem abstraction (`Medium` interface); `NewSandboxed`, `MockMedium` | -| `pkg/crypt` | Opinionated crypto: Argon2id passwords, ChaCha20 encryption, HMAC | -| `pkg/cache` | File-based JSON cache with TTL expiry | -| `pkg/i18n` | Grammar engine with pluralisation, verb conjugation, semantic sentences | - -### CLI and Interaction - -| Package | Description | -|---------|-------------| -| `pkg/cli` | CLI runtime: Cobra wrapping, ANSI styling, prompts, daemon lifecycle | -| `pkg/help` | Embedded documentation catalogue with in-memory full-text search | -| `pkg/session` | Claude Code JSONL transcript parser; HTML and MP4 export | -| `pkg/workspace` | Isolated, PGP-keyed workspace environments with IPC control | - -### Build and Release - -| Package | Description | -|---------|-------------| -| `pkg/build` | Project type detection, cross-compilation, archiving, checksums | -| `pkg/release` | Semantic versioning, conventional-commit changelogs, multi-target publishing | -| `pkg/container` | LinuxKit VM lifecycle via QEMU/Hyperkit; template management | -| `pkg/process` | `os/exec` wrapper with ring-buffer output, DAG task runner, ACTION streaming | -| `pkg/jobrunner` | Poll-dispatch automation engine with JSONL audit journal | - -### Source Control and Hosting - -| Package | Description | -|---------|-------------| -| `pkg/git` | Multi-repo status, push, pull; concurrent status checks | -| `pkg/repos` | `repos.yaml` registry loader; topological dependency ordering | -| `pkg/gitea` | Gitea API client with PR metadata extraction | -| `pkg/forge` | Forgejo API client with PR metadata extraction | -| `pkg/plugin` | Git-based CLI extension system | - -### AI and Agentic - -| Package | Description | -|---------|-------------| -| `pkg/mcp` | MCP server exposing file, process, RAG, and CDP tools to AI agents | -| `pkg/rag` | RAG pipeline: Markdown chunking, Ollama embeddings, Qdrant vector search | -| `pkg/ai` | Facade over RAG and metrics; `QueryRAGForTask` for prompt enrichment | -| `pkg/agentic` | REST client for core-agentic; `AutoCommit`, `CreatePR`, `BuildTaskContext` | -| `pkg/agentci` | Configuration bridge for AgentCI dispatch targets | -| `pkg/collect` | Data collection pipeline from GitHub, forums, market APIs | - -### Infrastructure and Networking - -| Package | Description | -|---------|-------------| -| `pkg/devops` | LinuxKit dev environment lifecycle; SSH bridging; project auto-detection | -| `pkg/ansible` | Native Go Ansible-lite engine; SSH playbook execution without the CLI | -| `pkg/webview` | Chrome DevTools Protocol client; Angular-aware automation | -| `pkg/ws` | WebSocket hub with channel-based subscriptions | -| `pkg/unifi` | UniFi controller client for network management | -| `pkg/auth` | OpenPGP challenge-response authentication; air-gapped flow | +| `pkg/core` | DI container, service registry, lifecycle, query/task bus | +| `pkg/log` | Structured logger service with Core integration | --- -## Workflows +## Quick Start -| Document | Description | -|----------|-------------| -| [Workflows](workflows.md) | Go build and release, PHP deploy, multi-repo daily workflow, hotfix | -| [Migration](migration.md) | Migrating from `push-all.sh`, raw `go` commands, `goreleaser`, or manual git | +```go +import ( + "forge.lthn.ai/core/go/pkg/core" + "forge.lthn.ai/core/go/pkg/log" +) + +c, err := core.New( + core.WithName("log", log.NewService(log.Options{Level: log.LevelInfo})), + core.WithName("myservice", mypackage.NewService(opts)), +) +// Services implementing Startable/Stoppable are called automatically. +``` + +### Type-safe service retrieval + +```go +svc, err := core.ServiceFor[*mypackage.Service](c, "myservice") +``` + +### Query/Task bus + +Services communicate via typed messages without direct imports: + +```go +// Query: request data from a service +result, err := c.Query(log.QueryLevel{}) + +// Task: ask a service to do something +c.Task(log.TaskSetLevel{Level: log.LevelDebug}) +``` --- -## Reference +## Architecture -| Document | Description | -|----------|-------------| -| [Configuration](configuration.md) | `.core/` directory, `release.yaml`, `build.yaml`, `php.yaml`, `repos.yaml`, environment variables | -| [Glossary](glossary.md) | Term definitions: target, workspace, registry, publisher, dry-run | -| [Troubleshooting](troubleshooting.md) | Installation failures, build errors, release issues, multi-repo problems, PHP issues | -| [Claude Code Skill](skill/) | Install the `core` skill to teach Claude Code how to use this CLI | +See [Package Standards](pkg/PACKAGE_STANDARDS.md) for the canonical patterns: +- Service struct with `core.ServiceRuntime[T]` +- Factory functions for registration +- Lifecycle hooks (`Startable`, `Stoppable`) +- Thread-safe APIs +- Query/Task handlers -### Historical Package Analysis - -The following documents were generated by an automated analysis pipeline (Gemini, February 2026) to extract architecture, public API, and test coverage notes from each package. They remain valid as architectural reference. - -| Document | Packages Covered | -|----------|-----------------| -| [pkg-batch1-analysis.md](pkg-batch1-analysis.md) | `pkg/log`, `pkg/config`, `pkg/io`, `pkg/crypt`, `pkg/auth` | -| [pkg-batch2-analysis.md](pkg-batch2-analysis.md) | `pkg/cli`, `pkg/help`, `pkg/session`, `pkg/workspace` | -| [pkg-batch3-analysis.md](pkg-batch3-analysis.md) | `pkg/build`, `pkg/container`, `pkg/process`, `pkg/jobrunner` | -| [pkg-batch4-analysis.md](pkg-batch4-analysis.md) | `pkg/git`, `pkg/repos`, `pkg/gitea`, `pkg/forge`, `pkg/release` | -| [pkg-batch5-analysis.md](pkg-batch5-analysis.md) | `pkg/agentci`, `pkg/agentic`, `pkg/ai`, `pkg/rag` | -| [pkg-batch6-analysis.md](pkg-batch6-analysis.md) | `pkg/ansible`, `pkg/devops`, `pkg/framework`, `pkg/mcp`, `pkg/plugin`, `pkg/unifi`, `pkg/webview`, `pkg/ws`, `pkg/collect`, `pkg/i18n`, `pkg/cache` | - -### Design Plans - -| Document | Description | -|----------|-------------| -| [plans/2026-02-05-core-ide-job-runner-design.md](plans/2026-02-05-core-ide-job-runner-design.md) | Autonomous job runner design for core-ide: poller, dispatcher, MCP handler registry, JSONL training data | -| [plans/2026-02-05-core-ide-job-runner-plan.md](plans/2026-02-05-core-ide-job-runner-plan.md) | Implementation plan for the job runner | -| [plans/2026-02-05-mcp-integration.md](plans/2026-02-05-mcp-integration.md) | MCP integration design notes | -| [plans/2026-02-17-lem-chat-design.md](plans/2026-02-17-lem-chat-design.md) | LEM Chat Web Components design: streaming SSE, zero-dependency vanilla UI | +See [Log Service](pkg/log.md) for the reference implementation within this repo. --- -## Satellite Packages +## Ecosystem -The Core ecosystem extends across 19 standalone Go modules, all hosted under `forge.lthn.ai/core/`. Each has its own repository and `docs/` directory. +This framework is consumed by 20+ standalone Go modules under `forge.lthn.ai/core/`. The CLI, AI, ML, and infrastructure packages all build on this DI container. -See [ecosystem.md](ecosystem.md) for the full map, module paths, and dependency graph. - -| Package | Purpose | -|---------|---------| -| [go-inference](ecosystem.md#go-inference) | Shared `TextModel`/`Backend`/`Token` interfaces — the common contract | -| [go-mlx](ecosystem.md#go-mlx) | Native Metal GPU inference via CGO/mlx-c (Apple Silicon) | -| [go-rocm](ecosystem.md#go-rocm) | AMD ROCm GPU inference via llama-server subprocess | -| [go-ml](ecosystem.md#go-ml) | Scoring engine, backends, agent orchestrator | -| [go-ai](ecosystem.md#go-ai) | MCP hub with 49 registered tools | -| [go-agentic](ecosystem.md#go-agentic) | Service lifecycle and allowance management for agents | -| [go-rag](ecosystem.md#go-rag) | Qdrant vector search and Ollama embeddings | -| [go-i18n](ecosystem.md#go-i18n) | Grammar engine, reversal, GrammarImprint | -| [go-html](ecosystem.md#go-html) | HLCRF DOM compositor and WASM target | -| [go-crypt](ecosystem.md#go-crypt) | Cryptographic primitives, auth, trust policies | -| [go-scm](ecosystem.md#go-scm) | SCM/CI integration and AgentCI dispatch | -| [go-p2p](ecosystem.md#go-p2p) | P2P mesh networking and UEPS wire protocol | -| [go-devops](ecosystem.md#go-devops) | Ansible automation, build tooling, infrastructure, release | -| [go-help](ecosystem.md#go-help) | YAML help catalogue with full-text search and HTTP server | -| [go-ratelimit](ecosystem.md#go-ratelimit) | Sliding-window rate limiter with SQLite backend | -| [go-session](ecosystem.md#go-session) | Claude Code JSONL transcript parser | -| [go-store](ecosystem.md#go-store) | SQLite key-value store with `Watch`/`OnChange` | -| [go-ws](ecosystem.md#go-ws) | WebSocket hub with Redis bridge | -| [go-webview](ecosystem.md#go-webview) | Chrome DevTools Protocol automation client | +For CLI documentation, see `forge.lthn.ai/core/cli`. diff --git a/docs/mcp/angular-testing.md b/docs/mcp/angular-testing.md deleted file mode 100644 index 2dcf37a..0000000 --- a/docs/mcp/angular-testing.md +++ /dev/null @@ -1,470 +0,0 @@ -# Angular Testing with Webview MCP Tools - -This guide explains how to use the webview MCP tools to automate testing of Angular applications via Chrome DevTools Protocol (CDP). - -## Prerequisites - -1. **Chrome/Chromium Browser**: Installed and accessible -2. **Remote Debugging Port**: Chrome must be started with remote debugging enabled - -### Starting Chrome with Remote Debugging - -```bash -# Linux -google-chrome --remote-debugging-port=9222 - -# macOS -/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 - -# Windows -"C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222 - -# Headless mode (no visible window) -google-chrome --headless --remote-debugging-port=9222 -``` - -## Available MCP Tools - -### Connection Management - -#### webview_connect -Connect to Chrome DevTools. - -```json -{ - "tool": "webview_connect", - "arguments": { - "debug_url": "http://localhost:9222", - "timeout": 30 - } -} -``` - -#### webview_disconnect -Disconnect from Chrome DevTools. - -```json -{ - "tool": "webview_disconnect", - "arguments": {} -} -``` - -### Navigation - -#### webview_navigate -Navigate to a URL. - -```json -{ - "tool": "webview_navigate", - "arguments": { - "url": "http://localhost:4200" - } -} -``` - -### DOM Interaction - -#### webview_click -Click an element by CSS selector. - -```json -{ - "tool": "webview_click", - "arguments": { - "selector": "#login-button" - } -} -``` - -#### webview_type -Type text into an element. - -```json -{ - "tool": "webview_type", - "arguments": { - "selector": "#email-input", - "text": "user@example.com" - } -} -``` - -#### webview_query -Query DOM elements. - -```json -{ - "tool": "webview_query", - "arguments": { - "selector": ".error-message", - "all": true - } -} -``` - -#### webview_wait -Wait for an element to appear. - -```json -{ - "tool": "webview_wait", - "arguments": { - "selector": ".loading-spinner", - "timeout": 10 - } -} -``` - -### JavaScript Evaluation - -#### webview_eval -Execute JavaScript in the browser context. - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "document.title" - } -} -``` - -### Console & Debugging - -#### webview_console -Get browser console output. - -```json -{ - "tool": "webview_console", - "arguments": { - "clear": false - } -} -``` - -#### webview_screenshot -Capture a screenshot. - -```json -{ - "tool": "webview_screenshot", - "arguments": { - "format": "png" - } -} -``` - -## Angular-Specific Testing Patterns - -### 1. Waiting for Angular Zone Stability - -Before interacting with Angular components, wait for Zone.js to become stable: - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "(function() { const roots = window.getAllAngularRootElements(); if (!roots.length) return true; const injector = window.ng.probe(roots[0]).injector; const zone = injector.get('NgZone'); return zone.isStable; })()" - } -} -``` - -### 2. Navigating with Angular Router - -Use the Angular Router for client-side navigation: - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "(function() { const roots = window.getAllAngularRootElements(); const injector = window.ng.probe(roots[0]).injector; const router = injector.get('Router'); router.navigateByUrl('/dashboard'); return true; })()" - } -} -``` - -### 3. Accessing Component Properties - -Read or modify component state: - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "(function() { const el = document.querySelector('app-user-profile'); const component = window.ng.probe(el).componentInstance; return component.user; })()" - } -} -``` - -### 4. Triggering Change Detection - -Force Angular to update the view: - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "(function() { const roots = window.getAllAngularRootElements(); const injector = window.ng.probe(roots[0]).injector; const appRef = injector.get('ApplicationRef'); appRef.tick(); return true; })()" - } -} -``` - -### 5. Testing Form Validation - -Check Angular form state: - -```json -{ - "tool": "webview_eval", - "arguments": { - "script": "(function() { const form = document.querySelector('form'); const component = window.ng.probe(form).componentInstance; return { valid: component.form.valid, errors: component.form.errors }; })()" - } -} -``` - -## Complete Test Flow Example - -Here's a complete example testing an Angular login flow: - -### Step 1: Connect to Chrome - -```json -{"tool": "webview_connect", "arguments": {"debug_url": "http://localhost:9222"}} -``` - -### Step 2: Navigate to the Application - -```json -{"tool": "webview_navigate", "arguments": {"url": "http://localhost:4200/login"}} -``` - -### Step 3: Wait for Angular to Load - -```json -{"tool": "webview_wait", "arguments": {"selector": "app-login"}} -``` - -### Step 4: Fill in Login Form - -```json -{"tool": "webview_type", "arguments": {"selector": "#email", "text": "test@example.com"}} -{"tool": "webview_type", "arguments": {"selector": "#password", "text": "password123"}} -``` - -### Step 5: Submit the Form - -```json -{"tool": "webview_click", "arguments": {"selector": "button[type='submit']"}} -``` - -### Step 6: Wait for Navigation - -```json -{"tool": "webview_wait", "arguments": {"selector": "app-dashboard", "timeout": 10}} -``` - -### Step 7: Verify Success - -```json -{"tool": "webview_eval", "arguments": {"script": "window.location.pathname === '/dashboard'"}} -``` - -### Step 8: Check Console for Errors - -```json -{"tool": "webview_console", "arguments": {"clear": true}} -``` - -### Step 9: Disconnect - -```json -{"tool": "webview_disconnect", "arguments": {}} -``` - -## Debugging Tips - -### 1. Check for JavaScript Errors - -Always check the console output after operations: - -```json -{"tool": "webview_console", "arguments": {}} -``` - -### 2. Take Screenshots on Failure - -Capture the current state when something unexpected happens: - -```json -{"tool": "webview_screenshot", "arguments": {"format": "png"}} -``` - -### 3. Inspect Element State - -Query elements to understand their current state: - -```json -{"tool": "webview_query", "arguments": {"selector": ".my-component", "all": false}} -``` - -### 4. Get Page Source - -Retrieve the current HTML for debugging: - -```json -{"tool": "webview_eval", "arguments": {"script": "document.documentElement.outerHTML"}} -``` - -## Common Issues - -### Element Not Found - -If `webview_click` or `webview_type` fails with "element not found": - -1. Check the selector is correct -2. Wait for the element to appear first -3. Verify the element is visible (not hidden) - -### Angular Not Detected - -If Angular-specific scripts fail: - -1. Ensure the Angular app has loaded completely -2. Check that you're using Angular 2+ (not AngularJS) -3. Verify the element has an Angular component attached - -### Timeout Errors - -If operations timeout: - -1. Increase the timeout value -2. Check for loading spinners or blocking operations -3. Verify the network is working correctly - -## Best Practices - -1. **Always wait for elements** before interacting with them -2. **Check console for errors** after each major step -3. **Use explicit selectors** like IDs or data attributes -4. **Clear console** at the start of each test -5. **Disconnect** when done to free resources -6. **Take screenshots** at key checkpoints -7. **Handle async operations** by waiting for stability - -## Go API Usage - -For direct Go integration, use the `pkg/webview` package: - -```go -package main - -import ( - "log" - "time" - - "forge.lthn.ai/core/cli/pkg/webview" -) - -func main() { - // Connect to Chrome - wv, err := webview.New( - webview.WithDebugURL("http://localhost:9222"), - webview.WithTimeout(30*time.Second), - ) - if err != nil { - log.Fatal(err) - } - defer wv.Close() - - // Navigate - if err := wv.Navigate("http://localhost:4200"); err != nil { - log.Fatal(err) - } - - // Wait for element - if err := wv.WaitForSelector("app-root"); err != nil { - log.Fatal(err) - } - - // Click button - if err := wv.Click("#login-button"); err != nil { - log.Fatal(err) - } - - // Type text - if err := wv.Type("#email", "test@example.com"); err != nil { - log.Fatal(err) - } - - // Get console output - messages := wv.GetConsole() - for _, msg := range messages { - log.Printf("[%s] %s", msg.Type, msg.Text) - } - - // Take screenshot - data, err := wv.Screenshot() - if err != nil { - log.Fatal(err) - } - // Save data to file... -} -``` - -### Using Angular Helper - -For Angular-specific operations: - -```go -package main - -import ( - "log" - "time" - - "forge.lthn.ai/core/cli/pkg/webview" -) - -func main() { - wv, err := webview.New(webview.WithDebugURL("http://localhost:9222")) - if err != nil { - log.Fatal(err) - } - defer wv.Close() - - // Create Angular helper - angular := webview.NewAngularHelper(wv) - - // Navigate using Angular Router - if err := angular.NavigateByRouter("/dashboard"); err != nil { - log.Fatal(err) - } - - // Wait for Angular to stabilize - if err := angular.WaitForAngular(); err != nil { - log.Fatal(err) - } - - // Get component property - value, err := angular.GetComponentProperty("app-user-profile", "user") - if err != nil { - log.Fatal(err) - } - log.Printf("User: %v", value) - - // Call component method - result, err := angular.CallComponentMethod("app-counter", "increment", 5) - if err != nil { - log.Fatal(err) - } - log.Printf("Result: %v", result) -} -``` - -## See Also - -- [Chrome DevTools Protocol Documentation](https://chromedevtools.github.io/devtools-protocol/) -- [pkg/webview package documentation](../../pkg/webview/) -- [MCP Tools Reference](../mcp/) diff --git a/docs/migration.md b/docs/migration.md deleted file mode 100644 index e5c4606..0000000 --- a/docs/migration.md +++ /dev/null @@ -1,233 +0,0 @@ -# Migration Guide - -Migrating from legacy scripts and tools to Core CLI. - -## From push-all.sh - -The `push-all.sh` script has been replaced by `core dev` commands. - -| Legacy | Core CLI | Notes | -|--------|----------|-------| -| `./push-all.sh --status` | `core dev work --status` | Status table | -| `./push-all.sh --commit` | `core dev commit` | Commit dirty repos | -| `./push-all.sh` | `core dev work` | Full workflow | - -### Quick Migration - -```bash -# Instead of -./push-all.sh --status - -# Use -core dev work --status -``` - -### New Features - -Core CLI adds features not available in the legacy script: - -```bash -# Quick health summary -core dev health -# Output: "18 repos │ clean │ synced" - -# Pull repos that are behind -core dev pull - -# GitHub integration -core dev issues # List open issues -core dev reviews # List PRs needing review -core dev ci # Check CI status - -# Dependency analysis -core dev impact core-php # What depends on core-php? -``` - ---- - -## From Raw Go Commands - -Core wraps Go commands with enhanced defaults and output. - -| Raw Command | Core CLI | Benefits | -|-------------|----------|----------| -| `go test ./...` | `core go test` | Filters warnings, sets CGO_ENABLED=0 | -| `go test -coverprofile=...` | `core go cov` | HTML reports, thresholds | -| `gofmt -w .` | `core go fmt --fix` | Uses goimports if available | -| `golangci-lint run` | `core go lint` | Consistent interface | -| `go build` | `core build` | Cross-compile, sign, archive | - -### Why Use Core? - -```bash -# Raw go test shows linker warnings on macOS -go test ./... -# ld: warning: -no_pie is deprecated... - -# Core filters noise -core go test -# PASS (clean output) -``` - -### Environment Setup - -Core automatically sets: -- `CGO_ENABLED=0` - Static binaries -- `MACOSX_DEPLOYMENT_TARGET=26.0` - Suppress macOS warnings -- Colour output for coverage reports - ---- - -## From Raw PHP Commands - -Core orchestrates Laravel development services. - -| Raw Command | Core CLI | Benefits | -|-------------|----------|----------| -| `php artisan serve` | `core php dev` | Adds Vite, Horizon, Reverb, Redis | -| `./vendor/bin/pest` | `core php test` | Auto-detects test runner | -| `./vendor/bin/pint` | `core php fmt --fix` | Consistent interface | -| Manual Coolify deploy | `core php deploy` | Tracked, scriptable | - -### Development Server Comparison - -```bash -# Raw: Start each service manually -php artisan serve & -npm run dev & -php artisan horizon & -php artisan reverb:start & - -# Core: One command -core php dev -# Starts all services, shows unified logs -``` - ---- - -## From goreleaser - -Core's release system is simpler than goreleaser for host-uk projects. - -| goreleaser | Core CLI | -|------------|----------| -| `.goreleaser.yaml` | `.core/release.yaml` | -| `goreleaser release --snapshot` | `core ci` (dry-run) | -| `goreleaser release` | `core ci --we-are-go-for-launch` | - -### Configuration Migration - -**goreleaser:** -```yaml -builds: - - main: ./cmd/app - goos: [linux, darwin, windows] - goarch: [amd64, arm64] - -archives: - - format: tar.gz - files: [LICENSE, README.md] - -release: - github: - owner: host-uk - name: app -``` - -**Core:** -```yaml -version: 1 - -project: - name: app - repository: host-uk/app - -targets: - - os: linux - arch: amd64 - - os: darwin - arch: arm64 - -publishers: - - type: github -``` - -### Key Differences - -1. **Separate build and release** - Core separates `core build` from `core ci` -2. **Safe by default** - `core ci` is dry-run unless `--we-are-go-for-launch` -3. **Simpler config** - Fewer options, sensible defaults - ---- - -## From Manual Git Operations - -Core automates multi-repo git workflows. - -| Manual | Core CLI | -|--------|----------| -| `cd repo1 && git status && cd ../repo2 && ...` | `core dev work --status` | -| Check each repo for uncommitted changes | `core dev health` | -| Commit each repo individually | `core dev commit` | -| Push each repo individually | `core dev push` | - -### Example: Committing Across Repos - -**Manual:** -```bash -cd core-php -git add -A -git commit -m "feat: add feature" -cd ../core-tenant -git add -A -git commit -m "feat: use new feature" -# ... repeat for each repo -``` - -**Core:** -```bash -core dev commit -# Interactive: reviews changes, suggests messages -# Adds Co-Authored-By automatically -``` - ---- - -## Deprecated Commands - -These commands have been removed or renamed: - -| Deprecated | Replacement | Version | -|------------|-------------|---------| -| `core sdk generate` | `core build sdk` | v0.5.0 | -| `core dev task*` | `core ai task*` | v0.8.0 | -| `core release` | `core ci` | v0.6.0 | - ---- - -## Version Compatibility - -| Core Version | Go Version | Breaking Changes | -|--------------|------------|------------------| -| v1.0.0+ | 1.23+ | Stable API | -| v0.8.0 | 1.22+ | Task commands moved to `ai` | -| v0.6.0 | 1.22+ | Release command renamed to `ci` | -| v0.5.0 | 1.21+ | SDK generation moved to `build sdk` | - ---- - -## Getting Help - -If you encounter issues during migration: - -1. Check [Troubleshooting](troubleshooting.md) -2. Run `core doctor` to verify setup -3. Use `--help` on any command: `core dev work --help` - ---- - -## See Also - -- [Getting Started](getting-started.md) - Fresh installation -- [Workflows](workflows.md) - Common task sequences -- [Configuration](configuration.md) - Config file reference diff --git a/docs/pkg-batch1-analysis.md b/docs/pkg-batch1-analysis.md deleted file mode 100644 index ef2dc88..0000000 --- a/docs/pkg-batch1-analysis.md +++ /dev/null @@ -1,213 +0,0 @@ -Here is the technical documentation for the Core framework packages. - -# Core Framework Documentation - -## Package: pkg/log - -### 1. Overview -`pkg/log` acts as the central observability and error handling primitive for the framework. It combines structured logging with a rich error type system (`Err`), allowing operational context (Operations, Codes) to travel with errors up the stack. It is designed to be used both standalone and as an injectable service within the Core framework. - -### 2. Public API - -**Error Types & Functions** -* `type Err`: Struct implementing `error` with fields for `Op` (operation), `Msg` (message), `Err` (wrapped error), and `Code` (machine-readable code). -* `func E(op, msg string, err error) error`: Creates a new error with operational context. -* `func Wrap(err error, op, msg string) error`: Wraps an existing error, preserving existing codes if present. -* `func WrapCode(err error, code, op, msg string) error`: Wraps an error and assigns a specific error code. -* `func NewCode(code, msg string) error`: Creates a sentinel error with a code. -* `func Is(err, target error) bool`: Wrapper for `errors.Is`. -* `func As(err error, target any) bool`: Wrapper for `errors.As`. -* `func Join(errs ...error) error`: Wrapper for `errors.Join`. -* `func Op(err error) string`: Extracts the operation name from an error chain. -* `func ErrCode(err error) string`: Extracts the error code from an error chain. -* `func StackTrace(err error) []string`: Returns a slice of operations leading to the error. -* `func LogError(err error, op, msg string) error`: Logs an error and returns it wrapped (reduces boilerplate). -* `func LogWarn(err error, op, msg string) error`: Logs a warning and returns it wrapped. -* `func Must(err error, op, msg string)`: Panics if error is not nil, logging it first. - -**Logging Types & Functions** -* `type Logger`: The main logging struct. -* `type Level`: Integer type for log verbosity (`LevelQuiet` to `LevelDebug`). -* `type Options`: Configuration struct for Logger (Level, Output, Rotation). -* `type RotationOptions`: Config for log file rotation (Size, Age, Backups, Compression). -* `func New(opts Options) *Logger`: Constructor. -* `func Default() *Logger`: Returns the global default logger. -* `func SetDefault(l *Logger)`: Sets the global default logger. -* `func (l *Logger) Debug/Info/Warn/Error/Security(msg string, keyvals ...any)`: Leveled logging methods. - -**Service Integration** -* `type Service`: Wraps `Logger` for framework integration. -* `func NewService(opts Options) func(*framework.Core) (any, error)`: Factory for dependency injection. -* `type QueryLevel`, `type TaskSetLevel`: Message types for runtime management. - -### 3. Internal Design -* **Contextual Errors**: The `Err` struct forms a linked list via the `Err` field (inner error), allowing the reconstruction of a logical stack trace (`op` sequence) distinct from the runtime stack trace. -* **Concurrency**: The `Logger` uses a `sync.RWMutex` to guard configuration and writes, ensuring thread safety. -* **Rotation Strategy**: The `RotatingWriter` implements `io.WriteCloser`. It lazily opens files and checks size thresholds on every write, leveraging `pkg/io` to abstract the filesystem. -* **Framework Integration**: The `Service` struct embeds `framework.ServiceRuntime`, utilizing the Actor pattern (Queries and Tasks) to allow dynamic log level adjustment at runtime without restarting the application. - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/io`: Used by `rotation.go` to handle file operations (renaming, deleting, writing) abstractly. -* `forge.lthn.ai/core/cli/pkg/framework`: Used by `service.go` to hook into the application lifecycle and message bus. -* Standard Lib: `errors`, `fmt`, `os`, `sync`, `time`. - -### 5. Test Coverage Notes -* **Error Unwrapping**: Verify `errors.Is` and `errors.As` work correctly through deep chains of `log.Err`. -* **Logical Stack Traces**: Ensure `StackTrace()` returns the correct order of operations `["app.Run", "db.Query", "net.Dial"]`. -* **Log Rotation**: Critical to test the boundary conditions of `MaxSize` and `MaxBackups` using a Mock Medium to avoid actual disk I/O. -* **Concurrency**: Race detection on `Logger` when changing levels while logging is active. - -### 6. Integration Points -* **Application-wide**: This is the most imported package. All other packages should use `log.E` or `log.Wrap` instead of `fmt.Errorf` or `errors.New`. -* **Core Framework**: The `Service` is designed to be passed to `core.New()`. - ---- - -## Package: pkg/config - -### 1. Overview -`pkg/config` provides 12-factor app configuration management. It layers configuration sources in a specific precedence (Environment > Config File > Defaults) and exposes them via a typed API or a dot-notation getter. It abstracts the underlying storage, allowing configs to be loaded from disk or memory. - -### 2. Public API -* `type Config`: The main configuration manager. -* `type Option`: Functional option pattern for configuration. -* `func New(opts ...Option) (*Config, error)`: Constructor. -* `func LoadEnv(prefix string) map[string]any`: Helper to parse environment variables into a map. -* `func (c *Config) Get(key string, out any) error`: Unmarshals a key (or root) into a struct. -* `func (c *Config) Set(key string, v any) error`: Sets a value and persists it to storage. -* `func (c *Config) LoadFile(m coreio.Medium, path string) error`: Merges a file into the current config. -* `type Service`: Framework service wrapper for `Config`. -* `func NewConfigService(c *core.Core) (any, error)`: Factory for dependency injection. - -### 3. Internal Design -* **Engine**: Uses `spf13/viper` as the underlying configuration engine for its merging and unmarshalling logic. -* **Abstraction**: Unlike standard Viper usage, this package decouples the filesystem using `pkg/io.Medium`. This allows the config system to work in sandboxed environments or with mock filesystems. -* **Persistence**: The `Set` method triggers an immediate write-back to the storage medium, making the config file the source of truth for runtime changes. -* **Environment Mapping**: Automatically maps `CORE_CONFIG_FOO_BAR` to `foo.bar` using a `strings.Replacer`. - -### 4. Dependencies -* `github.com/spf13/viper`: Core logic for map merging and unmarshalling. -* `gopkg.in/yaml.v3`: For marshalling data when saving. -* `forge.lthn.ai/core/cli/pkg/io`: For reading/writing config files. -* `forge.lthn.ai/core/cli/pkg/framework/core`: For service integration and error handling. - -### 5. Test Coverage Notes -* **Precedence**: Verify that Environment variables override File values. -* **Persistence**: Test that `Set()` writes valid YAML back to the `Medium`. -* **Type Safety**: Ensure `Get()` correctly unmarshals into complex structs and returns errors on type mismatches. - -### 6. Integration Points -* **Bootstrap**: Usually the first service initialized in `core.New()`. -* **Service Configuration**: Other services (like `auth` or `log`) should inject `config.Service` to retrieve their startup settings. - ---- - -## Package: pkg/io - -### 1. Overview -`pkg/io` provides a filesystem abstraction layer (`Medium`). Its philosophy is to decouple business logic from the `os` package, facilitating easier testing (via mocks) and security (via sandboxing). - -### 2. Public API -* `type Medium`: Interface defining filesystem operations (`Read`, `Write`, `List`, `Stat`, `Open`, `Create`, `Delete`, `Rename`, etc.). -* `var Local`: A pre-initialized `Medium` for the host root filesystem. -* `func NewSandboxed(root string) (Medium, error)`: Returns a `Medium` restricted to a specific directory. -* `type MockMedium`: In-memory implementation of `Medium` for testing. -* `func NewMockMedium() *MockMedium`: Constructor for the mock. -* **Helpers**: `Read`, `Write`, `Copy`, `EnsureDir`, `IsFile`, `ReadStream`, `WriteStream` (accept `Medium` as first arg). - -### 3. Internal Design -* **Interface Segregation**: The `Medium` interface mimics the capabilities of `os` and `io/fs` but bundles them into a single dependency. -* **Mocking**: `MockMedium` uses `map[string]string` for files and `map[string]bool` for directories. It implements manual path logic to simulate filesystem behavior (e.g., verifying a directory is empty before deletion) without touching the disk. -* **Sandboxing**: The `local` implementation (imported internally) enforces path scoping to prevent traversal attacks when using `NewSandboxed`. - -### 4. Dependencies -* Standard Lib: `io`, `io/fs`, `os`, `path/filepath`, `strings`, `time`. -* `forge.lthn.ai/core/cli/pkg/io/local`: (Implied) The concrete implementation for OS disk access. - -### 5. Test Coverage Notes -* **Mock fidelity**: The `MockMedium` must behave exactly like the OS. E.g., `Rename` should fail if the source doesn't exist; `Delete` should fail if a directory is not empty. -* **Sandboxing**: Verify that `..` traversal attempts in `NewSandboxed` cannot access files outside the root. - -### 6. Integration Points -* **Universal Dependency**: Used by `log` (rotation), `config` (loading), and `auth` (user DB). -* **Testing**: Application code should accept `io.Medium` in constructors rather than using `os.Open` directly, enabling unit tests to use `NewMockMedium()`. - ---- - -## Package: pkg/crypt - -### 1. Overview -`pkg/crypt` provides "batteries-included," opinionated cryptographic primitives. It abstracts away the complexity of parameter selection (salt length, iteration counts, nonce generation) to prevent misuse of crypto algorithms. - -### 2. Public API -* **Hashing**: `HashPassword` (Argon2id), `VerifyPassword`, `HashBcrypt`, `VerifyBcrypt`. -* **Symmetric**: `Encrypt`/`Decrypt` (ChaCha20-Poly1305), `EncryptAES`/`DecryptAES` (AES-GCM). -* **KDF**: `DeriveKey` (Argon2), `DeriveKeyScrypt`, `HKDF`. -* **Checksums**: `SHA256File`, `SHA512File`, `SHA256Sum`, `SHA512Sum`. -* **HMAC**: `HMACSHA256`, `HMACSHA512`, `VerifyHMAC`. - -### 3. Internal Design -* **Safe Defaults**: Uses Argon2id for password hashing with tuned parameters (64MB memory, 3 iterations). -* **Container Format**: Symmetric encryption functions return a concatenated byte slice: `[Salt (16b) | Nonce (Variable) | Ciphertext]`. This ensures the decryption function has everything it needs without separate state management. -* **Randomness**: Automatically handles salt and nonce generation using `crypto/rand`. - -### 4. Dependencies -* `golang.org/x/crypto`: For Argon2, ChaCha20, HKDF, Scrypt. -* Standard Lib: `crypto/aes`, `crypto/cipher`, `crypto/rand`, `crypto/sha256`. - -### 5. Test Coverage Notes -* **Interoperability**: Ensure `Encrypt` output can be read by `Decrypt`. -* **Tamper Resistance**: manually modifying a byte in the ciphertext or nonce must result in a decryption failure (AuthTag check). -* **Vectors**: Validate hashing against known test vectors where possible. - -### 6. Integration Points -* **Auth**: Heavily used by `pkg/auth` for password storage and potentially for encrypted user data. -* **Data Protection**: Any service requiring data at rest encryption should use `crypt.Encrypt`. - ---- - -## Package: pkg/auth - -### 1. Overview -`pkg/auth` implements a persistent user identity system based on OpenPGP challenge-response authentication. It supports a unique "Air-Gapped" workflow where challenges and responses are exchanged via files, alongside standard online methods. It manages user lifecycles, sessions, and key storage. - -### 2. Public API -* `type Authenticator`: Main logic controller. -* `type User`: User metadata struct. -* `type Session`: Active session token struct. -* `func New(m io.Medium, opts ...Option) *Authenticator`: Constructor. -* `func (a *Authenticator) Register(username, password string) (*User, error)`: Creates new user and PGP keys. -* `func (a *Authenticator) Login(userID, password string) (*Session, error)`: Password-based fallback login. -* `func (a *Authenticator) CreateChallenge(userID string) (*Challenge, error)`: Starts PGP auth flow. -* `func (a *Authenticator) ValidateResponse(userID string, signedNonce []byte) (*Session, error)`: Completes PGP auth flow. -* `func (a *Authenticator) ValidateSession(token string) (*Session, error)`: Checks token validity. -* `func (a *Authenticator) WriteChallengeFile(userID, path string) error`: For air-gapped flow. -* `func (a *Authenticator) ReadResponseFile(userID, path string) (*Session, error)`: For air-gapped flow. - -### 3. Internal Design -* **Storage Layout**: Uses a flat-file database approach on `io.Medium`: - * `users/{id}.pub`: Public Key. - * `users/{id}.key`: Encrypted Private Key. - * `users/{id}.lthn`: Password Hash. - * `users/{id}.json`: Encrypted metadata. -* **Identity**: User IDs are hashes of usernames to anonymize storage structure. -* **Flow**: - 1. Server generates random nonce. - 2. Server encrypts nonce with User Public Key. - 3. User decrypts nonce (client-side) and signs it. - 4. Server validates signature against User Public Key. - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/io`: For user database storage. -* `forge.lthn.ai/core/cli/pkg/crypt/lthn`: (Implied) Specific password hashing. -* `forge.lthn.ai/core/cli/pkg/crypt/pgp`: (Implied) OpenPGP operations. -* `forge.lthn.ai/core/cli/pkg/framework/core`: Error handling. - -### 5. Test Coverage Notes -* **Flow Verification**: Full integration test simulating a client: Register -> Get Challenge -> Decrypt/Sign (Mock Client) -> Validate -> Get Token. -* **Security**: Ensure `server` user cannot be deleted. Ensure expired sessions are rejected. -* **Persistence**: Ensure user data survives an `Authenticator` restart (i.e., data is actually written to medium). - -### 6. Integration Points -* **API Gateways**: HTTP handlers would call `ValidateSession` on every request. -* **CLI Tools**: Would use `WriteChallengeFile`/`ReadResponseFile` for offline authentication. diff --git a/docs/pkg-batch2-analysis.md b/docs/pkg-batch2-analysis.md deleted file mode 100644 index 9557a55..0000000 --- a/docs/pkg-batch2-analysis.md +++ /dev/null @@ -1,255 +0,0 @@ -# Package Analysis — Batch 2 - -Generated by: gemini-batch-runner.sh -Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview -Date: 2026-02-09 -Packages: cli help session workspace -Total tokens: 125308 - ---- - -Here is the documentation for the analyzed framework packages. - -# Core Framework Documentation - -## Package: `pkg/cli` - -### 1. Overview -The `cli` package is a comprehensive application runtime and UI framework designed to build uniform, aesthetic, and robust command-line interfaces. It acts as a high-level wrapper around `cobra`, handling application lifecycle (signals, daemonization), output styling (ANSI colors, glyphs, layouts), interactive prompts, and internationalization (i18n). Its design philosophy prioritizes developer ergonomics ("fluent" APIs) and consistent user experience across different execution modes (interactive vs. headless). - -### 2. Public API - -#### Application Lifecycle -- `func Init(opts Options) error`: Initialises the global CLI runtime, sets up the root command, and registers services. -- `func Main()`: The main entry point. Handles panic recovery, service initialization, and command execution. Exits process on completion. -- `func Execute() error`: Executes the root command structure. -- `func Shutdown()`: Triggers graceful shutdown of the runtime and all services. -- `func Run(ctx context.Context) error`: Blocking helper for daemon/simple modes. -- `func RunWithTimeout(timeout time.Duration) func()`: Returns a shutdown function that enforces a timeout. - -#### Command Building -- `func NewCommand(use, short, long string, run func(*Command, []string) error) *Command`: Factory for standard commands. -- `func NewGroup(use, short, long string) *Command`: Factory for parent commands (no run logic). -- `func WithCommands(name string, register func(root *Command)) framework.Option`: Registers a command group as a framework service. - -#### Output & Styling -- `type AnsiStyle`: Fluent builder for text styling (Bold, Dim, Foreground, Background). -- `func Success(msg string)`, `func Error(msg string)`, `func Warn(msg string)`, `func Info(msg string)`: Semantic logging to stdout/stderr with glyphs. -- `func Table`: Struct and methods for rendering ASCII/Unicode tables. -- `func Check(name string) *CheckBuilder`: Fluent builder for test/verification status lines (Pass/Fail/Skip). -- `func Task(label, message string)`: Prints a task header. -- `func Progress(verb string, current, total int, item ...string)`: Prints a transient progress line. -- `func Layout(variant string) *Composite`: Creates an HLCRF (Header, Left, Content, Right, Footer) terminal layout. - -#### Input & Interaction -- `func Confirm(prompt string, opts ...ConfirmOption) bool`: Interactive yes/no prompt. -- `func Prompt(label, defaultVal string) (string, error)`: Standard text input. -- `func Select(label string, options []string) (string, error)`: Interactive list selection. -- `func Choose[T](prompt string, items []T, opts ...ChooseOption[T]) T`: Generic selection helper. - -#### Utilities -- `func GhAuthenticated() bool`: Checks GitHub CLI authentication status. -- `func GitClone(ctx, org, repo, path string) error`: Smart clone (uses `gh` if auth, else `git`). - -### 3. Internal Design -- **Singleton Runtime**: The package relies on a package-level singleton `instance` (`runtime` struct) initialized via `Init`. This holds the `cobra.Command` tree and the Service Container. -- **Service Layering**: It integrates heavily with `pkg/framework`. Services like `log`, `i18n`, and `crypt` are injected into the runtime during initialization. -- **Mode Detection**: The `daemon.go` logic automatically detects if the app is running interactively (TTY), via pipe, or as a background daemon, adjusting output styling accordingly. -- **Global Error Handling**: Custom error types (`ExitError`) and wrappers (`WrapVerb`) utilize semantic grammar for consistent error messaging. -- **Glyph Abstraction**: The `Glyph` system abstracts symbols, allowing runtime switching between Unicode, Emoji, and ASCII themes based on terminal capabilities. - -### 4. Dependencies -- `github.com/spf13/cobra`: The underlying command routing engine. -- `forge.lthn.ai/core/cli/pkg/framework`: The dependency injection and service lifecycle container. -- `forge.lthn.ai/core/cli/pkg/i18n`: For translation and semantic grammar generation. -- `forge.lthn.ai/core/cli/pkg/log`: For structured logging. -- `golang.org/x/term`: For TTY detection. - -### 5. Test Coverage Notes -- **Interactive Prompts**: Tests must mock `stdin` to verify `Confirm`, `Prompt`, and `Select` behavior without hanging. -- **Command Registration**: Verify `WithCommands` services receive the root command during `OnStartup`. -- **Daemon Lifecycle**: Tests needed for `PIDFile` locking and `HealthServer` endpoints (/health, /ready). -- **Layout Rendering**: Snapshot testing is recommended for `Layout` and `Table` rendering to ensure ANSI codes and alignment are correct. - -### 6. Integration Points -- **Entry Point**: This package is the entry point for the entire application (`main.go` should call `cli.Main()`). -- **Service Registry**: Other packages (like `workspace` or custom logic) are registered as services via `cli.Options.Services`. -- **UI Standard**: All other packages should use `cli.Success`, `cli.Error`, etc., instead of `fmt.Println` to maintain visual consistency. - ---- - -## Package: `pkg/help` - -### 1. Overview -The `help` package provides an embedded documentation system. It treats documentation as data, parsing Markdown files into structured topics, and provides an in-memory full-text search engine to allow users to query help topics directly from the CLI. - -### 2. Public API -- `type Catalog`: The central registry of help topics. - - `func DefaultCatalog() *Catalog`: Creates a catalog with built-in topics. - - `func (c *Catalog) Add(t *Topic)`: Registers a topic. - - `func (c *Catalog) Search(query string) []*SearchResult`: Performs full-text search. - - `func (c *Catalog) Get(id string) (*Topic, error)`: Retrieves a specific topic. -- `func ParseTopic(path string, content []byte) (*Topic, error)`: Parses raw Markdown content into a Topic struct. -- `type Topic`: Struct representing a documentation page (includes Title, Content, Sections, Tags). - -### 3. Internal Design -- **In-Memory Indexing**: The `searchIndex` struct builds a reverse index (word -> topic IDs) on initialization. It does not use an external database. -- **Scoring Algorithm**: Search results are ranked based on a scoring system where matches in Titles > Section Headers > Content. -- **Markdown Parsing**: It uses Regex (`frontmatterRegex`, `headingRegex`) rather than a full AST parser to extract structure, prioritizing speed and simplicity for this specific use case. -- **Snippet Extraction**: The search logic includes a highlighter that extracts relevant text context around search terms. - -### 4. Dependencies -- `gopkg.in/yaml.v3`: Used to parse the YAML frontmatter at the top of Markdown files. - -### 5. Test Coverage Notes -- **Search Ranking**: Tests should verify that a keyword in a Title ranks higher than the same keyword in the body text. -- **Frontmatter Parsing**: Test with valid, invalid, and missing YAML frontmatter. -- **Tokenization**: Ensure `tokenize` handles punctuation and case insensitivity correctly to ensure search accuracy. - -### 6. Integration Points -- **CLI Help Command**: The `pkg/cli` package would likely have a `help` command that instantiates the `Catalog` and calls `Search` or `Get` based on user input. - ---- - -## Package: `pkg/session` - -### 1. Overview -The `session` package is a specialized toolkit for parsing, analyzing, and visualizing "Claude Code" session transcripts (`.jsonl` files). It allows developers to replay AI interactions, search through past sessions, and generate visual artifacts (HTML reports or MP4 videos). - -### 2. Public API -- `func ListSessions(projectsDir string) ([]Session, error)`: Scans a directory for session files. -- `func ParseTranscript(path string) (*Session, error)`: Reads a JSONL file and structures it into a `Session` object with a timeline of events. -- `func Search(projectsDir, query string) ([]SearchResult, error)`: specific search across all session files. -- `func RenderHTML(sess *Session, outputPath string) error`: Generates a self-contained HTML file visualizing the session. -- `func RenderMP4(sess *Session, outputPath string) error`: Uses `vhs` to render a video replay of the terminal session. - -### 3. Internal Design -- **Streaming Parser**: `ParseTranscript` uses `bufio.Scanner` to handle potentially large JSONL files line-by-line, reconstructing the state of tool use (e.g., matching a `tool_use` event with its corresponding `tool_result`). -- **External Dependency Wrapper**: `RenderMP4` generates a `.tape` file dynamically and executes the external `vhs` binary to produce video. -- **HTML embedding**: `RenderHTML` embeds CSS and JS directly into the Go source strings to produce a single-file portable output without static asset dependencies. - -### 4. Dependencies -- `github.com/charmbracelet/vhs` (Runtime dependency): The `vhs` binary must be installed for `RenderMP4` to work. -- Standard Library (`encoding/json`, `html/template` equivalents). - -### 5. Test Coverage Notes -- **JSON Parsing**: Critical to test against the exact schema of Claude Code logs, including edge cases like partial streams or error states. -- **VHS Generation**: Test that the generated `.tape` content follows the VHS syntax correctly. -- **Tool Mapping**: Verify that specific tools (Bash, Edit, Write) are correctly categorized and parsed from the raw JSON arguments. - -### 6. Integration Points -- **CLI Commands**: Likely used by commands like `core session list`, `core session play`, or `core session export`. -- **Filesystem**: Reads directly from the user's Claude Code project directory (usually `~/.claude/`). - ---- - -## Package: `pkg/workspace` - -### 1. Overview -The `workspace` package implements the `core.Workspace` interface, providing isolated, secure working environments. It manages the directory structure, file I/O, and cryptographic identity (PGP keys) associated with specific projects or contexts. - -### 2. Public API -- `func New(c *core.Core) (any, error)`: Service factory function compatible with the framework registry. -- `func (s *Service) CreateWorkspace(identifier, password string) (string, error)`: Initialises a new workspace directory with keys. -- `func (s *Service) SwitchWorkspace(name string) error`: Sets the active context. -- `func (s *Service) WorkspaceFileGet(filename string) (string, error)`: Reads a file from the active workspace. -- `func (s *Service) WorkspaceFileSet(filename, content string) error`: Writes a file to the active workspace. - -### 3. Internal Design -- **Service Implementation**: Implements `core.Workspace`. -- **IPC Handling**: Contains `HandleIPCEvents` to respond to generic framework messages (`workspace.create`, `workspace.switch`), allowing loose coupling with other components. -- **Path Hashing**: Uses SHA-256 to hash workspace identifiers into directory names (referred to as "LTHN proxy" in comments), likely to sanitize paths and obscure names. -- **Key Management**: Delegates actual key generation to the core's `Crypt()` service but manages the storage of the resulting keys within the workspace layout. - -### 4. Dependencies -- `forge.lthn.ai/core/cli/pkg/framework/core`: Interfaces. -- `forge.lthn.ai/core/cli/pkg/io`: File system abstraction (`io.Medium`). -- `crypt` service (Runtime dependency): Required for `CreateWorkspace`. - -### 5. Test Coverage Notes -- **Mocking IO**: Use an in-memory `io.Medium` implementation to test directory creation and file writing without touching the real disk. -- **State Management**: Test that `WorkspaceFileGet` fails correctly if `SwitchWorkspace` hasn't been called yet. -- **Concurrency**: `sync.RWMutex` is used; tests should verify race conditions aren't possible during rapid switching/reading. - -### 6. Integration Points -- **Core Framework**: Registered in `pkg/cli/app.go` via `framework.WithName("workspace", workspace.New)`. -- **IPC**: Can be controlled by other plugins or UI components via the framework's message bus. - ---- - -## Quick Reference (Flash Summary) - -### Package: `pkg/cli` -**Description:** A comprehensive CLI framework providing terminal styling, command management, interactive prompts, and daemon lifecycles. - -**Key Exported Types and Functions:** -- `AnsiStyle`: Struct for chaining terminal text styles (bold, colors, etc.). -- `Main()`: The primary entry point that initializes services and executes the root command. -- `Command`: Re-exported `cobra.Command` for simplified dependency management. -- `NewDaemon(opts)`: Manages background process lifecycles, PID files, and health checks. -- `Check(name)`: Fluent API for rendering status check lines (e.g., "✓ audit passed"). -- `Confirm/Question/Choose`: Interactive prompt utilities for user input and selection. -- `Composite`: Implements a region-based layout system (Header, Left, Content, Right, Footer). -- `Table`: Helper for rendering aligned tabular data in the terminal. - -**Dependencies:** -- `pkg/crypt/openpgp` -- `pkg/framework` -- `pkg/log` -- `pkg/workspace` -- `pkg/i18n` -- `pkg/io` - -**Complexity:** Complex - ---- - -### Package: `pkg/help` -**Description:** Manages display-agnostic help content with markdown parsing and full-text search capabilities. - -**Key Exported Types and Functions:** -- `Catalog`: Registry for managing and searching help topics. -- `Topic`: Represents a help page including content, sections, and metadata. -- `ParseTopic(path, content)`: Parses markdown files with YAML frontmatter into structured topics. -- `searchIndex`: Internal engine providing scored full-text search and snippet extraction. -- `GenerateID(title)`: Utility to create URL-safe identifiers from strings. - -**Dependencies:** -- *None* (Internal `pkg/*` imports) - -**Complexity:** Moderate - ---- - -### Package: `pkg/session` -**Description:** Parses, searches, and renders Claude Code session transcripts (JSONL) into HTML or video formats. - -**Key Exported Types and Functions:** -- `Session`: Holds metadata and a timeline of `Event` objects from a transcript. -- `ParseTranscript(path)`: Reads JSONL files and reconstructs tool usage, user, and assistant interactions. -- `RenderHTML(sess, path)`: Generates a self-contained, interactive HTML timeline of a session. -- `RenderMP4(sess, path)`: Uses VHS to generate a terminal-style video recording of a session. -- `Search(dir, query)`: Scans a directory of session files for specific text or tool usage. - -**Dependencies:** -- *None* - -**Complexity:** Moderate - ---- - -### Package: `pkg/workspace` -**Description:** Manages isolated, encrypted filesystem environments for different CLI projects. - -**Key Exported Types and Functions:** -- `Service`: Core service managing active workspaces and their storage roots. -- `CreateWorkspace(id, pass)`: Initializes a hashed directory structure and generates PGP keypairs. -- `SwitchWorkspace(name)`: Sets the active workspace for subsequent file operations. -- `WorkspaceFileSet/Get`: Encrypted file I/O within the active workspace context. -- `HandleIPCEvents`: Processes workspace-related commands via the internal message bus. - -**Dependencies:** -- `pkg/framework/core` -- `pkg/io` - -**Complexity:** Moderate diff --git a/docs/pkg-batch3-analysis.md b/docs/pkg-batch3-analysis.md deleted file mode 100644 index 36b7e2f..0000000 --- a/docs/pkg-batch3-analysis.md +++ /dev/null @@ -1,384 +0,0 @@ -# Package Analysis — Batch 3 - -Generated by: gemini-batch-runner.sh -Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview -Date: 2026-02-09 -Packages: build container process jobrunner -Total tokens: 96300 - ---- - -Here is the technical documentation for the Core framework packages, analyzing the provided source code. - -# Core Framework Package Documentation - -## Table of Contents -1. [pkg/build](#package-pkgbuild) -2. [pkg/container](#package-pkgcontainer) -3. [pkg/process](#package-pkgprocess) -4. [pkg/jobrunner](#package-pkgjobrunner) - ---- - -## Package: `pkg/build` - -### 1. Overview -The `build` package provides a standardized system for detecting project types, loading build configurations, and packaging artifacts. It is designed around an abstraction of the filesystem (`io.Medium`) to facilitate testing and cross-platform compatibility, handling compression formats (gzip, xz, zip) and SHA256 checksum generation. - -### 2. Public API - -#### Project Detection & Configuration -```go -// Represents a detected project type (e.g., "go", "wails", "node") -type ProjectType string - -// Detects project types in a directory based on marker files -func Discover(fs io.Medium, dir string) ([]ProjectType, error) -func PrimaryType(fs io.Medium, dir string) (ProjectType, error) - -// Helper predicates for detection -func IsGoProject(fs io.Medium, dir string) bool -func IsWailsProject(fs io.Medium, dir string) bool -func IsNodeProject(fs io.Medium, dir string) bool -func IsPHPProject(fs io.Medium, dir string) bool -func IsCPPProject(fs io.Medium, dir string) bool - -// Loads configuration from .core/build.yaml -func LoadConfig(fs io.Medium, dir string) (*BuildConfig, error) -func ConfigExists(fs io.Medium, dir string) bool -``` - -#### Artifact Management -```go -type Artifact struct { - Path, OS, Arch, Checksum string -} - -type ArchiveFormat string // "gz", "xz", "zip" - -// Archiving functions -func Archive(fs io.Medium, artifact Artifact) (Artifact, error) // Default gzip -func ArchiveXZ(fs io.Medium, artifact Artifact) (Artifact, error) -func ArchiveWithFormat(fs io.Medium, artifact Artifact, format ArchiveFormat) (Artifact, error) -func ArchiveAll(fs io.Medium, artifacts []Artifact) ([]Artifact, error) - -// Checksum functions -func Checksum(fs io.Medium, artifact Artifact) (Artifact, error) -func ChecksumAll(fs io.Medium, artifacts []Artifact) ([]Artifact, error) -func WriteChecksumFile(fs io.Medium, artifacts []Artifact, path string) error -``` - -#### Interfaces -```go -// Interface for project-specific build logic -type Builder interface { - Name() string - Detect(fs io.Medium, dir string) (bool, error) - Build(ctx context.Context, cfg *Config, targets []Target) ([]Artifact, error) -} -``` - -### 3. Internal Design -* **Filesystem Abstraction**: Heavily relies on dependency injection via `io.Medium` rather than direct `os` calls, enabling safe unit testing of file operations. -* **Strategy Pattern**: The `Builder` interface allows different build logic (Go, Docker, Node) to be swapped dynamically based on detection. -* **Priority Detection**: `Discovery` uses an ordered slice of markers (`markers` var) to handle hybrid projects (e.g., Wails is detected before Go). -* **Configuration Overlay**: Uses `mapstructure` to parse YAML config, applying sensible defaults via `applyDefaults` if fields are missing. - -### 4. Dependencies -* `archive/tar`, `archive/zip`, `compress/gzip`: Standard library for archiving. -* `github.com/Snider/Borg/pkg/compress`: External dependency for XZ compression support. -* `forge.lthn.ai/core/cli/pkg/io`: Internal interface for filesystem abstraction. -* `forge.lthn.ai/core/cli/pkg/config`: Internal centralized configuration loading. - -### 5. Test Coverage Notes -* **Mocking IO**: Tests must implement a mock `io.Medium` to simulate file existence (`Detect`) and write operations (`Archive`) without touching the disk. -* **Format Specifics**: Verify that Windows builds automatically default to `.zip` regardless of the requested format in `ArchiveWithFormat`. -* **Config Parsing**: Test `LoadConfig` with malformed YAML and missing fields to ensure defaults are applied correctly. - -### 6. Integration Points -* **CLI Build Commands**: This package is the backend for any `core build` CLI command. -* **CI Pipelines**: Used to generate release artifacts and `CHECKSUMS.txt` files for releases. - ---- - -## Package: `pkg/container` - -### 1. Overview -This package manages the lifecycle of local LinuxKit virtual machines. It abstracts underlying hypervisors (QEMU on Linux, Hyperkit on macOS) to provide a container-like experience (start, stop, logs, exec) for running VM images. - -### 2. Public API - -#### Manager & Lifecycle -```go -type Manager interface { - Run(ctx context.Context, image string, opts RunOptions) (*Container, error) - Stop(ctx context.Context, id string) error - List(ctx context.Context) ([]*Container, error) - Logs(ctx context.Context, id string, follow bool) (io.ReadCloser, error) - Exec(ctx context.Context, id string, cmd []string) error -} - -// Factory -func NewLinuxKitManager(m io.Medium) (*LinuxKitManager, error) -``` - -#### Templates -```go -type TemplateManager struct { ... } - -func NewTemplateManager(m io.Medium) *TemplateManager -func (tm *TemplateManager) ListTemplates() []Template -func (tm *TemplateManager) GetTemplate(name string) (string, error) -func (tm *TemplateManager) ApplyTemplate(name string, vars map[string]string) (string, error) -``` - -#### Types -```go -type Container struct { - ID, Name, Image string - Status Status // "running", "stopped", "error" - PID int - // ... ports, memory stats -} - -type RunOptions struct { - Name string - Detach bool - Memory, CPUs, SSHPort int - Ports, Volumes map[string]string -} -``` - -### 3. Internal Design -* **Hypervisor Abstraction**: The `Hypervisor` interface hides the complexity of building CLI arguments for `qemu-system-x86_64` vs `hyperkit`. -* **State Persistence**: Uses a JSON file (`.core/containers.json`) protected by a `sync.RWMutex` to track VM state across process restarts. -* **Embedded Assets**: Uses Go `embed` to package default LinuxKit YAML templates (`templates/*.yml`) inside the binary. -* **Log Following**: Implements a custom `followReader` to emulate `tail -f` behavior for VM logs. - -### 4. Dependencies -* `os/exec`: Essential for spawning the hypervisor processes. -* `embed`: For built-in templates. -* `forge.lthn.ai/core/cli/pkg/io`: Filesystem access for state and logs. - -### 5. Test Coverage Notes -* **Process Management**: Difficult to test `Run` in standard CI. Mocking `exec.Command` or the `Hypervisor` interface is required. -* **State Integrity**: Test `LoadState` and `SaveState` handles corruption or concurrent writes. -* **Template Interpolation**: Verify `ApplyVariables` correctly handles required vs optional `${VAR:-default}` syntax. - -### 6. Integration Points -* **Dev Environments**: Used to spin up isolated development environments defined by LinuxKit YAMLs. -* **Testing**: Can be used to launch disposable VMs for integration testing. - ---- - -## Package: `pkg/process` - -### 1. Overview -A sophisticated wrapper around `os/exec` that integrates with the Core framework's event bus. It features output streaming, ring-buffer capturing, dependency-based task execution (DAG), and a global singleton service for ease of use. - -### 2. Public API - -#### Service & Global Access -```go -// Global singletons (require Init) -func Init(c *framework.Core) error -func Start(ctx, cmd string, args ...string) (*Process, error) -func Run(ctx, cmd string, args ...string) (string, error) -func Kill(id string) error - -// Service Factory -func NewService(opts Options) func(*framework.Core) (any, error) -``` - -#### Process Control -```go -type Process struct { ... } - -func (p *Process) Wait() error -func (p *Process) Kill() error -func (p *Process) Output() string -func (p *Process) IsRunning() bool -func (p *Process) SendInput(input string) error -func (p *Process) Done() <-chan struct{} -``` - -#### Task Runner -```go -type Runner struct { ... } -type RunSpec struct { - Name, Command string - After []string // Dependencies - // ... args, env -} - -func NewRunner(svc *Service) *Runner -func (r *Runner) RunAll(ctx context.Context, specs []RunSpec) (*RunAllResult, error) -func (r *Runner) RunParallel(ctx context.Context, specs []RunSpec) (*RunAllResult, error) -``` - -### 3. Internal Design -* **Event Sourcing**: Instead of just logging, the service broadcasts events (`ActionProcessStarted`, `ActionProcessOutput`) via `framework.Core`. This allows UI frontends to subscribe to real-time output. -* **Ring Buffer**: Uses a fixed-size circular buffer (`RingBuffer`) to store logs, preventing memory exhaustion from long-running processes. -* **DAG Execution**: The `Runner.RunAll` method implements a dependency graph resolver to run tasks in parallel waves based on the `After` field. -* **Global Singleton**: Uses `atomic.Pointer` for a thread-safe global `Default()` service instance. - -### 4. Dependencies -* `os/exec`: The underlying execution engine. -* `forge.lthn.ai/core/cli/pkg/framework`: Creates the `ServiceRuntime` and provides the IPC/Action bus. - -### 5. Test Coverage Notes -* **Concurrency**: The `Runner` needs tests for race conditions during parallel execution. -* **Dependency Resolution**: Test circular dependencies (deadlock detection) and skip logic when a dependency fails. -* **Buffer Overflow**: Verify `RingBuffer` overwrites old data correctly when full. - -### 6. Integration Points -* **Task Runners**: The `Runner` struct is the engine for tools like `make` or `Taskfile`. -* **UI/TUI**: The Action-based output streaming is designed to feed data into a TUI or Web frontend in real-time. - ---- - -## Package: `pkg/jobrunner` - -### 1. Overview -A polling-based workflow engine designed to ingest "signals" (e.g., GitHub Issues/PRs), match them to specific handlers, and record execution results in a structured journal. It implements a "dry-run" capability and detailed audit logging. - -### 2. Public API - -#### Poller -```go -type Poller struct { ... } -type PollerConfig struct { - Sources []JobSource - Handlers []JobHandler - Journal *Journal - // ... interval, dryRun -} - -func NewPoller(cfg PollerConfig) *Poller -func (p *Poller) Run(ctx context.Context) error -func (p *Poller) AddSource(s JobSource) -func (p *Poller) AddHandler(h JobHandler) -``` - -#### Journaling -```go -type Journal struct { ... } -func NewJournal(baseDir string) (*Journal, error) -func (j *Journal) Append(signal *PipelineSignal, result *ActionResult) error -``` - -#### Interfaces -```go -type JobSource interface { - Poll(ctx context.Context) ([]*PipelineSignal, error) - Report(ctx context.Context, result *ActionResult) error -} - -type JobHandler interface { - Match(signal *PipelineSignal) bool - Execute(ctx context.Context, signal *PipelineSignal) (*ActionResult, error) -} -``` - -### 3. Internal Design -* **Poller Loop**: Runs a continuous loop (ticker-based) that snapshots sources and handlers at the start of every cycle to allow dynamic registration. -* **Data Models**: Defines rigid structures (`PipelineSignal`, `ActionResult`) to decouple data sources (GitHub) from logic handlers. -* **Journaling**: Writes `jsonl` (JSON Lines) files partitioned by repository and date (`baseDir/owner/repo/YYYY-MM-DD.jsonl`), ensuring an append-only audit trail. - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/log`: Internal logging. -* `encoding/json`: For journal serialization. - -### 5. Test Coverage Notes -* **Matching Logic**: Test that `findHandler` picks the correct handler for a given signal. -* **Dry Run**: Ensure `Execute` is *not* called when `dryRun` is true, but logs are generated. -* **Journal Locking**: Verify concurrent writes to the journal do not corrupt the JSONL file. - -### 6. Integration Points -* **CI Bots**: The primary framework for building bots that automate Pull Request management or Issue triage. -* **Dashboarding**: The generated JSONL journal files are structured to be ingested by analytics tools. - ---- - -## Quick Reference (Flash Summary) - -### Package: `pkg/build` -Provides project type detection, build configuration management, and cross-compilation utilities. - -**Key Exported Types and Functions** -* `Builder` (interface): Defines the interface for project-specific build implementations (Go, Node, PHP, etc.). -* `Config` / `BuildConfig` (structs): Hold runtime and file-based build parameters. -* `Artifact` (struct): Represents a build output file with path, OS, architecture, and checksum metadata. -* `ProjectType` (type): Constants identifying project types (e.g., `ProjectTypeGo`, `ProjectTypeWails`). -* `Archive`, `ArchiveXZ`, `ArchiveWithFormat`: Functions to create compressed archives (tar.gz, tar.xz, zip) of build artifacts. -* `Checksum`, `ChecksumAll`: Compute SHA256 hashes for build artifacts. -* `Discover`, `PrimaryType`: Detect project types based on marker files (e.g., `go.mod`, `package.json`). -* `LoadConfig`: Loads build settings from `.core/build.yaml`. - -**Dependencies** -* `pkg/io` -* `pkg/config` -* `pkg/build/signing` - -**Complexity Rating** -Moderate - ---- - -### Package: `pkg/container` -Manages the lifecycle of LinuxKit virtual machines using platform-native hypervisors. - -**Key Exported Types and Functions** -* `Manager` (interface): Defines container lifecycle operations (Run, Stop, List, Logs, Exec). -* `LinuxKitManager` (struct): Core implementation for managing LinuxKit VM instances. -* `Container` (struct): Represents a running or stopped VM instance with metadata like PID and status. -* `Hypervisor` (interface): Abstract interface for VM backends (QEMU, Hyperkit). -* `TemplateManager` (struct): Handles LinuxKit YAML templates and variable substitution. -* `State` (struct): Manages persistent storage of container metadata in JSON format. -* `DetectHypervisor`: Automatically selects the appropriate hypervisor for the current OS. -* `ApplyVariables`: Performs `${VAR:-default}` string interpolation in configuration files. - -**Dependencies** -* `pkg/io` - -**Complexity Rating** -Complex - ---- - -### Package: `pkg/process` -Advanced process management system featuring output streaming, circular buffering, and dependency-aware task execution. - -**Key Exported Types and Functions** -* `Service` (struct): Manages multiple processes with Core framework IPC integration. -* `Process` (struct): Represents a managed external process with non-blocking output capture. -* `Runner` (struct): Orchestrates complex task execution with dependency graph support (DAG). -* `RingBuffer` (struct): A thread-safe circular buffer for efficient process output storage. -* `RunOptions` (struct): Detailed configuration for spawning processes (env, dir, capture settings). -* `ActionProcessOutput`, `ActionProcessExited`: IPC message types for broadcasting process events via the Core framework. -* `Start`, `Run`, `Kill`: Global convenience functions for rapid process control. - -**Dependencies** -* `pkg/framework` - -**Complexity Rating** -Moderate/Complex - ---- - -### Package: `pkg/jobrunner` -A poll-dispatch automation system designed to process structural signals from issues or pull requests. - -**Key Exported Types and Functions** -* `Poller` (struct): Implements the main loop that discovers work from sources and dispatches to handlers. -* `PipelineSignal` (struct): A metadata snapshot of a work item (e.g., PR state, thread counts, mergeability). -* `JobSource` (interface): Interface for external systems (like GitHub) that provide actionable items. -* `JobHandler` (interface): Interface for logic that matches and executes actions on signals. -* `Journal` (struct): Provides persistent, date-partitioned JSONL audit logging for all actions. -* `ActionResult` (struct): Captures the success, failure, and duration of a completed job. - -**Dependencies** -* `pkg/log` - -**Complexity Rating** -Moderate diff --git a/docs/pkg-batch4-analysis.md b/docs/pkg-batch4-analysis.md deleted file mode 100644 index 346d8b7..0000000 --- a/docs/pkg-batch4-analysis.md +++ /dev/null @@ -1,366 +0,0 @@ -# Package Analysis — Batch 4 - -Generated by: gemini-batch-runner.sh -Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview -Date: 2026-02-09 -Packages: git repos gitea forge release -Total tokens: 92202 - ---- - -Here is the technical documentation for the analyzed packages, written from the perspective of a Senior Go Engineer. - -# Framework Package Documentation - -## 1. Package: `pkg/git` - -### Overview -The `git` package provides a high-level abstraction over local Git operations, specifically designed for multi-repo workspace management. It combines direct shell execution for complex operations (push/pull with interactive auth) with concurrent status checking. It is designed to run both as a standalone utility library and as a registered `framework.Service` within the Core application. - -### Public API - -**Types** -```go -type RepoStatus struct { - Name, Path string - Modified, Untracked, Staged, Ahead, Behind int - Branch string - Error error -} -func (s *RepoStatus) IsDirty() bool -func (s *RepoStatus) HasUnpushed() bool -func (s *RepoStatus) HasUnpulled() bool - -type StatusOptions struct { - Paths []string - Names map[string]string -} - -type PushResult struct { - Name, Path string - Success bool - Error error -} - -// Service integration -type Service struct { ... } -type ServiceOptions struct { WorkDir string } -``` - -**Functions** -```go -// Concurrent status checking -func Status(ctx context.Context, opts StatusOptions) []RepoStatus - -// Interactive operations (hooks into os.Stdin/Stdout) -func Push(ctx context.Context, path string) error -func Pull(ctx context.Context, path string) error -func PushMultiple(ctx context.Context, paths []string, names map[string]string) []PushResult - -// Error handling -func IsNonFastForward(err error) bool - -// Service Factory -func NewService(opts ServiceOptions) func(*framework.Core) (any, error) -``` - -### Internal Design -* **Shell Wrapper**: Uses `os/exec` to invoke the system `git` binary rather than using a native Go implementation (like go-git). This ensures 100% compatibility with the user's local git configuration (SSH keys, hooks, GPG signing). -* **Concurrency**: `Status()` uses a `sync.WaitGroup` pattern to check multiple repository statuses in parallel, significantly speeding up workspace checks. -* **Interactive Mode**: `Push` and `Pull` explicitly wire `os.Stdin` and `os.Stdout` to the subprocess to allow SSH passphrase prompts or GPG pin entry to function correctly in a terminal environment. -* **Service Pattern**: Implements the `framework.ServiceRuntime` interface, registering distinct Queries (`QueryStatus`) and Tasks (`TaskPush`) to decouple the UI/CLI from the git logic. - -### Dependencies -* `os/exec`: For invoking git commands. -* `forge.lthn.ai/core/cli/pkg/framework`: For service registration and message passing types. - -### Test Coverage Notes -* **Mocking**: Testing requires abstracting `exec.Command`. Since this package calls `exec.CommandContext` directly, tests likely require overriding a package-level variable or using a "fake exec" pattern during test initialization. -* **Parsing**: Unit tests should cover the parsing logic of `git status --porcelain` in `getStatus` to ensure modified/staged/untracked counts are accurate. -* **Concurrency**: Race detection should be enabled to ensure `Status()` result slice assignment is thread-safe (it uses index-based assignment, which is safe). - -### Integration Points -* **CLI**: The CLI command `core git status` consumes the `Service` via the framework's message bus. -* **Workspace Managers**: Packages managing multi-repo setups (like `pkg/repos`) use this to report health. - ---- - -## 2. Package: `pkg/repos` - -### Overview -This package manages the "Registry" of a multi-repository ecosystem. It acts as the source of truth for repository locations, types (foundation, module, product), and dependencies. It supports loading from a static `repos.yaml` or scanning the filesystem as a fallback. - -### Public API - -**Types** -```go -type Registry struct { - Repos map[string]*Repo - Defaults RegistryDefaults - ... -} - -type Repo struct { - Name, Type, Description, CI, Domain string - DependsOn []string - Docs bool - Path string // Computed -} - -type RepoType string // "foundation", "module", "product", "template" -``` - -**Functions** -```go -// Loading -func LoadRegistry(m io.Medium, path string) (*Registry, error) -func FindRegistry(m io.Medium) (string, error) -func ScanDirectory(m io.Medium, dir string) (*Registry, error) - -// Registry Methods -func (r *Registry) List() []*Repo -func (r *Registry) Get(name string) (*Repo, bool) -func (r *Registry) ByType(t string) []*Repo -func (r *Registry) TopologicalOrder() ([]*Repo, error) - -// Repo Methods -func (repo *Repo) Exists() bool -func (repo *Repo) IsGitRepo() bool -``` - -### Internal Design -* **Abstraction**: Uses `io.Medium` to abstract filesystem access, making the registry testable without disk I/O. -* **Computed Fields**: The YAML struct is separate from the logic; `LoadRegistry` enriches the raw data with computed absolute paths and back-references. -* **Graph Theory**: `TopologicalOrder` implements a Depth-First Search (DFS) with cycle detection (`visiting` vs `visited` maps) to resolve build orders based on the `depends_on` field. - -### Dependencies -* `gopkg.in/yaml.v3`: For parsing `repos.yaml`. -* `forge.lthn.ai/core/cli/pkg/io`: For filesystem abstraction (`io.Medium`). - -### Test Coverage Notes -* **Circular Dependencies**: Critical test cases must define a registry with `A->B->A` dependencies to ensure `TopologicalOrder` returns a clear error and doesn't stack overflow. -* **Path Expansion**: Verify `~` expansion logic works across different OS mocks in `LoadRegistry`. - -### Integration Points -* **Build System**: The build package uses `TopologicalOrder()` to determine the sequence in which to build libraries before products. -* **CI/CD**: Uses `Repo.Type` to apply different linting/testing rules (e.g., Foundation repos might require stricter coverage). - ---- - -## 3. Packages: `pkg/gitea` & `pkg/forge` - -*(Note: These packages share a very similar design pattern. `pkg/forge` is essentially a port of `pkg/gitea` for Forgejo.)* - -### Overview -These packages provide typed clients for Gitea and Forgejo instances. They abstract the underlying SDKs to provide "Configuration-Aware" clients that automatically resolve authentication (Config vs Env vs Flags) and provide specialized helper methods for AI-driven metadata extraction (`PRMeta`). - -### Public API (Common to both) - -**Types** -```go -type Client struct { ... } - -// Structural signals for AI analysis -type PRMeta struct { - Number int64 - Title, State, Author, Branch, BaseBranch string - Labels, Assignees []string - IsMerged bool - CommentCount int - ... -} -``` - -**Functions** -```go -// Construction -func New(url, token string) (*Client, error) -func NewFromConfig(flagURL, flagToken string) (*Client, error) - -// Meta-data Extraction -func (c *Client) GetPRMeta(owner, repo string, pr int64) (*PRMeta, error) -func (c *Client) GetCommentBodies(...) -func (c *Client) GetIssueBody(...) - -// Repo Management -func (c *Client) CreateMirror(...) // Gitea specific migration -func (c *Client) MigrateRepo(...) // Forgejo specific migration -func (c *Client) ListOrgRepos(...) -func (c *Client) ListUserRepos(...) -``` - -### Internal Design -* **Config Precedence Layer**: `ResolveConfig` implements a strict hierarchy: CLI Flags > Environment Variables > Config File (`~/.core/config.yaml`). This allows seamless switching between local dev and CI environments. -* **Dual-End Reader**: The `GetPRMeta` method aggregates data from multiple API endpoints (PR details + Issue Comments + Labels) into a flattened struct designed specifically to be fed into an LLM or policy engine. -* **Workarounds**: `pkg/forge/prs.go` implements a raw `net/http` PATCH request for `SetPRDraft` because the specific feature was missing or broken in the imported version of the Forgejo SDK. - -### Dependencies -* `code.gitea.io/sdk/gitea` (for `pkg/gitea`) -* `codeberg.org/mvdkleijn/forgejo-sdk` (for `pkg/forge`) -* `forge.lthn.ai/core/cli/pkg/config`: For persistent auth storage. - -### Test Coverage Notes -* **Draft Status**: The raw HTTP patch in `pkg/forge` needs integration testing against a real instance or a high-fidelity HTTP mock to ensure payload format matches Forgejo's API expectation. -* **Pagination**: `List*` methods implement manual pagination loops. Tests should simulate API responses with multiple pages to verify all items are collected. - -### Integration Points -* **CI Pipelines**: Used to fetch PR context for "Smart CI" decisions. -* **Migration Tools**: The `CreateMirror`/`MigrateRepo` functions are used to synchronize repositories between GitHub and local Gitea/Forgejo instances. - ---- - -## 4. Package: `pkg/release` - -### Overview -The `release` package allows fully automated releases. It handles Semantic Versioning detection, Conventional Commit parsing for changelogs, build orchestration, and publishing to multiple downstream targets (GitHub, Docker, LinuxKit, etc.). - -### Public API - -**Types** -```go -type Config struct { ... } // Maps to release.yaml -type Release struct { - Version string - Artifacts []build.Artifact - Changelog string - ... -} -``` - -**Functions** -```go -// Main Entry Points -func Run(ctx context.Context, cfg *Config, dryRun bool) (*Release, error) -func Publish(ctx context.Context, cfg *Config, dryRun bool) (*Release, error) -func RunSDK(ctx context.Context, cfg *Config, dryRun bool) (*SDKRelease, error) - -// Utilities -func DetermineVersion(dir string) (string, error) -func Generate(dir, fromRef, toRef string) (string, error) // Changelog -func IncrementVersion(current string) string -``` - -### Internal Design -* **Pipeline Architecture**: `Run()` executes a linear pipeline: - 1. **Versioner**: Checks Git tags -> parses SemVer -> increments patch (default). - 2. **Changelog**: Parses `git log` via `ParseCommitType` (Conventional Commits regex) -> Buckets by type (Feat, Fix) -> Renders Markdown. - 3. **Builder**: Delegates to `pkg/build` to compile binaries and generate checksums. - 4. **Publisher**: Iterates over `Config.Publishers`, instantiates specific strategies via a Factory pattern (`getPublisher`), and executes them. -* **Separation of Concerns**: `Publish()` exists separately from `Run()` to support CI workflows where the *Build* step is separate from the *Release* step. It locates pre-existing artifacts in `dist/`. -* **SDK Generation**: Includes a specialized sub-pipeline (`RunSDK`) that handles OpenAPI diffing and client generation. - -### Dependencies -* `forge.lthn.ai/core/cli/pkg/build`: For compiling artifacts. -* `forge.lthn.ai/core/cli/pkg/release/publishers`: Interface definitions for publishing targets. -* `golang.org/x/text`: For title casing in changelogs. - -### Test Coverage Notes -* **SemVer Logic**: Extensive unit tests needed for `DetermineVersion` and `IncrementVersion` covering edge cases (v-prefix, no tags, pre-releases). -* **Regex**: Validate `conventionalCommitRegex` against a corpus of valid and invalid commit messages to ensure changelogs are generated correctly. -* **Config Unmarshaling**: `LoadConfig` uses complex nesting; tests should verify that `release.yaml` maps correctly to the internal structs, especially the `map[string]any` used for publisher-specific config. - -### Integration Points -* **CI Runner**: This is the engine behind `core ci release`. -* **Build System**: Tightly coupled with `pkg/build`—it assumes artifacts are placed in `dist/` and accompanied by a `CHECKSUMS.txt`. - ---- - -## Quick Reference (Flash Summary) - -### Package: `pkg/git` -Provides utilities for git operations across multiple repositories and a service runtime for managing repository states. - -**Key Exported Types and Functions** -- `RepoStatus`: Struct representing the state of a repository (ahead/behind counts, dirty status, branch). -- `Status()`: Checks git status for multiple repositories in parallel using goroutines. -- `Push()` / `Pull()`: Performs git operations in interactive mode to support SSH passphrase prompts. -- `PushMultiple()`: Executes pushes for multiple repositories sequentially. -- `Service`: A framework-compatible service that handles git-related tasks and queries. -- `IsNonFastForward()`: Utility to detect specific git push rejection errors. - -**Dependencies** -- `pkg/framework` - -**Complexity Rating** -Moderate - ---- - -### Package: `pkg/repos` -Manages multi-repo workspaces by parsing a registry configuration and handling repository discovery. - -**Key Exported Types and Functions** -- `Registry`: Represents a collection of repositories defined in `repos.yaml`. -- `Repo`: Represents a single repository with metadata and dependency information. -- `LoadRegistry()`: Loads and parses the repository registry from a given storage medium. -- `FindRegistry()`: Searches for a `repos.yaml` file in local and config directories. -- `ScanDirectory()`: Fallback mechanism to generate a registry by scanning a filesystem for git folders. -- `TopologicalOrder()`: Sorts repositories based on their dependency graph for build ordering. - -**Dependencies** -- `pkg/io` - -**Complexity Rating** -Moderate - ---- - -### Package: `pkg/gitea` -A wrapper around the Gitea Go SDK for managing repositories, issues, and pull requests. - -**Key Exported Types and Functions** -- `Client`: Primary wrapper for the Gitea API client. -- `NewFromConfig()`: Resolves authentication (token/URL) from flags, environment, or config files. -- `GetPRMeta()`: Extracts structural metadata from a pull request for pipeline analysis. -- `ListOrgRepos()` / `ListUserRepos()`: Lists repositories for organizations or the authenticated user. -- `CreateMirror()`: Uses the migration API to set up a pull mirror from a remote source. -- `GetCommentBodies()`: Retrieves all text content for PR comments. - -**Dependencies** -- `pkg/log` -- `pkg/config` - -**Complexity Rating** -Moderate - ---- - -### Package: `pkg/forge` -A wrapper around the Forgejo Go SDK for repository management, issue tracking, and PR orchestration. - -**Key Exported Types and Functions** -- `Client`: Primary wrapper for the Forgejo API client. -- `NewFromConfig()`: Tiered configuration loader for Forgejo instance connectivity. -- `GetPRMeta()`: Collects PR metadata, including state, labels, and comment counts. -- `MergePullRequest()`: Merges a PR using squash, rebase, or merge styles. -- `SetPRDraft()`: Manages draft status via raw HTTP PATCH (working around SDK limitations). -- `MigrateRepo()`: Imports repositories and metadata from external services. - -**Dependencies** -- `pkg/log` -- `pkg/config` - -**Complexity Rating** -Moderate - ---- - -### Package: `pkg/release` -Orchestrates release automation, including changelog generation, versioning, and publishing to various targets. - -**Key Exported Types and Functions** -- `Generate()`: Parses conventional commits to create markdown changelogs. -- `DetermineVersion()`: Calculates the next semantic version based on git tags and commit history. -- `Run()` / `Publish()`: Orchestrates the full process of building, archiving, and distributing artifacts. -- `RunSDK()`: Handles OpenAPI-based SDK generation and breaking change detection. -- `LoadConfig()`: Parses `.core/release.yaml` to configure build targets and publishers. -- `Config`: Struct defining project metadata, build targets, and distribution channels (Docker, Homebrew, etc.). - -**Dependencies** -- `pkg/build` -- `pkg/io` -- `pkg/config` -- `pkg/log` - -**Complexity Rating** -Complex diff --git a/docs/pkg-batch5-analysis.md b/docs/pkg-batch5-analysis.md deleted file mode 100644 index 5fe5726..0000000 --- a/docs/pkg-batch5-analysis.md +++ /dev/null @@ -1,303 +0,0 @@ -# Package Analysis — Batch 5 - -Generated by: gemini-batch-runner.sh -Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview -Date: 2026-02-09 -Packages: agentci agentic ai rag -Total tokens: 78402 - ---- - -Here is the detailed documentation for the framework's AI and Agent capabilities. - -# Host-UK Core Framework: AI & Agent Packages - -This document outlines the architecture, API, and design patterns for the AI automation subsystem within the Core framework. These packages provide the foundation for LLM-assisted development, task management, and RAG (Retrieval Augmented Generation) context. - ---- - -## Package: `pkg/agentci` - -### 1. Overview -`pkg/agentci` serves as the configuration bridge between the Core config system and the Agent CI dispatch logic. Its primary purpose is to manage the definitions of "Agent Targets"—machines or environments capable of running AI workloads (e.g., specific GPU nodes or cloud runners)—allowing the job runner to dynamically load and dispatch tasks to active agents. - -### 2. Public API - -```go -type AgentConfig struct { - Host string `yaml:"host" mapstructure:"host"` - QueueDir string `yaml:"queue_dir" mapstructure:"queue_dir"` - ForgejoUser string `yaml:"forgejo_user" mapstructure:"forgejo_user"` - Model string `yaml:"model" mapstructure:"model"` - Runner string `yaml:"runner" mapstructure:"runner"` - Active bool `yaml:"active" mapstructure:"active"` -} - -// LoadAgents reads agent targets from config and returns a map suitable for the dispatch handler. -func LoadAgents(cfg *config.Config) (map[string]handlers.AgentTarget, error) - -// SaveAgent writes an agent config entry to the config file. -func SaveAgent(cfg *config.Config, name string, ac AgentConfig) error - -// RemoveAgent removes an agent from the config file. -func RemoveAgent(cfg *config.Config, name string) error - -// ListAgents returns all configured agents (active and inactive). -func ListAgents(cfg *config.Config) (map[string]AgentConfig, error) -``` - -### 3. Internal Design -* **Configuration Mapping**: The package acts as a Data Transfer Object (DTO) layer. It maps raw YAML/MapStructure data into the strictly typed `handlers.AgentTarget` struct required by the job runner. -* **Defaults Handling**: `LoadAgents` applies specific logic defaults (e.g., default queue directories, default models like "sonnet") to ensure the system works with minimal configuration. - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/config`: For reading/writing the persistent configuration state. -* `forge.lthn.ai/core/cli/pkg/jobrunner/handlers`: To map local config structs to the runtime types used by the job dispatch system. - -### 5. Test Coverage Notes -* **Configuration Persistence**: Tests should verify that `SaveAgent` correctly updates the underlying config file and that `LoadAgents` retrieves it accurately. -* **Validation**: Edge cases where `Host` is empty or defaults are applied need unit testing. - -### 6. Integration Points -* **Job Runner**: The main dispatch loop calls `LoadAgents` to determine where AI jobs can be sent. -* **CLI Tools**: CLI commands for managing build agents (e.g., `core agent add`) would use `SaveAgent` and `ListAgents`. - ---- - -## Package: `pkg/agentic` - -### 1. Overview -`pkg/agentic` is the heavy-lifting package for AI-assisted task management. It provides both a REST client for the `core-agentic` backend service and a Core Framework Service implementation to execute local AI operations (Git automation, context gathering, and Claude invocations). - -### 2. Public API - -**Client & API Types** -```go -type Client struct { /* ... */ } -type Task struct { /* ID, Title, Priority, Status, etc. */ } -type TaskContext struct { /* Task, Files, GitStatus, RAGContext, etc. */ } - -// Client Factory -func NewClient(baseURL, token string) *Client -func NewClientFromConfig(cfg *Config) *Client - -// API Operations -func (c *Client) ListTasks(ctx context.Context, opts ListOptions) ([]Task, error) -func (c *Client) GetTask(ctx context.Context, id string) (*Task, error) -func (c *Client) ClaimTask(ctx context.Context, id string) (*Task, error) -func (c *Client) UpdateTask(ctx context.Context, id string, update TaskUpdate) error -func (c *Client) CompleteTask(ctx context.Context, id string, result TaskResult) error -func (c *Client) Ping(ctx context.Context) error -``` - -**Git & Automation Ops** -```go -func AutoCommit(ctx context.Context, task *Task, dir string, message string) error -func CreatePR(ctx context.Context, task *Task, dir string, opts PROptions) (string, error) -func CreateBranch(ctx context.Context, task *Task, dir string) (string, error) -func CommitAndSync(ctx context.Context, client *Client, task *Task, dir string, message string, progress int) error -func BuildTaskContext(task *Task, dir string) (*TaskContext, error) -``` - -**Framework Service** -```go -type Service struct { /* ... */ } -type TaskCommit struct { Path, Name string; CanEdit bool } -type TaskPrompt struct { Prompt, WorkDir string; AllowedTools []string } - -func NewService(opts ServiceOptions) func(*framework.Core) (any, error) -``` - -### 3. Internal Design -* **Service Runtime**: Implements the `framework.ServiceRuntime` pattern, registering task handlers (`TaskCommit`, `TaskPrompt`) allowing other parts of the Core framework to request AI actions via the event bus. -* **Heuristic Context Gathering**: `BuildTaskContext` uses a mix of git commands (`git grep`, `git status`) and file reading to assemble a prompt context for the LLM automatically. -* **Tooling Integration**: Wraps the `claude` CLI binary directly via `os/exec` to perform actual inference, exposing tool capabilities (Bash, Read, Write) based on permissions. -* **Embeds**: Uses Go embed (`//go:embed`) to store system prompts (e.g., `prompts/commit.md`) within the binary. - -### 4. Dependencies -* `pkg/framework`: To integrate as a background service. -* `pkg/ai`: Uses `ai.QueryRAGForTask` to inject documentation context into task execution. -* `pkg/config` & `pkg/io`: For loading credentials and file operations. -* `pkg/log`: Structured logging. - -### 5. Test Coverage Notes -* **HTTP Client**: Requires mocking `http.Client` to verify request payload serialization and error handling for 4xx/5xx responses. -* **Git Operations**: Needs integration tests with a temporary git repository to verify `AutoCommit` and branch creation logic. -* **Context Building**: Unit tests should verify `extractKeywords` and `GatherRelatedFiles` logic on a known file structure. - -### 6. Integration Points -* **Developer CLI**: The `core` CLI uses this package to fetch tasks (`core task list`) and start work (`core task start`). -* **Agents**: Autonomous agents use the `Client` to claim work and the `Service` to execute the necessary code changes. - ---- - -## Package: `pkg/ai` - -### 1. Overview -`pkg/ai` is the canonical entry point and facade for the framework's AI capabilities. It unifies RAG (from `pkg/rag`) and metrics collection, providing a simplified interface for other packages to consume AI features without managing low-level clients. - -### 2. Public API - -```go -// Metrics -type Event struct { /* Type, Timestamp, AgentID, etc. */ } -func Record(event Event) (err error) -func ReadEvents(since time.Time) ([]Event, error) -func Summary(events []Event) map[string]any - -// RAG Facade -type TaskInfo struct { Title, Description string } -func QueryRAGForTask(task TaskInfo) string -``` - -### 3. Internal Design -* **Facade Pattern**: Hides the initialization complexity of `rag.QdrantClient` and `rag.OllamaClient`. `QueryRAGForTask` instantiates these on demand with sensible defaults, ensuring graceful degradation (returns empty string) if services aren't running. -* **Dependency Inversion**: It defines `TaskInfo` locally to accept task data from `pkg/agentic` without importing `pkg/agentic` directly, breaking potential circular dependencies. -* **Local Metrics Store**: Implements a lightweight, file-based (JSONL) telemetry system stored in `~/.core/ai/metrics`. - -### 4. Dependencies -* `pkg/rag`: For vector database and embedding operations. -* `pkg/agentic`: (Conceptually composed, though `ai` is the higher-level import). - -### 5. Test Coverage Notes -* **Metrics I/O**: Tests for `Record` and `ReadEvents` to ensure concurrent writes to the JSONL file do not corrupt data and dates are filtered correctly. -* **Graceful Failure**: `QueryRAGForTask` must be tested to ensure it does not panic if Qdrant/Ollama are offline. - -### 6. Integration Points -* **Agentic Context**: `pkg/agentic` calls `QueryRAGForTask` to enhance prompts. -* **Dashboard**: A UI or CLI dashboard would consume `Summary` to show AI usage stats. - ---- - -## Package: `pkg/rag` - -### 1. Overview -`pkg/rag` implements the Retrieval Augmented Generation pipeline. It handles the full lifecycle of document processing: reading Markdown files, chunking them intelligently, generating embeddings via Ollama, storing them in Qdrant, and performing semantic search. - -### 2. Public API - -**Ingestion & Chunking** -```go -type IngestConfig struct { /* Directory, Collection, ChunkConfig */ } -type Chunk struct { Text, Section string; Index int } - -func DefaultIngestConfig() IngestConfig -func Ingest(ctx context.Context, qdrant *QdrantClient, ollama *OllamaClient, cfg IngestConfig, progress IngestProgress) (*IngestStats, error) -func ChunkMarkdown(text string, cfg ChunkConfig) []Chunk -``` - -**Querying** -```go -type QueryConfig struct { /* Collection, Limit, Threshold */ } -type QueryResult struct { Text, Source, Score float32 /* ... */ } - -func Query(ctx context.Context, qdrant *QdrantClient, ollama *OllamaClient, query string, cfg QueryConfig) ([]QueryResult, error) -func FormatResultsContext(results []QueryResult) string -``` - -**Clients** -```go -type QdrantClient struct { /* ... */ } -func NewQdrantClient(cfg QdrantConfig) (*QdrantClient, error) - -type OllamaClient struct { /* ... */ } -func NewOllamaClient(cfg OllamaConfig) (*OllamaClient, error) -func (o *OllamaClient) Embed(ctx context.Context, text string) ([]float32, error) -``` - -### 3. Internal Design -* **Pipeline Architecture**: Separation of concerns between `Ingest` (Crawler/Loader), `Chunk` (Processor), `Embed` (Transformation), and `Qdrant` (Storage). -* **Semantic Chunking**: `ChunkMarkdown` is designed specifically for documentation, splitting by H2 (`##`) headers first, then by paragraphs, maintaining overlap to preserve context. -* **Adapter Pattern**: Wraps external libraries (`qdrant-go-client`, `ollama/api`) to strictly define the interface required by the Core framework. - -### 4. Dependencies -* `github.com/qdrant/go-client/qdrant`: Vector database driver. -* `github.com/ollama/ollama/api`: Embedding model API. -* `pkg/log`: Error reporting. - -### 5. Test Coverage Notes -* **Chunking Logic**: Critical to test `ChunkMarkdown` with various markdown structures (headers, lists, code blocks) to ensure chunks don't break mid-sentence or lose header context. -* **Embed Dimensions**: Tests should verify that the vector size created by the Ollama client matches the collection configuration in Qdrant. -* **Integration**: Requires running Qdrant and Ollama containers for full integration testing. - -### 6. Integration Points -* **CLI Admin**: An ingestion command (e.g., `core docs ingest`) would trigger the `Ingest` function. -* **AI Package**: `pkg/ai` consumes `Query` to augment prompts. - ---- - -## Quick Reference (Flash Summary) - -### Package: `pkg/agentci` -**Description**: Manages configuration and lifecycle for AgentCI dispatch targets and remote runner machines. - -**Key Exported Types and Functions**: -- `AgentConfig`: Struct representing an agent's host, queue directory, model, and runner type. -- `LoadAgents`: Reads agent configurations from the global config and maps them to dispatch targets. -- `SaveAgent`: Adds or updates a specific agent entry in the configuration file. -- `RemoveAgent`: Deletes an agent configuration entry by name. -- `ListAgents`: Retrieves all configured agents, including inactive ones. - -**Dependencies**: -- `pkg/config` -- `pkg/jobrunner/handlers` - -**Complexity**: Simple - ---- - -### Package: `pkg/agentic` -**Description**: Provides an API client and automation tools for AI-assisted task management, git operations, and context gathering. - -**Key Exported Types and Functions**: -- `Client`: API client for interacting with the core-agentic task service. -- `Task` / `TaskUpdate`: Data structures representing development tasks and their status updates. -- `BuildTaskContext`: Aggregates task details, relevant file contents, git status, and RAG data for AI consumption. -- `AutoCommit`: Automatically stages changes and creates a git commit with a task reference. -- `CreatePR`: Uses the `gh` CLI to create a pull request based on task metadata. -- `Service`: A framework-compatible service for handling asynchronous AI tasks like automated commits and prompts. -- `LoadConfig`: Multi-source configuration loader (Env, `.env` files, YAML) for API credentials. - -**Dependencies**: -- `pkg/log` -- `pkg/config` -- `pkg/io` -- `pkg/ai` -- `pkg/framework` - -**Complexity**: Complex - ---- - -### Package: `pkg/ai` -**Description**: Unified entry point for AI features, orchestrating vector search, task context, and usage metrics. - -**Key Exported Types and Functions**: -- `Event`: Represents a recorded AI or security metric event for telemetry. -- `Record`: Persists metric events to daily JSONL files in the user's home directory. -- `ReadEvents` / `Summary`: Retrieves and aggregates stored metrics for reporting. -- `QueryRAGForTask`: High-level helper that queries the vector database for documentation relevant to a specific task. -- `TaskInfo`: A minimal structure used to pass task data to the RAG system without circular dependencies. - -**Dependencies**: -- `pkg/rag` - -**Complexity**: Moderate - ---- - -### Package: `pkg/rag` -**Description**: Implements Retrieval-Augmented Generation (RAG) using Qdrant for vector storage and Ollama for embeddings. - -**Key Exported Types and Functions**: -- `QdrantClient`: Wrapper for the Qdrant database providing collection management and vector search. -- `OllamaClient`: Client for generating text embeddings using local models (e.g., `nomic-embed-text`). -- `ChunkMarkdown`: Semantically splits markdown text into smaller chunks based on headers and paragraphs. -- `Ingest`: Processes a directory of markdown files, generates embeddings, and stores them in Qdrant. -- `Query`: Performs vector similarity searches and filters results by score thresholds. -- `FormatResultsContext`: Formats retrieved document chunks into XML-style tags for LLM prompt injection. - -**Dependencies**: -- `pkg/log` - -**Complexity**: Moderate diff --git a/docs/pkg-batch6-analysis.md b/docs/pkg-batch6-analysis.md deleted file mode 100644 index 3847dc3..0000000 --- a/docs/pkg-batch6-analysis.md +++ /dev/null @@ -1,520 +0,0 @@ -# Package Analysis — Batch 6 - -Generated by: gemini-batch-runner.sh -Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview -Date: 2026-02-09 -Packages: ansible deploy devops framework mcp plugin unifi webview ws collect i18n cache -Total tokens: 458153 - ---- - -# Framework Documentation - -This document provides a detailed technical analysis of the core packages within the framework. - ---- - -## === Package: pkg/ansible === - -### 1. Overview -A native Go implementation of an Ansible playbook runner. Unlike wrappers that call the `ansible` CLI, this package parses YAML playbooks and inventories, handles variable interpolation (Jinja2-style), and executes modules directly over SSH or local connections. It is designed for embedding configuration management directly into the application without external dependencies. - -### 2. Public API - -#### Executor -The primary entry point for running playbooks. -```go -type Executor struct { ... } -func NewExecutor(basePath string) *Executor -func (e *Executor) SetInventory(path string) error -func (e *Executor) SetInventoryDirect(inv *Inventory) -func (e *Executor) SetVar(key string, value any) -func (e *Executor) Run(ctx context.Context, playbookPath string) error -func (e *Executor) Close() -``` - -#### Callback Hooks -Callbacks to monitor execution progress. -```go -e.OnPlayStart = func(play *Play) -e.OnTaskStart = func(host string, task *Task) -e.OnTaskEnd = func(host string, task *Task, result *TaskResult) -e.OnPlayEnd = func(play *Play) -``` - -#### Types -```go -type Playbook struct { Plays []Play } -type Play struct { ... } -type Task struct { ... } -type Inventory struct { ... } -type TaskResult struct { Changed, Failed bool; Msg, Stdout string; ... } -``` - -### 3. Internal Design -* **Parser**: `parser.go` handles recursive YAML parsing, resolving role paths, and `include_tasks`. -* **Module Dispatch**: `modules.go` contains a switch statement dispatching tasks to native Go functions (e.g., `moduleShell`, `moduleCopy`, `moduleApt`) based on the task name. -* **Templating**: Implements a custom Jinja2-subset parser in `executor.go` (`templateString`, `resolveExpr`) to handle variables like `{{ var }}` and filters. -* **SSH Abstraction**: `ssh.go` wraps `golang.org/x/crypto/ssh` to handle connection pooling, key management, and `sudo` escalation (become). - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/log`: structured logging. -* `golang.org/x/crypto/ssh`: Underlying SSH transport. -* `gopkg.in/yaml.v3`: YAML parsing. - -### 5. Test Coverage Notes -* **Templating Logic**: Critical to test variable resolution, filters (`default`, `bool`), and nested lookups. -* **Module Idempotency**: Verify that file/apt modules return `Changed: false` when state matches. -* **SSH/Sudo**: Test `become` functionality with password handling. - -### 6. Integration Points -Used by higher-level orchestration tools or the `pkg/devops` package to provision environments. - ---- - -## === Package: pkg/devops === - -### 1. Overview -Manages a portable, sandboxed development environment using LinuxKit images. It handles the lifecycle (download, install, boot, stop) of a QEMU-based VM and provides utilities to bridge the host and VM (SSH forwarding, file mounting). - -### 2. Public API - -#### Lifecycle Management -```go -type DevOps struct { ... } -func New(m io.Medium) (*DevOps, error) -func (d *DevOps) Install(ctx, progress func(int64, int64)) error -func (d *DevOps) Boot(ctx, opts BootOptions) error -func (d *DevOps) Stop(ctx) error -func (d *DevOps) Status(ctx) (*DevStatus, error) -``` - -#### Interaction -```go -func (d *DevOps) Shell(ctx, opts ShellOptions) error -func (d *DevOps) Serve(ctx, projectDir, opts ServeOptions) error -func (d *DevOps) Test(ctx, projectDir, opts TestOptions) error -func (d *DevOps) Claude(ctx, projectDir, opts ClaudeOptions) error -``` - -### 3. Internal Design -* **Image Management**: `images.go` handles versioning and downloading QCOW2 images from GitHub/CDN. -* **Container Abstraction**: Delegates low-level VM execution to `pkg/container` (likely a wrapper around QEMU/LinuxKit). -* **SSH Bridging**: Heavily relies on `exec.Command("ssh", ...)` to tunnel ports, mount filesystems via SSHFS, and forward agents. -* **Auto-Detection**: `DetectServeCommand` and `DetectTestCommand` inspect project files (`package.json`, `go.mod`) to determine how to run projects. - -### 4. Dependencies -* `pkg/container`: VM runtime management. -* `pkg/io`: Filesystem abstraction. -* `pkg/config`: Configuration loading. - -### 5. Test Coverage Notes -* **Command Detection**: Unit tests for `Detect*Command` with various mock file structures. -* **SSH Config**: Verify `ensureHostKey` correctly parses and updates `known_hosts`. - -### 6. Integration Points -The primary interface for CLI commands (`core dev ...`). Bridges `pkg/mcp` agents into a sandboxed environment. - ---- - -## === Package: pkg/framework === - -### 1. Overview -A facade package that re-exports types and functions from `pkg/framework/core`. It serves as the primary entry point for the Dependency Injection (DI) framework, providing a cleaner import path for consumers. - -### 2. Public API -Re-exports `Core`, `Option`, `Message`, `Startable`, `Stoppable`, and constructors like `New`, `WithService`, `ServiceFor`. - -### 3. Internal Design -Purely structural; contains type aliases and variable assignments to expose the internal `core` package. - -### 4. Dependencies -* `forge.lthn.ai/core/cli/pkg/framework/core` - -### 5. Test Coverage Notes -No logic to test directly; coverage belongs in `pkg/framework/core`. - -### 6. Integration Points -Imported by `main.go` and all service packages to register themselves with the DI container. - ---- - -## === Package: pkg/mcp === - -### 1. Overview -Implements a Model Context Protocol (MCP) server. It acts as a bridge between AI models (like Claude) and the system tools, exposing file operations, process management, web browsing, and RAG capabilities as callable tools. - -### 2. Public API -```go -type Service struct { ... } -func New(opts ...Option) (*Service, error) -func (s *Service) Run(ctx context.Context) error -func (s *Service) ServeTCP(ctx, addr string) error -func (s *Service) ServeStdio(ctx) error -``` - -#### Configuration Options -```go -func WithWorkspaceRoot(root string) Option -func WithProcessService(svc *process.Service) Option -func WithWSHub(hub *ws.Hub) Option -``` - -### 3. Internal Design -* **Tool Registry**: Registers functions (e.g., `s.readFile`, `s.processStart`) with the MCP SDK. -* **Sandboxing**: `WithWorkspaceRoot` creates a restricted `io.Medium` to prevent AI from accessing files outside the workspace. -* **Subsystems**: Segregates tools into files (`tools_rag.go`, `tools_webview.go`, etc.). -* **Transports**: Supports Stdio (for CLI pipes), TCP, and Unix sockets. - -### 4. Dependencies -* `github.com/modelcontextprotocol/go-sdk`: MCP protocol implementation. -* `pkg/process`, `pkg/ws`, `pkg/rag`, `pkg/webview`: Capability providers. -* `pkg/io`: Filesystem access. - -### 5. Test Coverage Notes -* **Security**: Verify `WithWorkspaceRoot` actually prevents accessing `/etc/passwd`. -* **Tool I/O**: Ensure JSON inputs/outputs for tools map correctly to internal service calls. - -### 6. Integration Points -Runs as a standalone server or subprocess for AI agents. Consumes `pkg/process` and `pkg/webview`. - ---- - -## === Package: pkg/plugin === - -### 1. Overview -Provides a plugin system for the CLI, allowing extensions to be installed from GitHub. It manages a local registry of installed plugins and handles their lifecycle (install, update, remove). - -### 2. Public API -```go -type Plugin interface { Name(); Version(); Init(); Start(); Stop() } -type Registry struct { ... } -func NewRegistry(m io.Medium, basePath string) *Registry -func (r *Registry) List() []*PluginConfig - -type Installer struct { ... } -func (i *Installer) Install(ctx, source string) error // source: "org/repo" -func (i *Installer) Update(ctx, name string) error -``` - -### 3. Internal Design -* **Manifest**: Relies on `plugin.json` in the root of the plugin repo. -* **Git Integration**: Uses the `gh` CLI via `exec` to clone/pull repositories. -* **Persistence**: Stores plugin metadata in a `registry.json` file. - -### 4. Dependencies -* `pkg/io`: Filesystem access. -* `pkg/framework/core`: Error handling. -* External `gh` and `git` binaries. - -### 5. Test Coverage Notes -* **Manifest Validation**: Test valid/invalid `plugin.json` parsing. -* **Source Parsing**: Test parsing of `org/repo`, `org/repo@v1`, etc. - -### 6. Integration Points -Used by the main CLI application to load dynamic commands at startup. - ---- - -## === Package: pkg/unifi === - -### 1. Overview -A strongly-typed client for Ubiquiti UniFi controllers. It wraps the `unpoller` SDK but adds configuration resolution (config file -> env var -> flags) and specific helper methods for data extraction not easily accessible in the raw SDK. - -### 2. Public API -```go -func NewFromConfig(...) (*Client, error) -func (c *Client) GetClients(filter ClientFilter) ([]*uf.Client, error) -func (c *Client) GetDeviceList(site, type string) ([]DeviceInfo, error) -func (c *Client) GetRoutes(site string) ([]Route, error) -func (c *Client) GetNetworks(site string) ([]NetworkConf, error) -``` - -### 3. Internal Design -* **Config Cascade**: `ResolveConfig` logic ensures hierarchical configuration overrides. -* **Raw API Fallback**: Methods like `GetRoutes` and `GetNetworks` bypass the SDK's high-level structs to hit specific API endpoints (`/api/s/%s/stat/routing`) and decode into custom structs. - -### 4. Dependencies -* `github.com/unpoller/unifi/v5`: Base SDK. -* `pkg/config`: Config file management. - -### 5. Test Coverage Notes -* **Config Resolution**: Verify priority order (Flag > Env > Config). -* **JSON Unmarshalling**: Test `GetRoutes`/`GetNetworks` against sample JSON responses from a controller. - -### 6. Integration Points -Used by network management plugins or diagnostic tools. - ---- - -## === Package: pkg/webview === - -### 1. Overview -A browser automation package using the Chrome DevTools Protocol (CDP). It is designed for headless testing, scraping, and AI-driven interaction. It supports advanced features like Angular-specific waiting strategies. - -### 2. Public API -```go -type Webview struct { ... } -func New(opts ...Option) (*Webview, error) -func (wv *Webview) Navigate(url string) error -func (wv *Webview) Click(selector string) error -func (wv *Webview) Type(selector, text string) error -func (wv *Webview) Screenshot() ([]byte, error) -func (wv *Webview) Evaluate(script string) (any, error) -``` - -#### Angular Helpers -```go -func NewAngularHelper(wv *Webview) *AngularHelper -func (ah *AngularHelper) WaitForAngular() error -func (ah *AngularHelper) GetNgModel(selector string) (any, error) -``` - -### 3. Internal Design -* **CDP Client**: `cdp.go` implements a raw WebSocket client for the DevTools protocol, managing message IDs and event dispatching. -* **Action Sequence**: `actions.go` implements the Command pattern (`Action` interface) to chain browser interactions. -* **Angular Awareness**: `angular.go` injects JS to probe `window.ng` or `getAllAngularRootElements` to interact with Angular's Zone.js and component state. - -### 4. Dependencies -* `github.com/gorilla/websocket`: WebSocket transport. - -### 5. Test Coverage Notes -* **CDP Protocol**: Mock the WebSocket server to ensure correct message serialization/response handling. -* **Angular Helpers**: Requires an actual Angular app (or mock environment) to verify Zone.js stabilization logic. - -### 6. Integration Points -Used by `pkg/mcp` to expose browser tools to AI agents. - ---- - -## === Package: pkg/ws === - -### 1. Overview -A concurrent WebSocket hub implementation. It handles client registration, broadcasting, and channel-based subscriptions (e.g., subscribing only to logs for a specific process). - -### 2. Public API -```go -type Hub struct { ... } -func NewHub() *Hub -func (h *Hub) Run(ctx) -func (h *Hub) Handler() http.HandlerFunc -func (h *Hub) SendProcessOutput(id, output string) error -func (h *Hub) SendEvent(type string, data any) error -``` - -### 3. Internal Design -* **Hub Pattern**: Central `Hub` struct manages a map of clients and channels. Uses unbuffered channels for registration to avoid race conditions. -* **Channel Routing**: Maintains a `map[string]map[*Client]bool` to route messages efficiently to subscribers. -* **Goroutines**: Each client spawns a `readPump` and `writePump` to handle I/O concurrently. - -### 4. Dependencies -* `github.com/gorilla/websocket` - -### 5. Test Coverage Notes -* **Concurrency**: Test registering/unregistering clients while broadcasting heavily. -* **Subscription**: Verify messages only go to subscribed clients. - -### 6. Integration Points -Used by `pkg/mcp` to stream process output to a web UI. - ---- - -## === Package: pkg/collect === - -### 1. Overview -A data collection pipeline for gathering data from various sources (GitHub, Forums, Market Data, Papers). It standardizes the collection process into a `Collector` interface and handles common concerns like rate limiting, state tracking (resume support), and formatting. - -### 2. Public API -```go -type Collector interface { - Name() string - Collect(ctx, cfg *Config) (*Result, error) -} -type Excavator struct { Collectors []Collector ... } -func (e *Excavator) Run(ctx, cfg) (*Result, error) -``` - -### 3. Internal Design -* **Excavator**: The orchestrator that runs collectors sequentially. -* **RateLimiter**: Implements token bucket-like delays per source type (e.g., GitHub, CoinGecko). -* **State Persistence**: Saves a JSON cursor file to resume interrupted collections. -* **Formatters**: `process.go` converts raw HTML/JSON into Markdown for easier consumption by LLMs. - -### 4. Dependencies -* `pkg/io`: File storage. -* `golang.org/x/net/html`: HTML parsing for forums/papers. -* `gh` CLI: Used for GitHub data fetching. - -### 5. Test Coverage Notes -* **HTML Parsing**: Test `ParsePostsFromHTML` with sample forum HTML. -* **Rate Limit**: Verify `Wait` respects context cancellation and time delays. - -### 6. Integration Points -Used as a standalone CLI command or by AI agents to gather context. - ---- - -## === Package: pkg/i18n === - -### 1. Overview -A sophisticated internationalization library that goes beyond simple key-value lookups. It includes a grammar engine to handle pluralization, verb conjugation, and semantic sentence generation ("Subject verbed object"). - -### 2. Public API -```go -func T(key string, args ...any) string // Main translation function -func S(noun string, value any) *Subject // Create a semantic subject -func N(format string, value any) string // Number formatting -func SetLanguage(lang string) error -``` - -### 3. Internal Design -* **Grammar Engine**: `grammar.go` applies rules for past tense, gerunds, and pluralization based on language-specific JSON rules or algorithmic fallbacks. -* **Namespace Handlers**: `handler.go` intercepts keys like `i18n.count.*` or `i18n.done.*` to auto-generate phrases based on the grammar engine. -* **Loader**: `loader.go` flattens nested JSON translation files and extracts grammar rules (`gram.verb.*`). - -### 4. Dependencies -* `golang.org/x/text/language`: Standard language tag parsing. - -### 5. Test Coverage Notes -* **Pluralization**: Test complex rules (e.g., Slavic/Arabic plural categories). -* **Grammar generation**: Test `PastTense` and `Gerund` for regular and irregular English verbs. - -### 6. Integration Points -Used pervasively across the CLI for all user-facing output. - ---- - -## === Package: pkg/cache === - -### 1. Overview -A simple, file-based JSON cache with Time-To-Live (TTL) support. - -### 2. Public API -```go -func New(m io.Medium, baseDir string, ttl time.Duration) (*Cache, error) -func (c *Cache) Get(key string, dest interface{}) (bool, error) -func (c *Cache) Set(key string, data interface{}) error -``` - -### 3. Internal Design -* Stores data as JSON files: `{ "data": ..., "expires_at": ... }`. -* Uses `pkg/io` abstraction for storage independence. - -### 4. Dependencies -* `pkg/io` - -### 5. Test Coverage Notes -* **Expiry**: Verify `Get` returns false after TTL expires. -* **Serialization**: Ensure struct round-tripping works correctly. - -### 6. Integration Points -Used by `pkg/collect` or `pkg/plugin` to cache API responses (e.g., GitHub releases). - ---- - -## Quick Reference (Flash Summary) - -### pkg/ansible -**Description:** Implements a Go-based Ansible-lite engine for executing playbooks and roles over SSH with YAML parsing and fact gathering. -- **Executor (Type):** Main runner that manages inventory, variables, and execution state. -- **NewExecutor (Func):** Initialises the executor with a base path for roles and playbooks. -- **Task (Type):** Represents a single Ansible task with module parameters and conditional logic. -- **Run (Func):** Parses and executes a playbook from a file path. -- **Inventory (Type):** Holds the host and group structure for targeting remote machines. -**Dependencies:** `pkg/log` -**Complexity:** Complex - -### pkg/devops -**Description:** Manages a portable development environment using LinuxKit VM images and QEMU/SSH integration. -- **DevOps (Type):** Core service for environment lifecycle, mounting, and tool execution. -- **Boot (Func):** Configures and starts the dev environment container. -- **Claude (Func):** Launches a sandboxed AI session with project mounting and auth forwarding. -- **Serve (Func):** Auto-detects project types and runs local development servers inside the VM. -- **ImageManager (Type):** Handles downloading and updating dev environment system images. -**Dependencies:** `pkg/config`, `pkg/container`, `pkg/io`, `pkg/devops/sources` -**Complexity:** Moderate - -### pkg/framework -**Description:** Provides a facade for the Core dependency injection and service runtime framework. -- **Core (Type):** The central DI container and service registry. -- **New (Func):** Creates and initialises a new Core instance. -- **ServiceFor (Func):** Retrieves a type-safe service from the container by name. -- **Runtime (Type):** Manages the lifecycle and configuration of application services. -**Dependencies:** `pkg/framework/core` -**Complexity:** Simple - -### pkg/mcp -**Description:** Implements a Model Context Protocol (MCP) server providing filesystem, process, and RAG tools to AI agents. -- **Service (Type):** The MCP server instance managing tools and transport. -- **New (Func):** Initialises the server with workspace sandboxing and optional service integration. -- **WithWorkspaceRoot (Option):** Restricts file operations to a specific directory for security. -- **ServeTCP / ServeStdio (Func):** Transport-specific server implementations. -- **Subsystem (Interface):** Allows external packages to register custom toolsets. -**Dependencies:** `pkg/io`, `pkg/io/local`, `pkg/log`, `pkg/process`, `pkg/ws`, `pkg/ai`, `pkg/rag`, `pkg/webview` -**Complexity:** Complex - -### pkg/plugin -**Description:** A dynamic plugin system that manages gits-based extensions for the core CLI. -- **Plugin (Interface):** Defines the lifecycle (Init/Start/Stop) for extensions. -- **Registry (Type):** Manages metadata and persistence for installed plugins. -- **Installer (Type):** Handles git-based installation and updates from GitHub. -- **Loader (Type):** Discovers and loads plugin manifests from the filesystem. -**Dependencies:** `pkg/framework/core`, `pkg/io` -**Complexity:** Moderate - -### pkg/unifi -**Description:** A wrapper for the UniFi SDK providing simplified access to network controller devices, clients, and routing. -- **Client (Type):** Main API client for interacting with UniFi controllers. -- **NewFromConfig (Func):** Resolves credentials from config/env and initialises a client. -- **GetClients (Func):** Returns a filtered list of connected wired and wireless clients. -- **GetDeviceList (Func):** Returns flat metadata for infrastructure hardware (APs, Switches, Gateways). -**Dependencies:** `pkg/config`, `pkg/log` -**Complexity:** Simple - -### pkg/webview -**Description:** Provides browser automation and framework-specific testing (Angular) via the Chrome DevTools Protocol (CDP). -- **Webview (Type):** High-level controller for browser navigation and interaction. -- **CDPClient (Type):** Manages raw WebSocket communication with Chrome. -- **AngularHelper (Type):** Specialized tools for waiting on Zone.js stability and interacting with Angular components. -- **ConsoleWatcher (Type):** Captures and filters browser console logs and exceptions. -- **ActionSequence (Type):** Chains multiple browser interactions (click, type, navigate) into a single execution. -**Dependencies:** None -**Complexity:** Complex - -### pkg/ws -**Description:** Implements a WebSocket hub for real-time message broadcasting and channel-based subscriptions. -- **Hub (Type):** Manages client connections, message loops, and channel routing. -- **Run (Func):** Starts the central event loop for broadcasting and registration. -- **Broadcast (Func):** Sends a message to every connected client. -- **SendToChannel (Func):** Targets messages to clients subscribed to specific topics (e.g., process logs). -**Dependencies:** None -**Complexity:** Moderate - -### pkg/collect -**Description:** An orchestration subsystem for scraping and processing data from GitHub, forums, market APIs, and academic sources. -- **Collector (Interface):** Standard interface for data sources (e.g., `GitHubCollector`, `BitcoinTalkCollector`). -- **Excavator (Type):** Orchestrates multiple collectors with rate limiting and state resume support. -- **Processor (Type):** Converts raw HTML/JSON data into cleaned Markdown files. -- **RateLimiter (Type):** Manages per-source API delays to prevent IP bans. -- **State (Type):** Persists progress to allow incremental collection runs. -**Dependencies:** `pkg/framework/core`, `pkg/io` -**Complexity:** Complex - -### pkg/i18n -**Description:** A localization engine supporting nested translations, grammatical rules (plurals, gender, verbs), and semantic composition. -- **Service (Type):** Manages loaded locales and message resolution logic. -- **T / Raw (Func):** Translates keys with or without automatic grammatical composition. -- **Subject (Type):** Provides context (count, gender, formality) for semantic intent templates. -- **RegisterLocales (Func):** Allows packages to register embedded translation files. -- **GrammarData (Type):** Defines language-specific rules for past tense, gerunds, and articles. -**Dependencies:** None -**Complexity:** Complex - -### pkg/cache -**Description:** Provides a persistent, file-based JSON cache with TTL-based expiration. -- **Cache (Type):** Main handler for storing and retrieving cached entries. -- **Entry (Type):** Internal wrapper for data, including cached and expiry timestamps. -- **Get / Set (Func):** Thread-safe operations for managing cached data. -- **Age (Func):** Calculates how long an item has been stored. -**Dependencies:** `pkg/io` -**Complexity:** Simple diff --git a/docs/pkg/PACKAGE_STANDARDS.md b/docs/pkg/PACKAGE_STANDARDS.md index 5dcb450..5b37a4e 100644 --- a/docs/pkg/PACKAGE_STANDARDS.md +++ b/docs/pkg/PACKAGE_STANDARDS.md @@ -1,6 +1,6 @@ # Core Package Standards -This document defines the standards for creating packages in the Core framework. The `pkg/i18n` package is the reference implementation; all new packages should follow its patterns. +This document defines the standards for creating packages in the Core framework. The `pkg/log` service is the reference implementation within this repo; standalone packages (go-session, go-store, etc.) follow the same patterns. ## Package Structure @@ -40,12 +40,12 @@ package mypackage import ( "sync" - "forge.lthn.ai/core/cli/pkg/framework" + "forge.lthn.ai/core/go/pkg/core" ) // Service provides mypackage functionality with Core integration. type Service struct { - *framework.ServiceRuntime[Options] + *core.ServiceRuntime[Options] // Internal state (protected by mutex) data map[string]any @@ -67,18 +67,18 @@ Create a factory function for Core registration: ```go // NewService creates a service factory for Core registration. // -// core, _ := framework.New( -// framework.WithName("mypackage", mypackage.NewService(mypackage.Options{})), +// core, _ := core.New( +// core.WithName("mypackage", mypackage.NewService(mypackage.Options{})), // ) -func NewService(opts Options) func(*framework.Core) (any, error) { - return func(c *framework.Core) (any, error) { +func NewService(opts Options) func(*core.Core) (any, error) { + return func(c *core.Core) (any, error) { // Apply defaults if opts.BufferSize == 0 { opts.BufferSize = DefaultBufferSize } svc := &Service{ - ServiceRuntime: framework.NewServiceRuntime(c, opts), + ServiceRuntime: core.NewServiceRuntime(c, opts), data: make(map[string]any), } return svc, nil @@ -88,10 +88,10 @@ func NewService(opts Options) func(*framework.Core) (any, error) { ### Lifecycle Hooks -Implement `framework.Startable` and/or `framework.Stoppable`: +Implement `core.Startable` and/or `core.Stoppable`: ```go -// OnStartup implements framework.Startable. +// OnStartup implements core.Startable. func (s *Service) OnStartup(ctx context.Context) error { // Register query/task handlers s.Core().RegisterQuery(s.handleQuery) @@ -99,7 +99,7 @@ func (s *Service) OnStartup(ctx context.Context) error { return nil } -// OnShutdown implements framework.Stoppable. +// OnShutdown implements core.Stoppable. func (s *Service) OnShutdown(ctx context.Context) error { // Cleanup resources return nil @@ -110,7 +110,7 @@ func (s *Service) OnShutdown(ctx context.Context) error { ## Global Default Pattern -Following `pkg/i18n`, provide a global default service with atomic access: +Provide a global default service with atomic access: ```go // pkg/mypackage/mypackage.go @@ -120,7 +120,7 @@ import ( "sync" "sync/atomic" - "forge.lthn.ai/core/cli/pkg/framework" + "forge.lthn.ai/core/go/pkg/core" ) // Global default service @@ -146,7 +146,7 @@ func SetDefault(s *Service) { } // Init initialises the default service with a Core instance. -func Init(c *framework.Core) error { +func Init(c *core.Core) error { defaultOnce.Do(func() { factory := NewService(Options{}) svc, err := factory(c) @@ -267,7 +267,7 @@ func (s *Service) CreateItem(name string) (*Item, error) { Consumers register handlers: ```go -core.RegisterAction(func(c *framework.Core, msg framework.Message) error { +core.RegisterAction(func(c *core.Core, msg core.Message) error { switch m := msg.(type) { case mypackage.ActionItemCreated: log.Printf("Item created: %s", m.Name) @@ -282,7 +282,7 @@ core.RegisterAction(func(c *framework.Core, msg framework.Message) error { ## Hooks Pattern -For user-customisable behaviour, use atomic handlers (see `pkg/i18n/hooks.go`): +For user-customisable behaviour, use atomic handlers: ```go // pkg/mypackage/hooks.go @@ -435,15 +435,15 @@ buffer.go → buffer_test.go Create helpers for common setup: ```go -func newTestService(t *testing.T) (*Service, *framework.Core) { +func newTestService(t *testing.T) (*Service, *core.Core) { t.Helper() - core, err := framework.New( - framework.WithName("mypackage", NewService(Options{})), + core, err := core.New( + core.WithName("mypackage", NewService(Options{})), ) require.NoError(t, err) - svc, err := framework.ServiceFor[*Service](core, "mypackage") + svc, err := core.ServiceFor[*Service](core, "mypackage") require.NoError(t, err) return svc, core @@ -476,14 +476,14 @@ Verify ACTION broadcasts: ```go func TestService_BroadcastsActions(t *testing.T) { - core, _ := framework.New( - framework.WithName("mypackage", NewService(Options{})), + core, _ := core.New( + core.WithName("mypackage", NewService(Options{})), ) var received []ActionItemCreated var mu sync.Mutex - core.RegisterAction(func(c *framework.Core, msg framework.Message) error { + core.RegisterAction(func(c *core.Core, msg core.Message) error { if m, ok := msg.(ActionItemCreated); ok { mu.Lock() received = append(received, m) @@ -492,7 +492,7 @@ func TestService_BroadcastsActions(t *testing.T) { return nil }) - svc, _ := framework.ServiceFor[*Service](core, "mypackage") + svc, _ := core.ServiceFor[*Service](core, "mypackage") svc.CreateItem("test") mu.Lock() @@ -520,8 +520,8 @@ Every package needs a doc comment in the main file: // // # Core Integration // -// core, _ := framework.New( -// framework.WithName("mypackage", mypackage.NewService(mypackage.Options{})), +// core, _ := core.New( +// core.WithName("mypackage", mypackage.NewService(mypackage.Options{})), // ) package mypackage ``` @@ -561,9 +561,9 @@ When creating a new package, ensure: ## Reference Implementations -- **`pkg/i18n`** - Full reference with handlers, modes, hooks, grammar -- **`pkg/process`** - Simpler example with ACTION events and runner orchestration -- **`pkg/cli`** - Service integration with runtime lifecycle +- **`pkg/log`** (this repo) — Service struct with Core integration, query/task handlers +- **`core/go-store`** — SQLite KV store with Watch/OnChange, full service pattern +- **`core/go-session`** — Transcript parser with analytics, factory pattern --- @@ -596,7 +596,7 @@ The framework automatically broadcasts lifecycle actions: For very long operations, the service handler should broadcast progress: ```go -func (s *Service) handleTask(c *framework.Core, t framework.Task) (any, bool, error) { +func (s *Service) handleTask(c *core.Core, t core.Task) (any, bool, error) { switch m := t.(type) { case MyLongTask: // Optional: If you need to report progress, you might need to pass diff --git a/docs/pkg/i18n/EXTENDING.md b/docs/pkg/i18n/EXTENDING.md deleted file mode 100644 index b979b0a..0000000 --- a/docs/pkg/i18n/EXTENDING.md +++ /dev/null @@ -1,399 +0,0 @@ -# Extending the i18n Package - -This guide covers how to extend the i18n package with custom loaders, handlers, and integrations. - -## Custom Loaders - -The `Loader` interface allows loading translations from any source: - -```go -type Loader interface { - Load(lang string) (map[string]Message, *GrammarData, error) - Languages() []string -} -``` - -### Database Loader Example - -```go -type PostgresLoader struct { - db *sql.DB -} - -func (l *PostgresLoader) Languages() []string { - rows, err := l.db.Query("SELECT DISTINCT lang FROM translations") - if err != nil { - return nil - } - defer rows.Close() - - var langs []string - for rows.Next() { - var lang string - rows.Scan(&lang) - langs = append(langs, lang) - } - return langs -} - -func (l *PostgresLoader) Load(lang string) (map[string]i18n.Message, *i18n.GrammarData, error) { - rows, err := l.db.Query( - "SELECT key, text, plural_one, plural_other FROM translations WHERE lang = $1", - lang, - ) - if err != nil { - return nil, nil, err - } - defer rows.Close() - - messages := make(map[string]i18n.Message) - for rows.Next() { - var key, text string - var one, other sql.NullString - rows.Scan(&key, &text, &one, &other) - - if one.Valid || other.Valid { - messages[key] = i18n.Message{One: one.String, Other: other.String} - } else { - messages[key] = i18n.Message{Text: text} - } - } - - return messages, nil, nil -} - -// Usage -svc, err := i18n.NewWithLoader(&PostgresLoader{db: db}) -``` - -### Remote API Loader Example - -```go -type APILoader struct { - baseURL string - client *http.Client -} - -func (l *APILoader) Languages() []string { - resp, _ := l.client.Get(l.baseURL + "/languages") - defer resp.Body.Close() - - var langs []string - json.NewDecoder(resp.Body).Decode(&langs) - return langs -} - -func (l *APILoader) Load(lang string) (map[string]i18n.Message, *i18n.GrammarData, error) { - resp, err := l.client.Get(l.baseURL + "/translations/" + lang) - if err != nil { - return nil, nil, err - } - defer resp.Body.Close() - - var data struct { - Messages map[string]i18n.Message `json:"messages"` - Grammar *i18n.GrammarData `json:"grammar"` - } - json.NewDecoder(resp.Body).Decode(&data) - - return data.Messages, data.Grammar, nil -} -``` - -### Multi-Source Loader - -Combine multiple loaders with fallback: - -```go -type FallbackLoader struct { - primary i18n.Loader - secondary i18n.Loader -} - -func (l *FallbackLoader) Languages() []string { - // Merge languages from both sources - langs := make(map[string]bool) - for _, lang := range l.primary.Languages() { - langs[lang] = true - } - for _, lang := range l.secondary.Languages() { - langs[lang] = true - } - - result := make([]string, 0, len(langs)) - for lang := range langs { - result = append(result, lang) - } - return result -} - -func (l *FallbackLoader) Load(lang string) (map[string]i18n.Message, *i18n.GrammarData, error) { - msgs, grammar, err := l.primary.Load(lang) - if err != nil { - return l.secondary.Load(lang) - } - - // Merge with secondary for missing keys - secondary, secGrammar, _ := l.secondary.Load(lang) - for k, v := range secondary { - if _, exists := msgs[k]; !exists { - msgs[k] = v - } - } - - if grammar == nil { - grammar = secGrammar - } - - return msgs, grammar, nil -} -``` - -## Custom Handlers - -Handlers process keys before standard lookup. Use for dynamic patterns. - -### Handler Interface - -```go -type KeyHandler interface { - Match(key string) bool - Handle(key string, args []any, next func() string) string -} -``` - -### Emoji Handler Example - -```go -type EmojiHandler struct{} - -func (h EmojiHandler) Match(key string) bool { - return strings.HasPrefix(key, "emoji.") -} - -func (h EmojiHandler) Handle(key string, args []any, next func() string) string { - name := strings.TrimPrefix(key, "emoji.") - emojis := map[string]string{ - "success": "✅", - "error": "❌", - "warning": "⚠️", - "info": "ℹ️", - } - if emoji, ok := emojis[name]; ok { - return emoji - } - return next() // Delegate to next handler -} - -// Usage -i18n.AddHandler(EmojiHandler{}) -i18n.T("emoji.success") // "✅" -``` - -### Conditional Handler Example - -```go -type FeatureFlagHandler struct { - flags map[string]bool -} - -func (h FeatureFlagHandler) Match(key string) bool { - return strings.HasPrefix(key, "feature.") -} - -func (h FeatureFlagHandler) Handle(key string, args []any, next func() string) string { - feature := strings.TrimPrefix(key, "feature.") - parts := strings.SplitN(feature, ".", 2) - - if len(parts) < 2 { - return next() - } - - flag, subkey := parts[0], parts[1] - if h.flags[flag] { - // Feature enabled - translate the subkey - return i18n.T(subkey, args...) - } - - // Feature disabled - return empty or fallback - return "" -} -``` - -### Handler Chain Priority - -```go -// Prepend for highest priority (runs first) -svc.PrependHandler(CriticalHandler{}) - -// Append for lower priority (runs after defaults) -svc.AddHandler(FallbackHandler{}) - -// Clear all handlers -svc.ClearHandlers() - -// Add back defaults -svc.AddHandler(i18n.DefaultHandlers()...) -``` - -## Integrating with Frameworks - -### Cobra CLI - -```go -func init() { - // Initialise i18n before command setup - if err := i18n.Init(); err != nil { - log.Fatal(err) - } -} - -var rootCmd = &cobra.Command{ - Use: "myapp", - Short: i18n.T("cmd.root.short"), - Long: i18n.T("cmd.root.long"), -} - -var buildCmd = &cobra.Command{ - Use: "build", - Short: i18n.T("cmd.build.short"), - RunE: func(cmd *cobra.Command, args []string) error { - fmt.Println(i18n.T("i18n.progress.build")) - // ... - fmt.Println(i18n.T("i18n.done.build", "project")) - return nil - }, -} -``` - -### Error Messages - -```go -type LocalisedError struct { - Key string - Args map[string]any -} - -func (e LocalisedError) Error() string { - return i18n.T(e.Key, e.Args) -} - -// Usage -return LocalisedError{ - Key: "error.file_not_found", - Args: map[string]any{"Name": filename}, -} -``` - -### Structured Logging - -```go -func LogInfo(key string, args ...any) { - msg := i18n.T(key, args...) - slog.Info(msg, "i18n_key", key) -} - -func LogError(key string, err error, args ...any) { - msg := i18n.T(key, args...) - slog.Error(msg, "i18n_key", key, "error", err) -} -``` - -## Testing - -### Mock Loader for Tests - -```go -type MockLoader struct { - messages map[string]map[string]i18n.Message -} - -func (l *MockLoader) Languages() []string { - langs := make([]string, 0, len(l.messages)) - for lang := range l.messages { - langs = append(langs, lang) - } - return langs -} - -func (l *MockLoader) Load(lang string) (map[string]i18n.Message, *i18n.GrammarData, error) { - if msgs, ok := l.messages[lang]; ok { - return msgs, nil, nil - } - return nil, nil, fmt.Errorf("language not found: %s", lang) -} - -// Usage in tests -func TestMyFeature(t *testing.T) { - loader := &MockLoader{ - messages: map[string]map[string]i18n.Message{ - "en-GB": { - "test.greeting": {Text: "Hello"}, - "test.farewell": {Text: "Goodbye"}, - }, - }, - } - - svc, _ := i18n.NewWithLoader(loader) - i18n.SetDefault(svc) - - // Test your code - assert.Equal(t, "Hello", i18n.T("test.greeting")) -} -``` - -### Testing Missing Keys - -```go -func TestMissingKeys(t *testing.T) { - svc, _ := i18n.New(i18n.WithMode(i18n.ModeCollect)) - i18n.SetDefault(svc) - - var missing []string - i18n.OnMissingKey(func(m i18n.MissingKey) { - missing = append(missing, m.Key) - }) - - // Run your code that uses translations - runMyFeature() - - // Check for missing keys - assert.Empty(t, missing, "Found missing translation keys: %v", missing) -} -``` - -## Hot Reloading - -Implement a loader that watches for file changes: - -```go -type HotReloadLoader struct { - base *i18n.FSLoader - service *i18n.Service - watcher *fsnotify.Watcher -} - -func (l *HotReloadLoader) Watch() { - for { - select { - case event := <-l.watcher.Events: - if event.Op&fsnotify.Write == fsnotify.Write { - // Reload translations - l.service.LoadFS(os.DirFS("."), "locales") - } - } - } -} -``` - -## Performance Considerations - -1. **Cache translations**: The service caches all loaded messages -2. **Template caching**: Parsed templates are cached in `sync.Map` -3. **Handler chain**: Keep chain short (6 default handlers is fine) -4. **Grammar cache**: Grammar lookups are cached per-language - -For high-throughput applications: -- Pre-warm the cache by calling common translations at startup -- Consider using `Raw()` to bypass handler chain when not needed -- Profile with `go test -bench` if performance is critical diff --git a/docs/pkg/i18n/GRAMMAR.md b/docs/pkg/i18n/GRAMMAR.md deleted file mode 100644 index 2736f96..0000000 --- a/docs/pkg/i18n/GRAMMAR.md +++ /dev/null @@ -1,228 +0,0 @@ -# Grammar Engine - -The i18n grammar engine automatically handles verb conjugation, noun pluralisation, and article selection. It uses a combination of locale-defined rules and built-in English defaults. - -## Verb Conjugation - -### Past Tense - -```go -i18n.PastTense("delete") // "deleted" -i18n.PastTense("create") // "created" -i18n.PastTense("run") // "ran" (irregular) -i18n.PastTense("build") // "built" (irregular) -``` - -**Rules applied (in order):** - -1. Check locale JSON `gram.verb.{verb}.past` -2. Check built-in irregular verbs map -3. Apply regular conjugation rules: - - Ends in 'e' → add 'd' (delete → deleted) - - Ends in consonant + 'y' → change to 'ied' (try → tried) - - Short verb ending in CVC → double consonant (stop → stopped) - - Otherwise → add 'ed' (walk → walked) - -### Gerund (-ing form) - -```go -i18n.Gerund("build") // "building" -i18n.Gerund("run") // "running" -i18n.Gerund("make") // "making" -i18n.Gerund("die") // "dying" -``` - -**Rules applied:** - -1. Check locale JSON `gram.verb.{verb}.gerund` -2. Check built-in irregular verbs map -3. Apply regular rules: - - Ends in 'ie' → change to 'ying' (die → dying) - - Ends in 'e' (not 'ee') → drop 'e', add 'ing' (make → making) - - Short verb ending in CVC → double consonant (run → running) - - Otherwise → add 'ing' (build → building) - -## Noun Pluralisation - -```go -i18n.Pluralize("file", 1) // "file" -i18n.Pluralize("file", 5) // "files" -i18n.Pluralize("child", 2) // "children" (irregular) -i18n.Pluralize("analysis", 3) // "analyses" (Latin) -``` - -**Rules applied (in order):** - -1. Check locale JSON `gram.noun.{noun}.other` -2. Check built-in irregular nouns map -3. Apply regular rules: - - Ends in 's', 'x', 'z', 'ch', 'sh' → add 'es' - - Ends in consonant + 'y' → change to 'ies' - - Ends in 'f' or 'fe' → change to 'ves' (leaf → leaves) - - Otherwise → add 's' - -### Built-in Irregular Nouns - -| Singular | Plural | -|----------|--------| -| child | children | -| person | people | -| man | men | -| woman | women | -| foot | feet | -| tooth | teeth | -| mouse | mice | -| datum | data | -| index | indices | -| crisis | crises | -| fish | fish | -| sheep | sheep | - -## Articles - -```go -i18n.Article("apple") // "an apple" -i18n.Article("banana") // "a banana" -i18n.Article("hour") // "an hour" (silent h) -i18n.Article("user") // "a user" (y sound) -i18n.Article("umbrella") // "an umbrella" -``` - -**Rules:** - -1. Vowel sound words get "an" (a, e, i, o, u start) -2. Consonant sound words get "a" -3. Exception lists handle: - - Silent 'h' words: hour, honest, honour, heir, herb - - 'Y' sound words: user, union, unique, unit, universe - -## Composed Messages - -### Labels - -```go -i18n.Label("status") // "Status:" -i18n.Label("version") // "Version:" -``` - -Uses `gram.punct.label` suffix (default `:`) from locale. - -### Progress Messages - -```go -i18n.Progress("build") // "Building..." -i18n.ProgressSubject("check", "config") // "Checking config..." -``` - -Uses `gram.punct.progress` suffix (default `...`) from locale. - -### Action Results - -```go -i18n.ActionResult("delete", "file") // "File deleted" -i18n.ActionResult("create", "project") // "Project created" -``` - -Pattern: `{Title(subject)} {past(verb)}` - -### Action Failures - -```go -i18n.ActionFailed("delete", "file") // "Failed to delete file" -i18n.ActionFailed("save", "config") // "Failed to save config" -``` - -Pattern: `Failed to {verb} {subject}` - -## Locale Configuration - -Define grammar in your locale JSON: - -```json -{ - "gram": { - "verb": { - "deploy": { - "past": "deployed", - "gerund": "deploying" - }, - "sync": { - "past": "synced", - "gerund": "syncing" - } - }, - "noun": { - "repository": { - "one": "repository", - "other": "repositories" - }, - "schema": { - "one": "schema", - "other": "schemata" - } - }, - "article": { - "indefinite": { - "default": "a", - "vowel": "an" - }, - "definite": "the" - }, - "punct": { - "label": ":", - "progress": "..." - }, - "word": { - "status": "status", - "version": "version" - } - } -} -``` - -## Template Functions - -Use grammar functions in templates: - -```go -template.New("").Funcs(i18n.TemplateFuncs()) -``` - -| Function | Example | Result | -|----------|---------|--------| -| `past` | `{{past "delete"}}` | "deleted" | -| `gerund` | `{{gerund "build"}}` | "building" | -| `plural` | `{{plural "file" 5}}` | "files" | -| `article` | `{{article "apple"}}` | "an apple" | -| `title` | `{{title "hello world"}}` | "Hello World" | -| `lower` | `{{lower "HELLO"}}` | "hello" | -| `upper` | `{{upper "hello"}}` | "HELLO" | -| `quote` | `{{quote "text"}}` | `"text"` | - -## Language-Specific Grammar - -The grammar engine loads language-specific data when available: - -```go -// Get grammar data for a language -data := i18n.GetGrammarData("de-DE") -if data != nil { - // Access verb forms, noun forms, etc. -} - -// Set grammar data programmatically -i18n.SetGrammarData("de-DE", &i18n.GrammarData{ - Verbs: map[string]i18n.VerbForms{ - "machen": {Past: "gemacht", Gerund: "machend"}, - }, -}) -``` - -## Performance - -Grammar results are computed on-demand but templates are cached: - -- First call: Parse template + apply grammar -- Subsequent calls: Reuse cached template - -The template cache uses `sync.Map` for thread-safe concurrent access. diff --git a/docs/pkg/i18n/README.md b/docs/pkg/i18n/README.md deleted file mode 100644 index 67775a9..0000000 --- a/docs/pkg/i18n/README.md +++ /dev/null @@ -1,420 +0,0 @@ -# i18n Package - -The `pkg/i18n` package provides internationalisation and localisation for Go CLI applications. It features a grammar engine for automatic verb conjugation and noun pluralisation, CLDR plural support, and an extensible handler chain for dynamic key patterns. - -## Quick Start - -```go -import "forge.lthn.ai/core/cli/pkg/i18n" - -func main() { - // Initialise with embedded locales - svc, err := i18n.New() - if err != nil { - log.Fatal(err) - } - i18n.SetDefault(svc) - - // Translate messages - fmt.Println(i18n.T("cli.success")) // "Operation completed" - fmt.Println(i18n.T("i18n.count.file", 5)) // "5 files" - fmt.Println(i18n.T("i18n.progress.build")) // "Building..." - fmt.Println(i18n.T("i18n.done.delete", "config.yaml")) // "Config.yaml deleted" -} -``` - -## Table of Contents - -- [Basic Translation](#basic-translation) -- [Template Variables](#template-variables) -- [Pluralisation](#pluralisation) -- [Magic Namespaces](#magic-namespaces) -- [Subjects](#subjects) -- [Grammar Engine](#grammar-engine) -- [Formality](#formality) -- [Modes](#modes) -- [Custom Loaders](#custom-loaders) -- [Custom Handlers](#custom-handlers) -- [Locale File Format](#locale-file-format) - -## Basic Translation - -The `T()` function translates message keys: - -```go -// Simple translation -msg := i18n.T("cli.success") - -// With template variables -msg := i18n.T("error.not_found", map[string]any{ - "Name": "config.yaml", -}) -``` - -Use `Raw()` to bypass magic namespace handling: - -```go -// T() handles i18n.* magic -i18n.T("i18n.label.status") // "Status:" - -// Raw() does direct lookup only -i18n.Raw("i18n.label.status") // Returns key as-is (not in JSON) -``` - -## Template Variables - -Translation strings support Go templates: - -```json -{ - "greeting": "Hello, {{.Name}}!", - "summary": "Found {{.Count}} {{if eq .Count 1}}item{{else}}items{{end}}" -} -``` - -```go -i18n.T("greeting", map[string]any{"Name": "World"}) // "Hello, World!" -i18n.T("summary", map[string]any{"Count": 3}) // "Found 3 items" -``` - -### Available Template Functions - -| Function | Description | Example | -|----------|-------------|---------| -| `title` | Title case | `{{title .Name}}` | -| `lower` | Lowercase | `{{lower .Name}}` | -| `upper` | Uppercase | `{{upper .Name}}` | -| `past` | Past tense | `{{past "delete"}}` → "deleted" | -| `gerund` | -ing form | `{{gerund "build"}}` → "building" | -| `plural` | Pluralise | `{{plural "file" .Count}}` | -| `article` | Add article | `{{article "apple"}}` → "an apple" | -| `quote` | Add quotes | `{{quote .Name}}` → `"name"` | - -## Pluralisation - -The package supports full CLDR plural categories: - -```json -{ - "item_count": { - "zero": "No items", - "one": "{{.Count}} item", - "two": "{{.Count}} items", - "few": "{{.Count}} items", - "many": "{{.Count}} items", - "other": "{{.Count}} items" - } -} -``` - -```go -i18n.T("item_count", map[string]any{"Count": 0}) // "No items" (if zero defined) -i18n.T("item_count", map[string]any{"Count": 1}) // "1 item" -i18n.T("item_count", map[string]any{"Count": 5}) // "5 items" -``` - -For simple cases, use `i18n.count.*`: - -```go -i18n.T("i18n.count.file", 1) // "1 file" -i18n.T("i18n.count.file", 5) // "5 files" -``` - -## Magic Namespaces - -The `i18n.*` namespace provides automatic message composition: - -### Labels (`i18n.label.*`) - -```go -i18n.T("i18n.label.status") // "Status:" -i18n.T("i18n.label.version") // "Version:" -``` - -### Progress (`i18n.progress.*`) - -```go -i18n.T("i18n.progress.build") // "Building..." -i18n.T("i18n.progress.check", "config") // "Checking config..." -``` - -### Counts (`i18n.count.*`) - -```go -i18n.T("i18n.count.file", 1) // "1 file" -i18n.T("i18n.count.file", 5) // "5 files" -i18n.T("i18n.count.repo", 10) // "10 repos" -``` - -### Done (`i18n.done.*`) - -```go -i18n.T("i18n.done.delete", "file") // "File deleted" -i18n.T("i18n.done.create", "project") // "Project created" -``` - -### Fail (`i18n.fail.*`) - -```go -i18n.T("i18n.fail.delete", "file") // "Failed to delete file" -i18n.T("i18n.fail.save", "config") // "Failed to save config" -``` - -### Numeric (`i18n.numeric.*`) - -```go -i18n.N("number", 1234567) // "1,234,567" -i18n.N("percent", 0.85) // "85%" -i18n.N("bytes", 1536000) // "1.46 MB" -i18n.N("ordinal", 1) // "1st" -``` - -## Subjects - -Subjects provide typed context for translations: - -```go -// Create a subject -subj := i18n.S("file", "config.yaml") - -// Chain methods for additional context -subj := i18n.S("file", files). - Count(len(files)). - In("workspace"). - Formal() - -// Use in translations -i18n.T("i18n.done.delete", subj.String()) -``` - -### Subject Methods - -| Method | Description | -|--------|-------------| -| `Count(n)` | Set count for pluralisation | -| `Gender(g)` | Set grammatical gender | -| `In(loc)` | Set location context | -| `Formal()` | Set formal address | -| `Informal()` | Set informal address | - -## Grammar Engine - -The grammar engine handles verb conjugation and noun forms: - -```go -// Verb conjugation -i18n.PastTense("delete") // "deleted" -i18n.PastTense("run") // "ran" (irregular) -i18n.Gerund("build") // "building" -i18n.Gerund("run") // "running" - -// Noun pluralisation -i18n.Pluralize("file", 1) // "file" -i18n.Pluralize("file", 5) // "files" -i18n.Pluralize("child", 2) // "children" (irregular) - -// Articles -i18n.Article("apple") // "an apple" -i18n.Article("banana") // "a banana" - -// Composed messages -i18n.Label("status") // "Status:" -i18n.Progress("build") // "Building..." -i18n.ProgressSubject("check", "cfg") // "Checking cfg..." -i18n.ActionResult("delete", "file") // "File deleted" -i18n.ActionFailed("save", "config") // "Failed to save config" -``` - -### Customising Grammar - -Add irregular forms in your locale JSON: - -```json -{ - "gram": { - "verb": { - "deploy": { "past": "deployed", "gerund": "deploying" } - }, - "noun": { - "repository": { "one": "repository", "other": "repositories" } - }, - "punct": { - "label": ":", - "progress": "..." - } - } -} -``` - -## Formality - -For languages with formal/informal address (German Sie/du, French vous/tu): - -```go -// Set service-wide formality -svc.SetFormality(i18n.FormalityFormal) - -// Per-translation formality via Subject -i18n.T("greeting", i18n.S("user", name).Formal()) -i18n.T("greeting", i18n.S("user", name).Informal()) - -// Per-translation via TranslationContext -i18n.T("greeting", i18n.C("customer support").Formal()) -``` - -Define formality variants in JSON: - -```json -{ - "greeting": "Hello", - "greeting._formal": "Good morning, sir", - "greeting._informal": "Hey there" -} -``` - -## Modes - -Three modes control missing key behaviour: - -```go -// Normal (default): Returns key as-is -i18n.SetMode(i18n.ModeNormal) -i18n.T("missing.key") // "missing.key" - -// Strict: Panics on missing keys (dev/CI) -i18n.SetMode(i18n.ModeStrict) -i18n.T("missing.key") // panic! - -// Collect: Dispatches to handler (QA testing) -i18n.SetMode(i18n.ModeCollect) -i18n.OnMissingKey(func(m i18n.MissingKey) { - log.Printf("MISSING: %s at %s:%d", m.Key, m.CallerFile, m.CallerLine) -}) -``` - -## Custom Loaders - -Implement the `Loader` interface for custom storage: - -```go -type Loader interface { - Load(lang string) (map[string]Message, *GrammarData, error) - Languages() []string -} -``` - -Example database loader: - -```go -type DBLoader struct { - db *sql.DB -} - -func (l *DBLoader) Languages() []string { - // Query available languages from database -} - -func (l *DBLoader) Load(lang string) (map[string]i18n.Message, *i18n.GrammarData, error) { - // Load translations from database -} - -// Use custom loader -svc, err := i18n.NewWithLoader(&DBLoader{db: db}) -``` - -## Custom Handlers - -Add custom key handlers for dynamic patterns: - -```go -type MyHandler struct{} - -func (h MyHandler) Match(key string) bool { - return strings.HasPrefix(key, "my.prefix.") -} - -func (h MyHandler) Handle(key string, args []any, next func() string) string { - // Handle the key or call next() to delegate - return "custom result" -} - -// Add to handler chain -svc.AddHandler(MyHandler{}) // Append (lower priority) -svc.PrependHandler(MyHandler{}) // Prepend (higher priority) -``` - -## Locale File Format - -Locale files use nested JSON with dot-notation access: - -```json -{ - "cli": { - "success": "Operation completed", - "error": { - "not_found": "{{.Name}} not found" - } - }, - "cmd": { - "build": { - "short": "Build the project", - "long": "Build compiles source files into an executable" - } - }, - "gram": { - "verb": { - "build": { "past": "built", "gerund": "building" } - }, - "noun": { - "file": { "one": "file", "other": "files" } - }, - "punct": { - "label": ":", - "progress": "..." - } - } -} -``` - -Access keys with dot notation: - -```go -i18n.T("cli.success") // "Operation completed" -i18n.T("cli.error.not_found") // "{{.Name}} not found" -i18n.T("cmd.build.short") // "Build the project" -``` - -## Configuration Options - -Use functional options when creating a service: - -```go -svc, err := i18n.New( - i18n.WithFallback("de-DE"), // Fallback language - i18n.WithFormality(i18n.FormalityFormal), // Default formality - i18n.WithMode(i18n.ModeStrict), // Missing key mode - i18n.WithDebug(true), // Show [key] prefix -) -``` - -## Thread Safety - -The package is fully thread-safe: - -- `Service` uses `sync.RWMutex` for state -- Global `Default()` uses `atomic.Pointer` -- `OnMissingKey` uses `atomic.Value` -- `FSLoader.Languages()` uses `sync.Once` - -Safe for concurrent use from multiple goroutines. - -## Debug Mode - -Enable debug mode to see translation keys: - -```go -i18n.SetDebug(true) -i18n.T("cli.success") // "[cli.success] Operation completed" -``` - -Useful for identifying which keys are used where. diff --git a/docs/pkg/log.md b/docs/pkg/log.md index c6cff6f..56bce7f 100644 --- a/docs/pkg/log.md +++ b/docs/pkg/log.md @@ -36,8 +36,8 @@ logger.Info("application started") When using the Core framework, logging is usually configured during application initialization: ```go -app := core.New( - framework.WithName("my-app", log.NewService(log.Options{ +app, _ := core.New( + core.WithName("log", log.NewService(log.Options{ Level: log.LevelDebug, Rotation: &log.RotationOptions{ Filename: "/var/log/my-app.log", diff --git a/docs/plans/2026-02-20-authentik-traefik-plan.md b/docs/plans/2026-02-20-authentik-traefik-plan.md deleted file mode 100644 index 091a082..0000000 --- a/docs/plans/2026-02-20-authentik-traefik-plan.md +++ /dev/null @@ -1,1163 +0,0 @@ -# Authentik + Traefik Integration Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Deploy Authentik as the identity provider, wire it into Traefik's forward auth, and add OIDC/header middleware to go-api so protected services get authenticated user context. - -**Architecture:** Authentik runs alongside existing services on de2 (production). Traefik's file provider loads a `forwardAuth` middleware definition pointing at Authentik's outpost. Services opt-in via Docker label `middlewares: authentik@file`. go-api gains a `WithAuthentik()` option that extracts user identity from Authentik headers (forward auth mode) or validates JWTs directly (API client mode). - -**Tech Stack:** Authentik 2025.2, Traefik v3.6, Go 1.25, coreos/go-oidc/v3, golang.org/x/oauth2 - -**Design doc:** `docs/plans/2026-02-20-go-api-design.md` (Authentik section) - -**Key references:** -- Traefik role: `/Users/snider/Code/DevOps/roles/traefik/` -- Authentik role: `/Users/snider/Code/DevOps/roles/authentik/` -- Forward auth template: `/Users/snider/Code/DevOps/roles/traefik/templates/dynamic-authentik.yml.j2` -- go-api repo: `/Users/snider/Code/go-api/` - ---- - -## Current State - -The Ansible infrastructure is **already built but not activated**: - -| Component | Status | Location | -|-----------|--------|----------| -| Traefik v3.6 role | Deployed on de2 | `roles/traefik/` | -| Authentik 2025.2 role | Written, **never deployed** | `roles/authentik/` | -| Forward auth middleware template | Written, conditional on `traefik_authentik_enabled` | `dynamic-authentik.yml.j2` | -| Outpost routing in Authentik compose | Pre-configured | `roles/authentik/templates/docker-compose.yml.j2` | -| 5 services with `authentik@file` | Labels present, middleware not yet available | `prod_rebuild.yml` | -| go-api Authentik middleware | **Not started** | — | - -**Headers Authentik will pass to go-api (via Traefik):** -``` -X-authentik-username, X-authentik-groups, X-authentik-entitlements, -X-authentik-email, X-authentik-name, X-authentik-uid, X-authentik-jwt, -X-authentik-meta-jwks, X-authentik-meta-outpost, X-authentik-meta-provider, -X-authentik-meta-app, X-authentik-meta-version -``` - ---- - -### Task 1: Enable Authentik in Production Inventory - -This task sets the Ansible variables to enable Authentik deployment on the production host. - -**Files:** -- Modify: `/Users/snider/Code/DevOps/inventory/host_vars/de2.yml` (or equivalent group_vars) - -**Step 1: Find the correct inventory file for de2** - -Run: -```bash -find /Users/snider/Code/DevOps/inventory -name "*.yml" -o -name "*.yaml" | head -20 -ls /Users/snider/Code/DevOps/inventory/ -``` - -Identify where de2's host vars live. - -**Step 2: Add Authentik variables** - -Add these variables for the de2 host: - -```yaml -# Authentik -traefik_authentik_enabled: true -traefik_authentik_url: "https://auth.host.uk.com" - -authentik_host: "auth.host.uk.com" -authentik_bootstrap_password: "" -authentik_bootstrap_token: "" -authentik_bootstrap_email: "admin@host.uk.com" -``` - -Note: `authentik_secret_key` auto-generates and persists on first run. `authentik_pg_password` auto-generates via lookup. The Authentik role handles both. - -**Step 3: Verify prerequisites exist on de2** - -Authentik requires PostgreSQL + Dragonfly (Redis). Check they're in the prod playbook: -```bash -grep -n "postgres\|dragonfly" /Users/snider/Code/DevOps/playbooks/prod_rebuild.yml | head -10 -``` - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/DevOps -git add inventory/ -git commit -m "feat(authentik): enable Authentik and Traefik forward auth on de2 - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: Add Authentik to Production Playbook - -The Authentik Ansible role exists but is not included in the prod rebuild playbook. This task adds it. - -**Files:** -- Modify: `/Users/snider/Code/DevOps/playbooks/prod_rebuild.yml` - -**Step 1: Read the playbook to find the right insertion point** - -Authentik must deploy AFTER PostgreSQL + Dragonfly (it needs them) and AFTER Traefik (it needs the proxy network), but BEFORE services that use `authentik@file`. - -```bash -grep -n "Phase\|traefik\|postgres\|dragonfly\|portainer\|glance" /Users/snider/Code/DevOps/playbooks/prod_rebuild.yml | head -20 -``` - -**Step 2: Add Authentik role include** - -Insert after the Traefik phase, before services: - -```yaml - # ── Phase N: Identity (Authentik) ── - - name: Deploy Authentik - ansible.builtin.include_role: - name: authentik - tags: [authentik] -``` - -**Step 3: Verify the playbook parses** - -```bash -cd /Users/snider/Code/DevOps -ansible-playbook playbooks/prod_rebuild.yml --syntax-check -``` - -Expected: No errors. - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/DevOps -git add playbooks/prod_rebuild.yml -git commit -m "feat(authentik): add Authentik phase to prod rebuild playbook - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: Deploy Authentik (Run Playbook) - -This is a manual step — run the Ansible playbook to deploy Authentik on de2. - -**Step 1: Dry-run the Authentik tag only** - -```bash -cd /Users/snider/Code/DevOps -ansible-playbook playbooks/prod_rebuild.yml --tags authentik --check --diff -``` - -Review the output. Expect: directories created, docker-compose deployed, containers started. - -Note: `--check` will skip shell/command tasks (like the PostgreSQL user creation). This is expected — the actual run will handle those. - -**Step 2: Deploy Authentik** - -```bash -ansible-playbook playbooks/prod_rebuild.yml --tags authentik -``` - -**Step 3: Re-deploy Traefik to pick up the forward auth middleware** - -The Traefik role conditionally deploys `dynamic-authentik.yml` based on `traefik_authentik_enabled`. Re-running the role with the new variable will create the middleware file: - -```bash -ansible-playbook playbooks/prod_rebuild.yml --tags traefik -``` - -**Step 4: Verify Authentik is accessible** - -```bash -curl -sI https://auth.host.uk.com | head -5 -``` - -Expected: HTTP 200 or 302 redirect to login page. - -**Step 5: Complete initial setup** - -Open `https://auth.host.uk.com/if/flow/initial-setup/` in a browser. Set the admin password (the bootstrap password from Task 1 is used for the API token, but the UI setup flow creates the actual admin account). - ---- - -### Task 4: Create Authentik OIDC Application for go-api - -This configures Authentik to issue tokens for go-api. Done via the Authentik admin UI or API. - -**Step 1: Create an OAuth2/OIDC Provider** - -In Authentik Admin → Providers → Create: - -| Field | Value | -|-------|-------| -| Name | `Core API` | -| Protocol | OAuth2/OIDC | -| Client type | Confidential | -| Client ID | `core-api` | -| Redirect URIs | `https://api.lthn.ai/auth/callback` (for auth code flow) | -| Signing key | Select auto-generated signing key | -| Scopes | `openid`, `email`, `profile` | -| Subject mode | Based on user's hashed ID | - -Record the **Client Secret** — needed for go-api config. - -**Step 2: Create an Application** - -In Authentik Admin → Applications → Create: - -| Field | Value | -|-------|-------| -| Name | `Core API` | -| Slug | `core-api` | -| Provider | Core API (from step 1) | -| Launch URL | `https://api.lthn.ai/` | - -**Step 3: Create a Forward Auth (Proxy) Provider for Traefik** - -In Authentik Admin → Providers → Create: - -| Field | Value | -|-------|-------| -| Name | `Traefik Forward Auth — Core API` | -| Protocol | Proxy | -| Mode | Forward auth (single application) | -| External host | `https://api.lthn.ai` | - -**Step 4: Create an Outpost (if not exists)** - -In Authentik Admin → Outposts: -- If no outpost exists: Create → Type: Proxy, Integration: Local Docker -- Add both providers to the outpost - -**Step 5: Test forward auth is working** - -```bash -# This should redirect to Authentik login -curl -sI https://api.lthn.ai/ -``` - -Once authenticated, Traefik passes the X-authentik-* headers through. - ---- - -### Task 5: go-api Authentik User Type (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/authentik.go` -- Create: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Create `authentik_test.go`: -```go -package api_test - -import ( - "testing" - - api "forge.lthn.ai/core/go-api" -) - -func TestAuthentikUser_Good(t *testing.T) { - user := &api.AuthentikUser{ - Username: "alice", - Email: "alice@example.com", - Name: "Alice Smith", - UID: "abc-123", - Groups: []string{"admins", "developers"}, - } - - if user.Username != "alice" { - t.Fatalf("expected username alice, got %s", user.Username) - } - if len(user.Groups) != 2 { - t.Fatalf("expected 2 groups, got %d", len(user.Groups)) - } -} - -func TestAuthentikUserHasGroup_Good(t *testing.T) { - user := &api.AuthentikUser{ - Groups: []string{"admins", "developers"}, - } - - if !user.HasGroup("admins") { - t.Fatal("expected user to have admins group") - } - if user.HasGroup("viewers") { - t.Fatal("expected user to not have viewers group") - } -} - -func TestAuthentikUserHasGroup_Bad_Empty(t *testing.T) { - user := &api.AuthentikUser{} - - if user.HasGroup("admins") { - t.Fatal("expected empty user to have no groups") - } -} - -func TestAuthentikConfig_Good(t *testing.T) { - cfg := api.AuthentikConfig{ - Issuer: "https://auth.host.uk.com/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, - } - - if cfg.Issuer == "" { - t.Fatal("expected non-empty issuer") - } - if !cfg.TrustedProxy { - t.Fatal("expected TrustedProxy to be true") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestAuthentik -``` - -Expected: Compilation errors — `api.AuthentikUser`, `api.AuthentikConfig` not defined. - -**Step 3: Implement authentik.go** - -Create `authentik.go`: -```go -package api - -// AuthentikConfig configures Authentik OIDC integration. -type AuthentikConfig struct { - // Issuer is the OIDC issuer URL (e.g. "https://auth.host.uk.com/application/o/core-api/"). - // Used for JWT validation via OIDC discovery. - Issuer string - - // ClientID is the OAuth2 client identifier registered in Authentik. - ClientID string - - // TrustedProxy enables reading X-authentik-* headers set by Traefik forward auth. - // Only enable this when go-api sits behind a trusted reverse proxy. - TrustedProxy bool - - // PublicPaths lists path prefixes that skip authentication entirely. - // /health and /swagger are always public regardless of this setting. - PublicPaths []string -} - -// AuthentikUser represents an authenticated user extracted from Authentik headers or JWT claims. -type AuthentikUser struct { - Username string `json:"username"` - Email string `json:"email"` - Name string `json:"name"` - UID string `json:"uid"` - Groups []string `json:"groups"` - Entitlements []string `json:"entitlements,omitempty"` - JWT string `json:"-"` -} - -// HasGroup returns true if the user belongs to the named group. -func (u *AuthentikUser) HasGroup(group string) bool { - for _, g := range u.Groups { - if g == group { - return true - } - } - return false -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestAuthentik -``` - -Expected: All 4 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go -git commit -m "feat: add AuthentikUser and AuthentikConfig types - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: go-api Header Extraction Middleware (TDD) - -This implements the forward auth path — extracting user identity from X-authentik-* headers set by Traefik. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -import ( - "encoding/json" - "net/http" - "net/http/httptest" - - "github.com/gin-gonic/gin" -) - -// authentikTestGroup returns JSON with the user from context. -type authentikTestGroup struct{} - -func (g *authentikTestGroup) Name() string { return "authtest" } -func (g *authentikTestGroup) BasePath() string { return "/v1/authtest" } -func (g *authentikTestGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/whoami", func(c *gin.Context) { - user := api.GetUser(c) - if user == nil { - c.JSON(200, api.OK[any](nil)) - return - } - c.JSON(200, api.OK(user)) - }) -} - -func TestForwardAuthHeaders_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-email", "alice@example.com") - req.Header.Set("X-authentik-name", "Alice Smith") - req.Header.Set("X-authentik-uid", "abc-123") - req.Header.Set("X-authentik-groups", "admins|developers") - req.Header.Set("X-authentik-entitlements", "core:read|core:write") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data == nil { - t.Fatal("expected non-nil user data") - } - if resp.Data.Username != "alice" { - t.Fatalf("expected username alice, got %s", resp.Data.Username) - } - if resp.Data.Email != "alice@example.com" { - t.Fatalf("expected email alice@example.com, got %s", resp.Data.Email) - } - if len(resp.Data.Groups) != 2 { - t.Fatalf("expected 2 groups, got %d", len(resp.Data.Groups)) - } - if resp.Data.Groups[0] != "admins" { - t.Fatalf("expected first group admins, got %s", resp.Data.Groups[0]) - } -} - -func TestForwardAuthHeaders_Good_NoHeaders(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - // Request without Authentik headers — should pass through (middleware is permissive) - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user when no headers present") - } -} - -func TestForwardAuthHeaders_Bad_NotTrusted(t *testing.T) { - gin.SetMode(gin.TestMode) - // TrustedProxy: false — should NOT read X-authentik-* headers - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: false, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("X-authentik-username", "alice") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user when TrustedProxy is false") - } -} - -func TestHealthBypassesAuthentik_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for /health, got %d", w.Code) - } -} - -func TestGetUser_Good_NilContext(t *testing.T) { - gin.SetMode(gin.TestMode) - // Test GetUser with no user in context (no Authentik middleware) - engine, _ := api.New() - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestForwardAuth\|TestHealthBypassesAuthentik\|TestGetUser -``` - -Expected: Compilation errors — `api.WithAuthentik`, `api.GetUser` not defined. - -**Step 3: Add GetUser helper and middleware to authentik.go** - -Append to `authentik.go`: -```go -import ( - "strings" - - "github.com/gin-gonic/gin" -) - -const authentikUserKey = "authentik_user" - -// GetUser returns the authenticated Authentik user from the Gin context, or nil -// if no user is authenticated. -func GetUser(c *gin.Context) *AuthentikUser { - val, exists := c.Get(authentikUserKey) - if !exists { - return nil - } - user, ok := val.(*AuthentikUser) - if !ok { - return nil - } - return user -} - -// authentikMiddleware extracts user identity from X-authentik-* headers -// (when TrustedProxy is true) and stores it in the Gin context. -// This middleware is PERMISSIVE — it does not reject unauthenticated requests. -// Handlers must check GetUser() and decide whether to require auth. -func authentikMiddleware(cfg AuthentikConfig) gin.HandlerFunc { - publicPaths := append([]string{"/health", "/swagger"}, cfg.PublicPaths...) - - return func(c *gin.Context) { - // Skip public paths entirely. - for _, path := range publicPaths { - if strings.HasPrefix(c.Request.URL.Path, path) { - c.Next() - return - } - } - - // Forward auth mode: read trusted headers from Traefik. - if cfg.TrustedProxy { - username := c.GetHeader("X-authentik-username") - if username != "" { - user := &AuthentikUser{ - Username: username, - Email: c.GetHeader("X-authentik-email"), - Name: c.GetHeader("X-authentik-name"), - UID: c.GetHeader("X-authentik-uid"), - JWT: c.GetHeader("X-authentik-jwt"), - } - - if groups := c.GetHeader("X-authentik-groups"); groups != "" { - user.Groups = strings.Split(groups, "|") - } - if ent := c.GetHeader("X-authentik-entitlements"); ent != "" { - user.Entitlements = strings.Split(ent, "|") - } - - c.Set(authentikUserKey, user) - } - } - - c.Next() - } -} -``` - -**Step 4: Add WithAuthentik option to options.go** - -Append to `options.go`: -```go -// WithAuthentik adds Authentik identity middleware. -// When TrustedProxy is true, reads X-authentik-* headers from Traefik forward auth. -// When Issuer is set, also validates JWT Bearer tokens via OIDC discovery. -func WithAuthentik(cfg AuthentikConfig) Option { - return func(e *Engine) { - e.middlewares = append(e.middlewares, authentikMiddleware(cfg)) - } -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS (existing 36 + new 5). - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go options.go -git commit -m "feat: add Authentik header extraction middleware and GetUser helper - -Forward auth mode reads X-authentik-* headers from Traefik. -Middleware is permissive — handlers decide whether auth is required. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 7: go-api JWT Validation Middleware (TDD) - -This implements the direct OIDC path — validating JWT Bearer tokens for API clients. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` -- Modify: `/Users/snider/Code/go-api/go.mod` (new dependency) - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -func TestJWTValidation_Bad_InvalidToken(t *testing.T) { - gin.SetMode(gin.TestMode) - // Use a fake issuer — OIDC discovery will fail, but we test the flow - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.example.com/application/o/test/", - ClientID: "test-client", - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("Authorization", "Bearer invalid-jwt-token") - handler.ServeHTTP(w, req) - - // Without a reachable OIDC endpoint, JWT validation can't succeed. - // The middleware should pass through (permissive) with no user. - if w.Code != 200 { - t.Fatalf("expected 200 (permissive), got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user for invalid JWT") - } -} - -func TestBearerAndAuthentikCoexist_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - // Both WithBearerAuth and WithAuthentik should work together. - // Bearer auth gates access, Authentik extracts user identity. - engine, _ := api.New( - api.WithBearerAuth("secret-token"), - api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true}), - ) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - // With bearer token + Authentik headers → 200 with user - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("Authorization", "Bearer secret-token") - req.Header.Set("X-authentik-username", "bob") - req.Header.Set("X-authentik-email", "bob@example.com") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data == nil { - t.Fatal("expected user data") - } - if resp.Data.Username != "bob" { - t.Fatalf("expected username bob, got %s", resp.Data.Username) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestJWTValidation\|TestBearerAndAuthentikCoexist -``` - -**Step 3: Add OIDC validation to authentik middleware** - -Update `authentikMiddleware` in `authentik.go` to handle JWT Bearer tokens when `Issuer` is configured. Add the go-oidc dependency: - -```bash -cd /Users/snider/Code/go-api -go get github.com/coreos/go-oidc/v3/oidc -go get golang.org/x/oauth2 -``` - -Add JWT validation logic to the middleware — after the header extraction block, before `c.Next()`: - -```go -// Direct OIDC mode: validate JWT from Authorization header. -if cfg.Issuer != "" && cfg.ClientID != "" { - // Only attempt JWT validation if no user was extracted from headers - // (headers take priority — they're pre-validated by Authentik). - if GetUser(c) == nil { - authHeader := c.GetHeader("Authorization") - if strings.HasPrefix(authHeader, "Bearer ") { - token := strings.TrimPrefix(authHeader, "Bearer ") - user, err := validateJWT(c.Request.Context(), cfg, token) - if err == nil && user != nil { - c.Set(authentikUserKey, user) - } - // Permissive: if validation fails, continue without user. - } - } -} -``` - -Add the validation function: -```go -import ( - "context" - "sync" - - oidc "github.com/coreos/go-oidc/v3/oidc" -) - -var ( - oidcProviderMu sync.Mutex - oidcProviders = make(map[string]*oidc.Provider) -) - -// getOIDCProvider returns a cached OIDC provider for the given issuer. -func getOIDCProvider(ctx context.Context, issuer string) (*oidc.Provider, error) { - oidcProviderMu.Lock() - defer oidcProviderMu.Unlock() - - if p, ok := oidcProviders[issuer]; ok { - return p, nil - } - - p, err := oidc.NewProvider(ctx, issuer) - if err != nil { - return nil, err - } - oidcProviders[issuer] = p - return p, nil -} - -// validateJWT verifies a JWT token against the OIDC provider and extracts the user. -func validateJWT(ctx context.Context, cfg AuthentikConfig, rawToken string) (*AuthentikUser, error) { - provider, err := getOIDCProvider(ctx, cfg.Issuer) - if err != nil { - return nil, err - } - - verifier := provider.Verifier(&oidc.Config{ClientID: cfg.ClientID}) - idToken, err := verifier.Verify(ctx, rawToken) - if err != nil { - return nil, err - } - - var claims struct { - PreferredUsername string `json:"preferred_username"` - Email string `json:"email"` - Name string `json:"name"` - Sub string `json:"sub"` - Groups []string `json:"groups"` - } - if err := idToken.Claims(&claims); err != nil { - return nil, err - } - - return &AuthentikUser{ - Username: claims.PreferredUsername, - Email: claims.Email, - Name: claims.Name, - UID: claims.Sub, - Groups: claims.Groups, - JWT: rawToken, - }, nil -} -``` - -**Step 4: Run go mod tidy** - -```bash -cd /Users/snider/Code/go-api -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go go.mod go.sum -git commit -m "feat: add OIDC JWT validation for direct API client auth - -Uses coreos/go-oidc for OIDC discovery and JWT verification. -Cached provider instances. Permissive — fails open if OIDC unreachable. -Forward auth headers take priority over JWT when both present. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: go-api RequireAuth Middleware Helper (TDD) - -The Authentik middleware is permissive. This task adds a helper for routes that REQUIRE authentication. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -// protectedGroup uses RequireAuth on its routes. -type protectedGroup struct{} - -func (g *protectedGroup) Name() string { return "protected" } -func (g *protectedGroup) BasePath() string { return "/v1/protected" } -func (g *protectedGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/data", api.RequireAuth(), func(c *gin.Context) { - user := api.GetUser(c) - c.JSON(200, api.OK(user.Username)) - }) -} - -func TestRequireAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-email", "alice@example.com") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 with auth, got %d", w.Code) - } -} - -func TestRequireAuth_Bad_NoUser(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 without auth, got %d", w.Code) - } -} - -func TestRequireAuth_Bad_NoAuthentikMiddleware(t *testing.T) { - gin.SetMode(gin.TestMode) - // No WithAuthentik — RequireAuth should still return 401 - engine, _ := api.New() - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401, got %d", w.Code) - } -} - -// groupRequireGroup uses RequireGroup. -type groupRequireGroup struct{} - -func (g *groupRequireGroup) Name() string { return "adminonly" } -func (g *groupRequireGroup) BasePath() string { return "/v1/admin" } -func (g *groupRequireGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/panel", api.RequireGroup("admins"), func(c *gin.Context) { - c.JSON(200, api.OK("admin panel")) - }) -} - -func TestRequireGroup_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&groupRequireGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/admin/panel", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-groups", "admins|developers") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for admin user, got %d", w.Code) - } -} - -func TestRequireGroup_Bad_WrongGroup(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&groupRequireGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/admin/panel", nil) - req.Header.Set("X-authentik-username", "bob") - req.Header.Set("X-authentik-groups", "developers") - handler.ServeHTTP(w, req) - - if w.Code != 403 { - t.Fatalf("expected 403 for non-admin user, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestRequireAuth\|TestRequireGroup -``` - -Expected: Compilation errors — `api.RequireAuth`, `api.RequireGroup` not defined. - -**Step 3: Implement RequireAuth and RequireGroup** - -Append to `authentik.go`: -```go -import "net/http" - -// RequireAuth is a Gin middleware that returns 401 if no authenticated user -// is present in the context. Use after WithAuthentik() middleware. -func RequireAuth() gin.HandlerFunc { - return func(c *gin.Context) { - if GetUser(c) == nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - Fail("unauthorised", "Authentication required")) - return - } - c.Next() - } -} - -// RequireGroup is a Gin middleware that returns 403 if the authenticated user -// does not belong to the specified group. Implies RequireAuth. -func RequireGroup(group string) gin.HandlerFunc { - return func(c *gin.Context) { - user := GetUser(c) - if user == nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - Fail("unauthorised", "Authentication required")) - return - } - if !user.HasGroup(group) { - c.AbortWithStatusJSON(http.StatusForbidden, - Fail("forbidden", "Insufficient permissions")) - return - } - c.Next() - } -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go -git commit -m "feat: add RequireAuth and RequireGroup middleware helpers - -RequireAuth returns 401 when no user in context. -RequireGroup returns 403 when user lacks the specified group. -Both use British English 'unauthorised' in error responses. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Update go-api Documentation - -**Files:** -- Modify: `/Users/snider/Code/go-api/CLAUDE.md` -- Modify: `/Users/snider/Code/go-api/README.md` - -**Step 1: Update CLAUDE.md** - -Add to the Project Overview section: -```markdown -## Authentik Integration - -go-api supports Authentik as the identity provider: - -- **Forward auth mode**: Reads `X-authentik-*` headers from Traefik (requires `TrustedProxy: true`) -- **OIDC mode**: Validates JWT Bearer tokens via OIDC discovery -- **Permissive middleware**: `WithAuthentik()` extracts user but doesn't block. Use `RequireAuth()` / `RequireGroup()` on routes that need auth. -- **Coexists with `WithBearerAuth()`** for service-to-service tokens - -```go -engine, _ := api.New( - api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.host.uk.com/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, - }), -) -``` -``` - -**Step 2: Update README.md** - -Add Authentik section with quick-start example showing `WithAuthentik()`, `GetUser()`, `RequireAuth()`, and `RequireGroup()`. - -**Step 3: Commit** - -```bash -cd /Users/snider/Code/go-api -git add CLAUDE.md README.md -git commit -m "docs: add Authentik integration guide to CLAUDE.md and README - -Co-Authored-By: Virgil " -``` - ---- - -### Task 10: Push go-api and DevOps Changes - -**Step 1: Push go-api** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 # Final verification -git push forge main -``` - -**Step 2: Push DevOps** - -```bash -cd /Users/snider/Code/DevOps -git push forge main -``` - -**Step 3: Update go-ecosystem memory** - -Update the go-api entry in the ecosystem inventory to note Authentik middleware. - ---- - -## Dependency Summary - -``` -Task 1 (enable vars) → Task 2 (playbook) → Task 3 (deploy) → Task 4 (OIDC app) - ↓ -Task 5 (user type) → Task 6 (header middleware) → Task 7 (JWT) → Task 8 (RequireAuth) - ↓ - Task 9 (docs) → Task 10 (push) -``` - -Tasks 1-4 (DevOps) and Tasks 5-8 (Go) are independent tracks that can run in parallel. Task 9-10 depend on both tracks. - -## Estimated Sizes - -| Task | LOC | Tests | -|------|-----|-------| -| Task 5: User type | ~50 | 4 | -| Task 6: Header middleware | ~60 | 5 | -| Task 7: JWT validation | ~80 | 2 | -| Task 8: RequireAuth/Group | ~30 | 5 | -| **go-api total** | **~220** | **16** | diff --git a/docs/plans/2026-02-21-core-help-design.md b/docs/plans/2026-02-21-core-help-design.md deleted file mode 100644 index 2943178..0000000 --- a/docs/plans/2026-02-21-core-help-design.md +++ /dev/null @@ -1,155 +0,0 @@ -# core.help Documentation Website — Design - -**Date:** 2026-02-21 -**Author:** Virgil -**Status:** Design approved -**Domain:** https://core.help - -## Problem - -Documentation is scattered across 39 repos (18 Go packages, 20 PHP packages, 1 CLI). There is no unified docs site. Developers need a single entry point to find CLI commands, Go package APIs, MCP tool references, and PHP module guides. - -## Solution - -A Hugo + Docsy static site at core.help, built from existing markdown docs aggregated by `core docs sync`. No new content — just collect and present what already exists across the ecosystem. - -## Architecture - -### Stack - -- **Hugo** — Go-native static site generator, sub-second builds -- **Docsy theme** — Purpose-built for technical docs (used by Kubernetes, gRPC, Knative) -- **BunnyCDN** — Static hosting with pull zone -- **`core docs sync --target hugo`** — Collects markdown from all repos into Hugo content tree - -### Why Hugo + Docsy (not VitePress or mdBook) - -- Go-native, no Node.js dependency -- Handles multi-section navigation (CLI, Go packages, PHP modules, MCP tools) -- Sub-second builds for ~250 markdown files -- Docsy has built-in search, versioned nav, API reference sections - -## Content Structure - -``` -docs-site/ -├── hugo.toml -├── content/ -│ ├── _index.md # Landing page -│ ├── getting-started/ # CLI top-level guides -│ │ ├── _index.md -│ │ ├── installation.md -│ │ ├── configuration.md -│ │ ├── user-guide.md -│ │ ├── troubleshooting.md -│ │ └── faq.md -│ ├── cli/ # CLI command reference (43 commands) -│ │ ├── _index.md -│ │ ├── dev/ # core dev commit, push, pull, etc. -│ │ ├── ai/ # core ai commands -│ │ ├── go/ # core go test, lint, etc. -│ │ └── ... -│ ├── go/ # Go ecosystem packages (18) -│ │ ├── _index.md # Ecosystem overview -│ │ ├── go-api/ # README + architecture/development/history -│ │ ├── go-ai/ -│ │ ├── go-mlx/ -│ │ ├── go-i18n/ -│ │ └── ... -│ ├── mcp/ # MCP tool reference (49 tools) -│ │ ├── _index.md -│ │ ├── file-operations.md -│ │ ├── process-management.md -│ │ ├── rag.md -│ │ └── ... -│ ├── php/ # PHP packages (from core-php/docs/packages/) -│ │ ├── _index.md -│ │ ├── admin/ -│ │ ├── tenant/ -│ │ ├── commerce/ -│ │ └── ... -│ └── kb/ # Knowledge base (wiki pages from go-mlx, go-i18n) -│ ├── _index.md -│ ├── mlx/ -│ └── i18n/ -├── static/ # Logos, favicons -├── layouts/ # Custom template overrides (minimal) -└── go.mod # Hugo modules (Docsy as module dep) -``` - -## Sync Pipeline - -`core docs sync --target hugo --output site/content/` performs: - -### Source Mapping - -``` -cli/docs/index.md → content/getting-started/_index.md -cli/docs/getting-started.md → content/getting-started/installation.md -cli/docs/user-guide.md → content/getting-started/user-guide.md -cli/docs/configuration.md → content/getting-started/configuration.md -cli/docs/troubleshooting.md → content/getting-started/troubleshooting.md -cli/docs/faq.md → content/getting-started/faq.md - -core/docs/cmd/**/*.md → content/cli/**/*.md - -go-*/README.md → content/go/{name}/_index.md -go-*/docs/*.md → content/go/{name}/*.md -go-*/KB/*.md → content/kb/{name-suffix}/*.md - -core-*/docs/**/*.md → content/php/{name-suffix}/**/*.md -``` - -### Front Matter Injection - -If a markdown file doesn't start with `---`, prepend: - -```yaml ---- -title: "{derived from filename}" -linkTitle: "{short name}" -weight: {auto-incremented} ---- -``` - -No other content transformations. Markdown stays as-is. - -### Build & Deploy - -```bash -core docs sync --target hugo --output docs-site/content/ -cd docs-site && hugo build -hugo deploy --target bunnycdn -``` - -Hugo deploy config in `hugo.toml`: - -```toml -[deployment] -[[deployment.targets]] -name = "bunnycdn" -URL = "s3://core-help?endpoint=storage.bunnycdn.com®ion=auto" -``` - -Credentials via env vars. - -## Registry - -All 39 repos registered in `.core/repos.yaml` with `docs: true`. Go repos use explicit `path:` fields since they live outside the PHP `base_path`. `FindRegistry()` checks `.core/repos.yaml` alongside `repos.yaml`. - -## Prerequisites Completed - -- [x] `.core/repos.yaml` created with all 39 repos -- [x] `FindRegistry()` updated to find `.core/repos.yaml` -- [x] `Repo.Path` supports explicit YAML override -- [x] go-api docs gap filled (architecture.md, development.md, history.md) -- [x] All 18 Go repos have standard docs trio - -## What Remains (Implementation Plan) - -1. Create docs-site repo with Hugo + Docsy scaffold -2. Extend `core docs sync` with `--target hugo` mode -3. Write section _index.md files (landing page, section intros) -4. Hugo config (navigation, search, theme colours) -5. BunnyCDN deployment config -6. CI pipeline on Forge (optional — can deploy manually initially) diff --git a/docs/plans/2026-02-21-core-help-plan.md b/docs/plans/2026-02-21-core-help-plan.md deleted file mode 100644 index e3bf5e1..0000000 --- a/docs/plans/2026-02-21-core-help-plan.md +++ /dev/null @@ -1,642 +0,0 @@ -# core.help Hugo Documentation Site — Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build a Hugo + Docsy documentation site at core.help that aggregates markdown from 39 repos via `core docs sync --target hugo`. - -**Architecture:** Hugo static site with Docsy theme, populated by extending `core docs sync` with a `--target hugo` flag that maps repo docs into Hugo's `content/` tree with auto-injected front matter. Deploy to BunnyCDN. - -**Tech Stack:** Hugo (Go SSG), Docsy theme (Hugo module), BunnyCDN, `core docs sync` CLI - ---- - -## Context - -The docs sync command lives in `/Users/snider/Code/host-uk/cli/cmd/docs/`. The site will be scaffolded at `/Users/snider/Code/host-uk/docs-site/`. The registry at `/Users/snider/Code/host-uk/.core/repos.yaml` already contains all 39 repos (20 PHP + 18 Go + 1 CLI) with explicit paths for Go repos. - -Key files: -- `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_sync.go` — sync command (modify) -- `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_scan.go` — repo scanner (modify) -- `/Users/snider/Code/host-uk/docs-site/` — Hugo site (create) - -## Task 1: Scaffold Hugo + Docsy site - -**Files:** -- Create: `/Users/snider/Code/host-uk/docs-site/hugo.toml` -- Create: `/Users/snider/Code/host-uk/docs-site/go.mod` -- Create: `/Users/snider/Code/host-uk/docs-site/content/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/getting-started/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/cli/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/go/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/mcp/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/php/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/kb/_index.md` - -This is the one-time Hugo scaffolding. No tests — just files. - -**`hugo.toml`:** -```toml -baseURL = "https://core.help/" -title = "Core Documentation" -languageCode = "en" -defaultContentLanguage = "en" - -enableRobotsTXT = true -enableGitInfo = false - -[outputs] -home = ["HTML", "JSON"] -section = ["HTML"] - -[params] -description = "Documentation for the Core CLI, Go packages, PHP modules, and MCP tools" -copyright = "Host UK — EUPL-1.2" - -[params.ui] -sidebar_menu_compact = true -breadcrumb_disable = false -sidebar_search_disable = false -navbar_logo = false - -[params.ui.readingtime] -enable = false - -[module] -proxy = "direct" - -[module.hugoVersion] -extended = true -min = "0.120.0" - -[[module.imports]] -path = "github.com/google/docsy" -disable = false - -[markup.goldmark.renderer] -unsafe = true - -[menu] -[[menu.main]] -name = "Getting Started" -weight = 10 -url = "/getting-started/" -[[menu.main]] -name = "CLI Reference" -weight = 20 -url = "/cli/" -[[menu.main]] -name = "Go Packages" -weight = 30 -url = "/go/" -[[menu.main]] -name = "MCP Tools" -weight = 40 -url = "/mcp/" -[[menu.main]] -name = "PHP Packages" -weight = 50 -url = "/php/" -[[menu.main]] -name = "Knowledge Base" -weight = 60 -url = "/kb/" -``` - -**`go.mod`:** -``` -module github.com/host-uk/docs-site - -go 1.22 - -require github.com/google/docsy v0.11.0 -``` - -Note: Run `hugo mod get` after creating these files to populate `go.sum` and download Docsy. - -**Section `_index.md` files** — each needs Hugo front matter: - -`content/_index.md`: -```markdown ---- -title: "Core Documentation" -description: "Documentation for the Core CLI, Go packages, PHP modules, and MCP tools" ---- - -Welcome to the Core ecosystem documentation. - -## Sections - -- [Getting Started](/getting-started/) — Installation, configuration, and first steps -- [CLI Reference](/cli/) — Command reference for `core` CLI -- [Go Packages](/go/) — Go ecosystem package documentation -- [MCP Tools](/mcp/) — Model Context Protocol tool reference -- [PHP Packages](/php/) — PHP module documentation -- [Knowledge Base](/kb/) — Wiki articles and deep dives -``` - -`content/getting-started/_index.md`: -```markdown ---- -title: "Getting Started" -linkTitle: "Getting Started" -weight: 10 -description: "Installation, configuration, and first steps with the Core CLI" ---- -``` - -`content/cli/_index.md`: -```markdown ---- -title: "CLI Reference" -linkTitle: "CLI Reference" -weight: 20 -description: "Command reference for the core CLI tool" ---- -``` - -`content/go/_index.md`: -```markdown ---- -title: "Go Packages" -linkTitle: "Go Packages" -weight: 30 -description: "Documentation for the Go ecosystem packages" ---- -``` - -`content/mcp/_index.md`: -```markdown ---- -title: "MCP Tools" -linkTitle: "MCP Tools" -weight: 40 -description: "Model Context Protocol tool reference — file operations, RAG, ML inference, process management" ---- -``` - -`content/php/_index.md`: -```markdown ---- -title: "PHP Packages" -linkTitle: "PHP Packages" -weight: 50 -description: "Documentation for the PHP module ecosystem" ---- -``` - -`content/kb/_index.md`: -```markdown ---- -title: "Knowledge Base" -linkTitle: "Knowledge Base" -weight: 60 -description: "Wiki articles, deep dives, and reference material" ---- -``` - -**Verify:** After creating files, run from `/Users/snider/Code/host-uk/docs-site/`: -```bash -hugo mod get -hugo server -``` -The site should start and show the landing page with Docsy theme at `localhost:1313`. - -**Commit:** -```bash -cd /Users/snider/Code/host-uk/docs-site -git init -git add . -git commit -m "feat: scaffold Hugo + Docsy documentation site" -``` - ---- - -## Task 2: Extend scanRepoDocs to collect KB/ and README - -**Files:** -- Modify: `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_scan.go` - -Currently `scanRepoDocs` only collects files from `docs/`. For the Hugo target we also need: -- `KB/**/*.md` files (wiki pages from go-mlx, go-i18n) -- `README.md` content (becomes the package _index.md) - -Add a `KBFiles []string` field to `RepoDocInfo` and scan `KB/` alongside `docs/`: - -```go -type RepoDocInfo struct { - Name string - Path string - HasDocs bool - Readme string - ClaudeMd string - Changelog string - DocsFiles []string // All files in docs/ directory (recursive) - KBFiles []string // All files in KB/ directory (recursive) -} -``` - -In `scanRepoDocs`, after the `docs/` walk, add a second walk for `KB/`: - -```go -// Recursively scan KB/ directory for .md files -kbDir := filepath.Join(repo.Path, "KB") -if _, err := io.Local.List(kbDir); err == nil { - _ = filepath.WalkDir(kbDir, func(path string, d fs.DirEntry, err error) error { - if err != nil { - return nil - } - if d.IsDir() || !strings.HasSuffix(d.Name(), ".md") { - return nil - } - relPath, _ := filepath.Rel(kbDir, path) - info.KBFiles = append(info.KBFiles, relPath) - info.HasDocs = true - return nil - }) -} -``` - -**Tests:** The existing tests should still pass. No new test file needed — this is a data-collection change. - -**Verify:** `cd /Users/snider/Code/host-uk/cli && GOWORK=off go build ./cmd/docs/...` - -**Commit:** -```bash -git add cmd/docs/cmd_scan.go -git commit -m "feat(docs): scan KB/ directory alongside docs/" -``` - ---- - -## Task 3: Add `--target hugo` flag and Hugo sync logic - -**Files:** -- Modify: `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_sync.go` - -This is the main task. Add a `--target` flag (default `"php"`) and a new `runHugoSync` function that maps repos to Hugo's content tree. - -**Add flag variable and registration:** - -```go -var ( - docsSyncRegistryPath string - docsSyncDryRun bool - docsSyncOutputDir string - docsSyncTarget string -) - -func init() { - docsSyncCmd.Flags().StringVar(&docsSyncRegistryPath, "registry", "", i18n.T("common.flag.registry")) - docsSyncCmd.Flags().BoolVar(&docsSyncDryRun, "dry-run", false, i18n.T("cmd.docs.sync.flag.dry_run")) - docsSyncCmd.Flags().StringVar(&docsSyncOutputDir, "output", "", i18n.T("cmd.docs.sync.flag.output")) - docsSyncCmd.Flags().StringVar(&docsSyncTarget, "target", "php", "Target format: php (default) or hugo") -} -``` - -**Update RunE to pass target:** -```go -RunE: func(cmd *cli.Command, args []string) error { - return runDocsSync(docsSyncRegistryPath, docsSyncOutputDir, docsSyncDryRun, docsSyncTarget) -}, -``` - -**Update `runDocsSync` signature and add target dispatch:** -```go -func runDocsSync(registryPath string, outputDir string, dryRun bool, target string) error { - reg, basePath, err := loadRegistry(registryPath) - if err != nil { - return err - } - - switch target { - case "hugo": - return runHugoSync(reg, basePath, outputDir, dryRun) - default: - return runPHPSync(reg, basePath, outputDir, dryRun) - } -} -``` - -**Rename current sync body to `runPHPSync`** — extract lines 67-159 of current `runDocsSync` into `runPHPSync(reg, basePath, outputDir string, dryRun bool) error`. This is a pure extract, no logic changes. - -**Add `hugoOutputName` mapping function:** -```go -// hugoOutputName maps repo name to Hugo content section and folder. -// Returns (section, folder) where section is the top-level content dir. -func hugoOutputName(repoName string) (string, string) { - // CLI guides - if repoName == "cli" { - return "getting-started", "" - } - // Core CLI command docs - if repoName == "core" { - return "cli", "" - } - // Go packages - if strings.HasPrefix(repoName, "go-") { - return "go", repoName - } - // PHP packages - if strings.HasPrefix(repoName, "core-") { - return "php", strings.TrimPrefix(repoName, "core-") - } - return "go", repoName -} -``` - -**Add front matter injection helper:** -```go -// injectFrontMatter prepends Hugo front matter to markdown content if missing. -func injectFrontMatter(content []byte, title string, weight int) []byte { - // Already has front matter - if bytes.HasPrefix(bytes.TrimSpace(content), []byte("---")) { - return content - } - fm := fmt.Sprintf("---\ntitle: %q\nweight: %d\n---\n\n", title, weight) - return append([]byte(fm), content...) -} - -// titleFromFilename derives a human-readable title from a filename. -func titleFromFilename(filename string) string { - name := strings.TrimSuffix(filepath.Base(filename), ".md") - name = strings.ReplaceAll(name, "-", " ") - name = strings.ReplaceAll(name, "_", " ") - // Title case - words := strings.Fields(name) - for i, w := range words { - if len(w) > 0 { - words[i] = strings.ToUpper(w[:1]) + w[1:] - } - } - return strings.Join(words, " ") -} -``` - -**Add `runHugoSync` function:** -```go -func runHugoSync(reg *repos.Registry, basePath string, outputDir string, dryRun bool) error { - if outputDir == "" { - outputDir = filepath.Join(basePath, "docs-site", "content") - } - - // Scan all repos - var docsInfo []RepoDocInfo - for _, repo := range reg.List() { - if repo.Name == "core-template" || repo.Name == "core-claude" { - continue - } - info := scanRepoDocs(repo) - if info.HasDocs { - docsInfo = append(docsInfo, info) - } - } - - if len(docsInfo) == 0 { - cli.Text("No documentation found") - return nil - } - - cli.Print("\n Hugo sync: %d repos with docs → %s\n\n", len(docsInfo), outputDir) - - // Show plan - for _, info := range docsInfo { - section, folder := hugoOutputName(info.Name) - target := section - if folder != "" { - target = section + "/" + folder - } - fileCount := len(info.DocsFiles) + len(info.KBFiles) - if info.Readme != "" { - fileCount++ - } - cli.Print(" %s → %s/ (%d files)\n", repoNameStyle.Render(info.Name), target, fileCount) - } - - if dryRun { - cli.Print("\n Dry run — no files written\n") - return nil - } - - cli.Blank() - if !confirm("Sync to Hugo content directory?") { - cli.Text("Aborted") - return nil - } - - cli.Blank() - var synced int - for _, info := range docsInfo { - section, folder := hugoOutputName(info.Name) - - // Build destination path - destDir := filepath.Join(outputDir, section) - if folder != "" { - destDir = filepath.Join(destDir, folder) - } - - // Copy docs/ files - weight := 10 - docsDir := filepath.Join(info.Path, "docs") - for _, f := range info.DocsFiles { - src := filepath.Join(docsDir, f) - dst := filepath.Join(destDir, f) - if err := copyWithFrontMatter(src, dst, weight); err != nil { - cli.Print(" %s %s: %s\n", errorStyle.Render("✗"), f, err) - continue - } - weight += 10 - } - - // Copy README.md as _index.md (if not CLI/core which use their own index) - if info.Readme != "" && folder != "" { - dst := filepath.Join(destDir, "_index.md") - if err := copyWithFrontMatter(info.Readme, dst, 1); err != nil { - cli.Print(" %s README: %s\n", errorStyle.Render("✗"), err) - } - } - - // Copy KB/ files to kb/{suffix}/ - if len(info.KBFiles) > 0 { - // Extract suffix: go-mlx → mlx, go-i18n → i18n - suffix := strings.TrimPrefix(info.Name, "go-") - kbDestDir := filepath.Join(outputDir, "kb", suffix) - kbDir := filepath.Join(info.Path, "KB") - kbWeight := 10 - for _, f := range info.KBFiles { - src := filepath.Join(kbDir, f) - dst := filepath.Join(kbDestDir, f) - if err := copyWithFrontMatter(src, dst, kbWeight); err != nil { - cli.Print(" %s KB/%s: %s\n", errorStyle.Render("✗"), f, err) - continue - } - kbWeight += 10 - } - } - - cli.Print(" %s %s\n", successStyle.Render("✓"), info.Name) - synced++ - } - - cli.Print("\n Synced %d repos to Hugo content\n", synced) - return nil -} - -// copyWithFrontMatter copies a markdown file, injecting front matter if missing. -func copyWithFrontMatter(src, dst string, weight int) error { - if err := io.Local.EnsureDir(filepath.Dir(dst)); err != nil { - return err - } - content, err := io.Local.Read(src) - if err != nil { - return err - } - title := titleFromFilename(src) - result := injectFrontMatter([]byte(content), title, weight) - return io.Local.Write(dst, string(result)) -} -``` - -**Add imports** at top of file: -```go -import ( - "bytes" - "fmt" - "path/filepath" - "strings" - - "forge.lthn.ai/core/go/pkg/cli" - "forge.lthn.ai/core/go/pkg/i18n" - "forge.lthn.ai/core/go/pkg/io" - "forge.lthn.ai/core/go/pkg/repos" -) -``` - -**Verify:** `cd /Users/snider/Code/host-uk/cli && GOWORK=off go build ./cmd/docs/...` - -**Commit:** -```bash -git add cmd/docs/cmd_sync.go -git commit -m "feat(docs): add --target hugo sync mode for core.help" -``` - ---- - -## Task 4: Test the full pipeline - -**No code changes.** Run the pipeline end-to-end. - -**Step 1:** Sync docs to Hugo: -```bash -cd /Users/snider/Code/host-uk -core docs sync --target hugo --dry-run -``` -Verify all 39 repos appear with correct section mappings. - -**Step 2:** Run actual sync: -```bash -core docs sync --target hugo -``` - -**Step 3:** Build and preview: -```bash -cd /Users/snider/Code/host-uk/docs-site -hugo server -``` -Open `localhost:1313` and verify: -- Landing page renders with section links -- Getting Started section has CLI guides -- CLI Reference section has command docs -- Go Packages section has 18 packages with architecture/development/history -- PHP Packages section has PHP module docs -- Knowledge Base has MLX and i18n wiki pages -- Navigation works, search works - -**Step 4:** Fix any issues found during preview. - -**Commit docs-site content:** -```bash -cd /Users/snider/Code/host-uk/docs-site -git add content/ -git commit -m "feat: sync initial content from 39 repos" -``` - ---- - -## Task 5: BunnyCDN deployment config - -**Files:** -- Modify: `/Users/snider/Code/host-uk/docs-site/hugo.toml` - -Add deployment target: - -```toml -[deployment] -[[deployment.targets]] -name = "production" -URL = "s3://core-help?endpoint=storage.bunnycdn.com®ion=auto" -``` - -Add a `Taskfile.yml` for convenience: - -**Create:** `/Users/snider/Code/host-uk/docs-site/Taskfile.yml` -```yaml -version: '3' - -tasks: - dev: - desc: Start Hugo dev server - cmds: - - hugo server --buildDrafts - - build: - desc: Build static site - cmds: - - hugo --minify - - sync: - desc: Sync docs from all repos - dir: .. - cmds: - - core docs sync --target hugo - - deploy: - desc: Build and deploy to BunnyCDN - cmds: - - task: sync - - task: build - - hugo deploy --target production - - clean: - desc: Remove generated content (keeps _index.md files) - cmds: - - find content -name "*.md" ! -name "_index.md" -delete -``` - -**Verify:** `task dev` starts the site. - -**Commit:** -```bash -git add hugo.toml Taskfile.yml -git commit -m "feat: add BunnyCDN deployment config and Taskfile" -``` - ---- - -## Dependency Sequencing - -``` -Task 1 (Hugo scaffold) — independent, do first -Task 2 (scan KB/) — independent, can parallel with Task 1 -Task 3 (--target hugo) — depends on Task 2 -Task 4 (test pipeline) — depends on Tasks 1 + 3 -Task 5 (deploy config) — depends on Task 1 -``` - -## Verification - -After all tasks: -1. `core docs sync --target hugo` populates `docs-site/content/` from all repos -2. `cd docs-site && hugo server` renders the full site -3. Navigation has 6 sections: Getting Started, CLI, Go, MCP, PHP, KB -4. All existing markdown renders correctly with auto-injected front matter -5. `hugo build` produces `public/` with no errors diff --git a/docs/plans/completed/2026-02-05-mcp-integration-original.md b/docs/plans/completed/2026-02-05-mcp-integration-original.md deleted file mode 100644 index 9b3a109..0000000 --- a/docs/plans/completed/2026-02-05-mcp-integration-original.md +++ /dev/null @@ -1,851 +0,0 @@ -# MCP Integration Implementation Plan - -> **Status:** Completed. MCP command now lives in `go-ai/cmd/mcpcmd/`. Code examples below use the old `init()` + `RegisterCommands()` pattern — the current approach uses `cli.WithCommands()` (see cli-meta-package-design.md). - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add `core mcp serve` command with RAG and metrics tools, then configure the agentic-flows plugin to use it. - -**Architecture:** Create a new `mcp` command package that starts the pkg/mcp server with extended tools. RAG tools call the existing exported functions in internal/cmd/rag. Metrics tools call pkg/ai directly. The agentic-flows plugin gets a `.mcp.json` that spawns `core mcp serve`. - -**Tech Stack:** Go 1.25, github.com/modelcontextprotocol/go-sdk/mcp, pkg/rag, pkg/ai - ---- - -## Task 1: Add RAG tools to pkg/mcp - -**Files:** -- Create: `pkg/mcp/tools_rag.go` -- Modify: `pkg/mcp/mcp.go:99-101` (registerTools) -- Test: `pkg/mcp/tools_rag_test.go` - -**Step 1: Write the failing test** - -Create `pkg/mcp/tools_rag_test.go`: - -```go -package mcp - -import ( - "context" - "testing" - - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -func TestRAGQueryTool_Good(t *testing.T) { - // This test verifies the tool is registered and callable. - // It doesn't require Qdrant/Ollama running - just checks structure. - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - // Check that rag_query tool is registered - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "rag_query" { - found = true - break - } - } - if !found { - t.Error("rag_query tool not registered") - } -} - -func TestRAGQueryInput_Good(t *testing.T) { - input := RAGQueryInput{ - Question: "how do I deploy?", - Collection: "hostuk-docs", - TopK: 5, - } - if input.Question == "" { - t.Error("Question should not be empty") - } -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -run TestRAGQueryTool ./pkg/mcp/... -v` -Expected: FAIL with "rag_query tool not registered" - -**Step 3: Create tools_rag.go with types and tool registration** - -Create `pkg/mcp/tools_rag.go`: - -```go -package mcp - -import ( - "context" - "fmt" - - ragcmd "forge.lthn.ai/core/cli/internal/cmd/rag" - "forge.lthn.ai/core/cli/pkg/rag" - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -// RAG tool input/output types - -// RAGQueryInput contains parameters for querying the vector database. -type RAGQueryInput struct { - Question string `json:"question"` - Collection string `json:"collection,omitempty"` - TopK int `json:"top_k,omitempty"` -} - -// RAGQueryOutput contains the query results. -type RAGQueryOutput struct { - Results []RAGResult `json:"results"` - Context string `json:"context"` -} - -// RAGResult represents a single search result. -type RAGResult struct { - Content string `json:"content"` - Score float32 `json:"score"` - Source string `json:"source"` - Metadata map[string]string `json:"metadata,omitempty"` -} - -// RAGIngestInput contains parameters for ingesting documents. -type RAGIngestInput struct { - Path string `json:"path"` - Collection string `json:"collection,omitempty"` - Recreate bool `json:"recreate,omitempty"` -} - -// RAGIngestOutput contains the ingestion results. -type RAGIngestOutput struct { - Success bool `json:"success"` - Path string `json:"path"` - Chunks int `json:"chunks"` - Message string `json:"message,omitempty"` -} - -// RAGCollectionsInput contains parameters for listing collections. -type RAGCollectionsInput struct { - ShowStats bool `json:"show_stats,omitempty"` -} - -// RAGCollectionsOutput contains the list of collections. -type RAGCollectionsOutput struct { - Collections []CollectionInfo `json:"collections"` -} - -// CollectionInfo describes a Qdrant collection. -type CollectionInfo struct { - Name string `json:"name"` - PointsCount uint64 `json:"points_count,omitempty"` - Status string `json:"status,omitempty"` -} - -// registerRAGTools adds RAG tools to the MCP server. -func (s *Service) registerRAGTools(server *mcp.Server) { - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_query", - Description: "Query the vector database for relevant documents using semantic search", - }, s.ragQuery) - - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_ingest", - Description: "Ingest a file or directory into the vector database", - }, s.ragIngest) - - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_collections", - Description: "List available vector database collections", - }, s.ragCollections) -} - -func (s *Service) ragQuery(ctx context.Context, req *mcp.CallToolRequest, input RAGQueryInput) (*mcp.CallToolResult, RAGQueryOutput, error) { - s.logger.Info("MCP tool execution", "tool", "rag_query", "question", input.Question) - - collection := input.Collection - if collection == "" { - collection = "hostuk-docs" - } - topK := input.TopK - if topK <= 0 { - topK = 5 - } - - results, err := ragcmd.QueryDocs(ctx, input.Question, collection, topK) - if err != nil { - return nil, RAGQueryOutput{}, fmt.Errorf("query failed: %w", err) - } - - // Convert to output format - out := RAGQueryOutput{ - Results: make([]RAGResult, 0, len(results)), - Context: rag.FormatResultsContext(results), - } - for _, r := range results { - out.Results = append(out.Results, RAGResult{ - Content: r.Content, - Score: r.Score, - Source: r.Source, - Metadata: r.Metadata, - }) - } - - return nil, out, nil -} - -func (s *Service) ragIngest(ctx context.Context, req *mcp.CallToolRequest, input RAGIngestInput) (*mcp.CallToolResult, RAGIngestOutput, error) { - s.logger.Security("MCP tool execution", "tool", "rag_ingest", "path", input.Path) - - collection := input.Collection - if collection == "" { - collection = "hostuk-docs" - } - - // Check if path is a file or directory - info, err := s.medium.Stat(input.Path) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("path not found: %w", err) - } - - if info.IsDir() { - err = ragcmd.IngestDirectory(ctx, input.Path, collection, input.Recreate) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("ingest directory failed: %w", err) - } - return nil, RAGIngestOutput{ - Success: true, - Path: input.Path, - Message: fmt.Sprintf("Ingested directory into collection %s", collection), - }, nil - } - - chunks, err := ragcmd.IngestFile(ctx, input.Path, collection) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("ingest file failed: %w", err) - } - - return nil, RAGIngestOutput{ - Success: true, - Path: input.Path, - Chunks: chunks, - Message: fmt.Sprintf("Ingested %d chunks into collection %s", chunks, collection), - }, nil -} - -func (s *Service) ragCollections(ctx context.Context, req *mcp.CallToolRequest, input RAGCollectionsInput) (*mcp.CallToolResult, RAGCollectionsOutput, error) { - s.logger.Info("MCP tool execution", "tool", "rag_collections") - - client, err := rag.NewQdrantClient(rag.DefaultQdrantConfig()) - if err != nil { - return nil, RAGCollectionsOutput{}, fmt.Errorf("connect to Qdrant: %w", err) - } - defer func() { _ = client.Close() }() - - names, err := client.ListCollections(ctx) - if err != nil { - return nil, RAGCollectionsOutput{}, fmt.Errorf("list collections: %w", err) - } - - out := RAGCollectionsOutput{ - Collections: make([]CollectionInfo, 0, len(names)), - } - - for _, name := range names { - info := CollectionInfo{Name: name} - if input.ShowStats { - cinfo, err := client.CollectionInfo(ctx, name) - if err == nil { - info.PointsCount = cinfo.PointsCount - info.Status = cinfo.Status.String() - } - } - out.Collections = append(out.Collections, info) - } - - return nil, out, nil -} -``` - -**Step 4: Update mcp.go to call registerRAGTools** - -In `pkg/mcp/mcp.go`, modify the `registerTools` function (around line 104) to add: - -```go -func (s *Service) registerTools(server *mcp.Server) { - // File operations (existing) - // ... existing code ... - - // RAG operations - s.registerRAGTools(server) -} -``` - -**Step 5: Run test to verify it passes** - -Run: `go test -run TestRAGQuery ./pkg/mcp/... -v` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/mcp/tools_rag.go pkg/mcp/tools_rag_test.go pkg/mcp/mcp.go -git commit -m "feat(mcp): add RAG tools (query, ingest, collections)" -``` - ---- - -## Task 2: Add metrics tools to pkg/mcp - -**Files:** -- Create: `pkg/mcp/tools_metrics.go` -- Modify: `pkg/mcp/mcp.go` (registerTools) -- Test: `pkg/mcp/tools_metrics_test.go` - -**Step 1: Write the failing test** - -Create `pkg/mcp/tools_metrics_test.go`: - -```go -package mcp - -import ( - "testing" -) - -func TestMetricsRecordTool_Good(t *testing.T) { - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "metrics_record" { - found = true - break - } - } - if !found { - t.Error("metrics_record tool not registered") - } -} - -func TestMetricsQueryTool_Good(t *testing.T) { - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "metrics_query" { - found = true - break - } - } - if !found { - t.Error("metrics_query tool not registered") - } -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -run TestMetrics ./pkg/mcp/... -v` -Expected: FAIL - -**Step 3: Create tools_metrics.go** - -Create `pkg/mcp/tools_metrics.go`: - -```go -package mcp - -import ( - "context" - "fmt" - "time" - - "forge.lthn.ai/core/cli/pkg/ai" - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -// Metrics tool input/output types - -// MetricsRecordInput contains parameters for recording a metric event. -type MetricsRecordInput struct { - Type string `json:"type"` - AgentID string `json:"agent_id,omitempty"` - Repo string `json:"repo,omitempty"` - Data map[string]any `json:"data,omitempty"` -} - -// MetricsRecordOutput contains the result of recording. -type MetricsRecordOutput struct { - Success bool `json:"success"` - Timestamp time.Time `json:"timestamp"` -} - -// MetricsQueryInput contains parameters for querying metrics. -type MetricsQueryInput struct { - Since string `json:"since,omitempty"` // e.g., "7d", "24h" -} - -// MetricsQueryOutput contains the query results. -type MetricsQueryOutput struct { - Total int `json:"total"` - ByType []MetricCount `json:"by_type"` - ByRepo []MetricCount `json:"by_repo"` - ByAgent []MetricCount `json:"by_agent"` - Events []MetricEventBrief `json:"events,omitempty"` -} - -// MetricCount represents a count by key. -type MetricCount struct { - Key string `json:"key"` - Count int `json:"count"` -} - -// MetricEventBrief is a simplified event for output. -type MetricEventBrief struct { - Type string `json:"type"` - Timestamp time.Time `json:"timestamp"` - AgentID string `json:"agent_id,omitempty"` - Repo string `json:"repo,omitempty"` -} - -// registerMetricsTools adds metrics tools to the MCP server. -func (s *Service) registerMetricsTools(server *mcp.Server) { - mcp.AddTool(server, &mcp.Tool{ - Name: "metrics_record", - Description: "Record a metric event (AI task, security scan, job creation, etc.)", - }, s.metricsRecord) - - mcp.AddTool(server, &mcp.Tool{ - Name: "metrics_query", - Description: "Query recorded metrics with aggregation by type, repo, and agent", - }, s.metricsQuery) -} - -func (s *Service) metricsRecord(ctx context.Context, req *mcp.CallToolRequest, input MetricsRecordInput) (*mcp.CallToolResult, MetricsRecordOutput, error) { - s.logger.Info("MCP tool execution", "tool", "metrics_record", "type", input.Type) - - if input.Type == "" { - return nil, MetricsRecordOutput{}, fmt.Errorf("type is required") - } - - event := ai.Event{ - Type: input.Type, - Timestamp: time.Now(), - AgentID: input.AgentID, - Repo: input.Repo, - Data: input.Data, - } - - if err := ai.Record(event); err != nil { - return nil, MetricsRecordOutput{}, fmt.Errorf("record event: %w", err) - } - - return nil, MetricsRecordOutput{ - Success: true, - Timestamp: event.Timestamp, - }, nil -} - -func (s *Service) metricsQuery(ctx context.Context, req *mcp.CallToolRequest, input MetricsQueryInput) (*mcp.CallToolResult, MetricsQueryOutput, error) { - s.logger.Info("MCP tool execution", "tool", "metrics_query", "since", input.Since) - - since := input.Since - if since == "" { - since = "7d" - } - - duration, err := parseDuration(since) - if err != nil { - return nil, MetricsQueryOutput{}, fmt.Errorf("invalid since value: %w", err) - } - - sinceTime := time.Now().Add(-duration) - events, err := ai.ReadEvents(sinceTime) - if err != nil { - return nil, MetricsQueryOutput{}, fmt.Errorf("read events: %w", err) - } - - summary := ai.Summary(events) - - out := MetricsQueryOutput{ - Total: summary["total"].(int), - } - - // Convert by_type - if byType, ok := summary["by_type"].([]map[string]any); ok { - for _, entry := range byType { - out.ByType = append(out.ByType, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Convert by_repo - if byRepo, ok := summary["by_repo"].([]map[string]any); ok { - for _, entry := range byRepo { - out.ByRepo = append(out.ByRepo, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Convert by_agent - if byAgent, ok := summary["by_agent"].([]map[string]any); ok { - for _, entry := range byAgent { - out.ByAgent = append(out.ByAgent, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Include last 10 events for context - limit := 10 - if len(events) < limit { - limit = len(events) - } - for i := len(events) - limit; i < len(events); i++ { - ev := events[i] - out.Events = append(out.Events, MetricEventBrief{ - Type: ev.Type, - Timestamp: ev.Timestamp, - AgentID: ev.AgentID, - Repo: ev.Repo, - }) - } - - return nil, out, nil -} - -// parseDuration parses a human-friendly duration like "7d", "24h", "30d". -func parseDuration(s string) (time.Duration, error) { - if len(s) < 2 { - return 0, fmt.Errorf("invalid duration: %s", s) - } - - unit := s[len(s)-1] - value := s[:len(s)-1] - - var n int - if _, err := fmt.Sscanf(value, "%d", &n); err != nil { - return 0, fmt.Errorf("invalid duration: %s", s) - } - - if n <= 0 { - return 0, fmt.Errorf("duration must be positive: %s", s) - } - - switch unit { - case 'd': - return time.Duration(n) * 24 * time.Hour, nil - case 'h': - return time.Duration(n) * time.Hour, nil - case 'm': - return time.Duration(n) * time.Minute, nil - default: - return 0, fmt.Errorf("unknown unit %c in duration: %s", unit, s) - } -} -``` - -**Step 4: Update mcp.go to call registerMetricsTools** - -In `pkg/mcp/mcp.go`, add to `registerTools`: - -```go -func (s *Service) registerTools(server *mcp.Server) { - // ... existing file operations ... - - // RAG operations - s.registerRAGTools(server) - - // Metrics operations - s.registerMetricsTools(server) -} -``` - -**Step 5: Run test to verify it passes** - -Run: `go test -run TestMetrics ./pkg/mcp/... -v` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/mcp/tools_metrics.go pkg/mcp/tools_metrics_test.go pkg/mcp/mcp.go -git commit -m "feat(mcp): add metrics tools (record, query)" -``` - ---- - -## Task 3: Create `core mcp serve` command - -**Files:** -- Create: `internal/cmd/mcpcmd/cmd_mcp.go` -- Modify: `internal/variants/full.go` (add import) -- Test: Manual test via `core mcp serve` - -**Step 1: Create the mcp command package** - -Create `internal/cmd/mcpcmd/cmd_mcp.go`: - -```go -package mcpcmd - -import ( - "context" - "os" - "os/signal" - "syscall" - - "forge.lthn.ai/core/cli/pkg/cli" - "forge.lthn.ai/core/cli/pkg/i18n" - "forge.lthn.ai/core/cli/pkg/mcp" -) - -func init() { - cli.RegisterCommands(AddMCPCommands) -} - -var ( - mcpWorkspace string -) - -var mcpCmd = &cli.Command{ - Use: "mcp", - Short: i18n.T("cmd.mcp.short"), - Long: i18n.T("cmd.mcp.long"), -} - -var serveCmd = &cli.Command{ - Use: "serve", - Short: i18n.T("cmd.mcp.serve.short"), - Long: i18n.T("cmd.mcp.serve.long"), - RunE: func(cmd *cli.Command, args []string) error { - return runServe() - }, -} - -func AddMCPCommands(root *cli.Command) { - initMCPFlags() - mcpCmd.AddCommand(serveCmd) - root.AddCommand(mcpCmd) -} - -func initMCPFlags() { - serveCmd.Flags().StringVar(&mcpWorkspace, "workspace", "", i18n.T("cmd.mcp.serve.flag.workspace")) -} - -func runServe() error { - opts := []mcp.Option{} - - if mcpWorkspace != "" { - opts = append(opts, mcp.WithWorkspaceRoot(mcpWorkspace)) - } else { - // Default to unrestricted for MCP server - opts = append(opts, mcp.WithWorkspaceRoot("")) - } - - svc, err := mcp.New(opts...) - if err != nil { - return cli.Wrap(err, "create MCP service") - } - - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() - - // Handle shutdown signals - sigCh := make(chan os.Signal, 1) - signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM) - go func() { - <-sigCh - cancel() - }() - - return svc.Run(ctx) -} -``` - -**Step 2: Add i18n strings** - -Create or update `pkg/i18n/en.yaml` (if it exists) or add to the existing i18n mechanism: - -```yaml -cmd.mcp.short: "MCP (Model Context Protocol) server" -cmd.mcp.long: "Start an MCP server for Claude Code integration with file, RAG, and metrics tools." -cmd.mcp.serve.short: "Start the MCP server" -cmd.mcp.serve.long: "Start the MCP server in stdio mode. Use MCP_ADDR env var for TCP mode." -cmd.mcp.serve.flag.workspace: "Restrict file operations to this directory (empty = unrestricted)" -``` - -**Step 3: Add import to full.go** - -Modify `internal/variants/full.go` to add: - -```go -import ( - // ... existing imports ... - _ "forge.lthn.ai/core/cli/internal/cmd/mcpcmd" -) -``` - -**Step 4: Build and test** - -Run: `go build && ./core mcp serve --help` -Expected: Help output showing the serve command - -**Step 5: Test MCP server manually** - -Run: `echo '{"jsonrpc":"2.0","method":"tools/list","id":1}' | ./core mcp serve` -Expected: JSON response listing all tools including rag_query, metrics_record, etc. - -**Step 6: Commit** - -```bash -git add internal/cmd/mcpcmd/cmd_mcp.go internal/variants/full.go -git commit -m "feat: add 'core mcp serve' command" -``` - ---- - -## Task 4: Configure agentic-flows plugin with .mcp.json - -**Files:** -- Create: `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.mcp.json` -- Modify: `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.claude-plugin/plugin.json` (optional, add mcpServers) - -**Step 1: Create .mcp.json** - -Create `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.mcp.json`: - -```json -{ - "core-cli": { - "command": "core", - "args": ["mcp", "serve"], - "env": { - "MCP_WORKSPACE": "" - } - } -} -``` - -**Step 2: Verify plugin loads** - -Restart Claude Code and run `/mcp` to verify the core-cli server appears. - -**Step 3: Test MCP tools** - -Test that tools are available: -- `mcp__plugin_agentic-flows_core-cli__rag_query` -- `mcp__plugin_agentic-flows_core-cli__rag_ingest` -- `mcp__plugin_agentic-flows_core-cli__rag_collections` -- `mcp__plugin_agentic-flows_core-cli__metrics_record` -- `mcp__plugin_agentic-flows_core-cli__metrics_query` -- `mcp__plugin_agentic-flows_core-cli__file_read` -- etc. - -**Step 4: Commit plugin changes** - -```bash -cd /home/shared/hostuk/claude-plugins -git add plugins/agentic-flows/.mcp.json -git commit -m "feat(agentic-flows): add MCP server configuration for core-cli" -``` - ---- - -## Task 5: Update documentation - -**Files:** -- Modify: `/home/claude/.claude/projects/-home-claude/memory/MEMORY.md` -- Modify: `/home/claude/.claude/projects/-home-claude/memory/plugin-dev-notes.md` - -**Step 1: Update MEMORY.md** - -Add under "Core CLI MCP Server" section: - -```markdown -### Core CLI MCP Server -- **Command:** `core mcp serve` (stdio mode) or `MCP_ADDR=:9000 core mcp serve` (TCP) -- **Tools available:** - - File ops: file_read, file_write, file_edit, file_delete, file_rename, file_exists, dir_list, dir_create - - RAG: rag_query, rag_ingest, rag_collections - - Metrics: metrics_record, metrics_query - - Language: lang_detect, lang_list -- **Plugin config:** `plugins/agentic-flows/.mcp.json` -``` - -**Step 2: Update plugin-dev-notes.md** - -Add section: - -```markdown -## MCP Server (core mcp serve) - -### Available Tools -| Tool | Description | -|------|-------------| -| file_read | Read file contents | -| file_write | Write file contents | -| file_edit | Edit file (replace string) | -| file_delete | Delete file | -| file_rename | Rename/move file | -| file_exists | Check if file exists | -| dir_list | List directory contents | -| dir_create | Create directory | -| rag_query | Query vector DB | -| rag_ingest | Ingest file/directory | -| rag_collections | List collections | -| metrics_record | Record event | -| metrics_query | Query events | -| lang_detect | Detect file language | -| lang_list | List supported languages | - -### Example .mcp.json -```json -{ - "core-cli": { - "command": "core", - "args": ["mcp", "serve"] - } -} -``` -``` - -**Step 3: Commit documentation** - -```bash -git add ~/.claude/projects/-home-claude/memory/*.md -git commit -m "docs: update memory with MCP server tools" -``` - ---- - -## Summary - -| Task | Files | Purpose | -|------|-------|---------| -| 1 | `pkg/mcp/tools_rag.go` | RAG tools (query, ingest, collections) | -| 2 | `pkg/mcp/tools_metrics.go` | Metrics tools (record, query) | -| 3 | `internal/cmd/mcpcmd/cmd_mcp.go` | `core mcp serve` command | -| 4 | `plugins/agentic-flows/.mcp.json` | Plugin MCP configuration | -| 5 | Memory docs | Documentation updates | - -## Services Required - -- **Qdrant:** localhost:6333 (verified running) -- **Ollama:** localhost:11434 with nomic-embed-text (verified running) -- **InfluxDB:** localhost:8086 (optional, for future time-series metrics) diff --git a/docs/plans/completed/2026-02-17-lem-chat-design.md b/docs/plans/completed/2026-02-17-lem-chat-design.md deleted file mode 100644 index 3ff9f36..0000000 --- a/docs/plans/completed/2026-02-17-lem-chat-design.md +++ /dev/null @@ -1,82 +0,0 @@ -# LEM Chat — Web Components Design - -**Date**: 2026-02-17 -**Status**: Approved - -## Summary - -Standalone chat UI built with vanilla Web Components (Custom Elements + Shadow DOM). Connects to the MLX inference server's OpenAI-compatible SSE streaming endpoint. Zero framework dependencies. Single JS file output, embeddable anywhere. - -## Components - -| Element | Purpose | -|---------|---------| -| `` | Container. Conversation state, SSE connection, config via attributes | -| `` | Scrollable message list with auto-scroll anchoring | -| `` | Single message bubble. Streams tokens for assistant messages | -| `` | Text input, Enter to send, Shift+Enter for newline | - -## Data Flow - -``` -User types in - → dispatches 'lem-send' CustomEvent - → catches it - → adds user message to - → POST /v1/chat/completions {stream: true, messages: [...history]} - → reads SSE chunks via fetch + ReadableStream - → appends tokens to streaming - → on [DONE], finalises message -``` - -## Configuration - -```html - -``` - -Attributes: `endpoint`, `model`, `system-prompt`, `max-tokens`, `temperature` - -## Theming - -Shadow DOM with CSS custom properties: - -```css ---lem-bg: #1a1a1e; ---lem-msg-user: #2a2a3e; ---lem-msg-assistant: #1e1e2a; ---lem-accent: #5865f2; ---lem-text: #e0e0e0; ---lem-font: system-ui; -``` - -## Markdown - -Minimal inline parsing: fenced code blocks, inline code, bold, italic. No library. - -## File Structure - -``` -lem-chat/ -├── index.html # Demo page -├── src/ -│ ├── lem-chat.ts # Main container + SSE client -│ ├── lem-messages.ts # Message list with scroll anchoring -│ ├── lem-message.ts # Single message with streaming -│ ├── lem-input.ts # Text input -│ ├── markdown.ts # Minimal markdown → HTML -│ └── styles.ts # CSS template literals -├── package.json # typescript + esbuild -└── tsconfig.json -``` - -Build: `esbuild src/lem-chat.ts --bundle --outfile=dist/lem-chat.js` - -## Not in v1 - -- Model selection UI -- Conversation persistence -- File/image upload -- Syntax highlighting -- Typing indicators -- User avatars diff --git a/docs/plans/completed/2026-02-20-go-api-design-original.md b/docs/plans/completed/2026-02-20-go-api-design-original.md deleted file mode 100644 index c979f81..0000000 --- a/docs/plans/completed/2026-02-20-go-api-design-original.md +++ /dev/null @@ -1,657 +0,0 @@ -# go-api Design — HTTP Gateway + OpenAPI SDK Generation - -**Date:** 2026-02-20 -**Author:** Virgil -**Status:** Phase 1 + Phase 2 + Phase 3 Complete (176 tests in go-api) -**Module:** `forge.lthn.ai/core/go-api` - -## Problem - -The Core Go ecosystem exposes 42+ tools via MCP (JSON-RPC), which is ideal for AI agents but inaccessible to regular HTTP clients, frontend applications, and third-party integrators. There is no unified HTTP gateway, no OpenAPI specification, and no generated SDKs. - -Both external customers (Host UK products) and Lethean network peers need programmatic access to the same services. The gateway also serves web routes, static assets, and streaming endpoints — not just REST APIs. - -## Solution - -A `go-api` package that acts as the central HTTP gateway: - -1. **Gin-based HTTP gateway** with extensible middleware via gin-contrib plugins -2. **RouteGroup interface** that subsystems implement to register their own endpoints (API, web, or both) -3. **WebSocket + SSE integration** for real-time streaming -4. **OpenAPI 3.1 spec generation** via runtime SpecBuilder (not swaggo annotations) -5. **SDK generation pipeline** targeting 11 languages via openapi-generator-cli - -## Architecture - -### Four-Protocol Access - -Same backend services, four client protocols: - -``` - ┌─── REST (go-api) POST /v1/ml/generate → JSON - │ - ├─── GraphQL (gqlgen) mutation { mlGenerate(...) { response } } -Client ────────────┤ - ├─── WebSocket (go-ws) subscribe ml.generate → streaming - │ - └─── MCP (go-ai) ml_generate → JSON-RPC -``` - -### Dependency Graph - -``` -go-api (Gin engine + middleware + OpenAPI) - ↑ imported by (each registers its own routes) - ├── go-ai/api/ → /v1/file/*, /v1/process/*, /v1/metrics/* - ├── go-ml/api/ → /v1/ml/* - ├── go-rag/api/ → /v1/rag/* - ├── go-agentic/api/ → /v1/tasks/* - ├── go-help/api/ → /v1/help/* - └── go-ws/api/ → /ws (WebSocket upgrade) -``` - -go-api has zero internal ecosystem dependencies. Subsystems import go-api, not the other way round. - -### Subsystem Opt-In - -Not every MCP tool becomes a REST endpoint. Each subsystem decides what to expose via a separate `RegisterAPI()` method, independent of MCP's `RegisterTools()`. A subsystem with 15 MCP tools might expose 5 REST endpoints. - -## Package Structure - -``` -forge.lthn.ai/core/go-api -├── api.go # Engine struct, New(), Serve(), Shutdown() -├── middleware.go # Auth, CORS, rate limiting, request logging, recovery -├── options.go # WithAddr, WithAuth, WithCORS, WithRateLimit, etc. -├── group.go # RouteGroup interface + registration -├── response.go # Envelope type, error responses, pagination -├── docs/ # Generated swagger docs (swaggo output) -├── sdk/ # SDK generation tooling / Makefile targets -└── go.mod # forge.lthn.ai/core/go-api -``` - -## Core Interface - -```go -// RouteGroup registers API routes onto a Gin router group. -// Subsystems implement this to expose their endpoints. -type RouteGroup interface { - // Name returns the route group identifier (e.g. "ml", "rag", "tasks") - Name() string - // BasePath returns the URL prefix (e.g. "/v1/ml") - BasePath() string - // RegisterRoutes adds handlers to the provided router group - RegisterRoutes(rg *gin.RouterGroup) -} - -// StreamGroup optionally declares WebSocket channels a subsystem publishes to. -type StreamGroup interface { - Channels() []string -} -``` - -### Subsystem Example (go-ml) - -```go -// In go-ml/api/routes.go -package api - -type Routes struct { - service *ml.Service -} - -func NewRoutes(svc *ml.Service) *Routes { - return &Routes{service: svc} -} - -func (r *Routes) Name() string { return "ml" } -func (r *Routes) BasePath() string { return "/v1/ml" } - -func (r *Routes) RegisterRoutes(rg *gin.RouterGroup) { - rg.POST("/generate", r.Generate) - rg.POST("/score", r.Score) - rg.GET("/backends", r.Backends) - rg.GET("/status", r.Status) -} - -func (r *Routes) Channels() []string { - return []string{"ml.generate", "ml.status"} -} - -// @Summary Generate text via ML backend -// @Tags ml -// @Accept json -// @Produce json -// @Param input body MLGenerateInput true "Generation parameters" -// @Success 200 {object} Response[MLGenerateOutput] -// @Router /v1/ml/generate [post] -func (r *Routes) Generate(c *gin.Context) { - var input MLGenerateInput - if err := c.ShouldBindJSON(&input); err != nil { - c.JSON(400, api.Fail("invalid_input", err.Error())) - return - } - result, err := r.service.Generate(c.Request.Context(), input.Backend, input.Prompt, ml.GenOpts{ - Temperature: input.Temperature, - MaxTokens: input.MaxTokens, - Model: input.Model, - }) - if err != nil { - c.JSON(500, api.Fail("ml.generate_failed", err.Error())) - return - } - c.JSON(200, api.OK(MLGenerateOutput{ - Response: result, - Backend: input.Backend, - Model: input.Model, - })) -} -``` - -### Engine Wiring (in core CLI) - -```go -engine := api.New( - api.WithAddr(":8080"), - api.WithCORS("*"), - api.WithAuth(api.BearerToken(cfg.APIKey)), - api.WithRateLimit(100, time.Minute), - api.WithWSHub(wsHub), -) - -engine.Register(mlapi.NewRoutes(mlService)) -engine.Register(ragapi.NewRoutes(ragService)) -engine.Register(agenticapi.NewRoutes(agenticService)) - -engine.Serve(ctx) // Blocks until context cancelled -``` - -## Response Envelope - -All endpoints return a consistent envelope: - -```go -type Response[T any] struct { - Success bool `json:"success"` - Data T `json:"data,omitempty"` - Error *Error `json:"error,omitempty"` - Meta *Meta `json:"meta,omitempty"` -} - -type Error struct { - Code string `json:"code"` - Message string `json:"message"` - Details any `json:"details,omitempty"` -} - -type Meta struct { - RequestID string `json:"request_id"` - Duration string `json:"duration"` - Page int `json:"page,omitempty"` - PerPage int `json:"per_page,omitempty"` - Total int `json:"total,omitempty"` -} -``` - -Helper functions: - -```go -func OK[T any](data T) Response[T] -func Fail(code, message string) Response[any] -func Paginated[T any](data T, page, perPage, total int) Response[T] -``` - -## Middleware Stack - -```go -api.New( - api.WithAddr(":8080"), - api.WithCORS(api.CORSConfig{...}), // gin-contrib/cors - api.WithAuth(api.BearerToken("...")), // Phase 1: simple bearer token - api.WithRateLimit(100, time.Minute), // Per-IP sliding window - api.WithRequestID(), // X-Request-ID header generation - api.WithRecovery(), // Panic recovery → 500 response - api.WithLogger(slog.Default()), // Structured request logging -) -``` - -Auth evolution path: bearer token → API keys → Authentik (OIDC/forward auth). Middleware slot stays the same. - -## WebSocket Integration - -go-api wraps the existing go-ws Hub as a first-class transport: - -```go -// Automatic registration: -// GET /ws → WebSocket upgrade (go-ws Hub) - -// Client subscribes: {"type":"subscribe","channel":"ml.generate"} -// Events arrive: {"type":"event","channel":"ml.generate","data":{...}} -// Client unsubscribes: {"type":"unsubscribe","channel":"ml.generate"} -``` - -Subsystems implementing `StreamGroup` declare which channels they publish to. This metadata feeds into the OpenAPI spec as documentation. - -## OpenAPI + SDK Generation - -### Runtime Spec Generation (SpecBuilder) - -swaggo annotations were rejected because routes are dynamic via RouteGroup, Response[T] generics break swaggo, and MCP tools already carry JSON Schema at runtime. Instead, a `SpecBuilder` constructs the full OpenAPI 3.1 spec from registered RouteGroups at runtime. - -```go -// Groups that implement DescribableGroup contribute endpoint metadata -type DescribableGroup interface { - RouteGroup - Describe() []RouteDescription -} - -// SpecBuilder assembles the spec from all groups -builder := &api.SpecBuilder{Title: "Core API", Description: "...", Version: "1.0.0"} -spec, _ := builder.Build(engine.Groups()) -``` - -### MCP-to-REST Bridge (ToolBridge) - -The `ToolBridge` converts MCP tool descriptors into REST POST endpoints and implements both `RouteGroup` and `DescribableGroup`. Each tool becomes `POST /{tool_name}`. Generic types are captured at MCP registration time via closures, enabling JSON unmarshalling to the correct input type at request time. - -```go -bridge := api.NewToolBridge("/v1/tools") -mcp.BridgeToAPI(mcpService, bridge) // Populates bridge from MCP tool registry -engine.Register(bridge) // Registers REST endpoints + OpenAPI metadata -``` - -### Swagger UI - -```go -// Built-in at GET /swagger/*any -// SpecBuilder output served via gin-swagger, cached via sync.Once -api.New(api.WithSwagger("Core API", "...", "1.0.0")) -``` - -### SDK Generation - -```bash -# Via openapi-generator-cli (11 languages supported) -core api sdk --lang go # Generate Go SDK -core api sdk --lang typescript-fetch,python # Multiple languages -core api sdk --lang rust --output ./sdk/ # Custom output dir -``` - -### CLI Commands - -```bash -core api spec # Emit OpenAPI JSON to stdout -core api spec --format yaml # YAML variant -core api spec --output spec.json # Write to file -core api sdk --lang python # Generate Python SDK -core api sdk --lang go,rust # Multiple SDKs -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `github.com/gin-gonic/gin` | HTTP framework | -| `github.com/swaggo/gin-swagger` | Swagger UI middleware | -| `github.com/gin-contrib/cors` | CORS middleware | -| `github.com/gin-contrib/secure` | Security headers | -| `github.com/gin-contrib/sessions` | Server-side sessions | -| `github.com/gin-contrib/authz` | Casbin authorisation | -| `github.com/gin-contrib/httpsign` | HTTP signature verification | -| `github.com/gin-contrib/slog` | Structured request logging | -| `github.com/gin-contrib/timeout` | Per-request timeouts | -| `github.com/gin-contrib/gzip` | Gzip compression | -| `github.com/gin-contrib/static` | Static file serving | -| `github.com/gin-contrib/pprof` | Runtime profiling | -| `github.com/gin-contrib/expvar` | Runtime metrics | -| `github.com/gin-contrib/location/v2` | Reverse proxy detection | -| `github.com/99designs/gqlgen` | GraphQL endpoint | -| `go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin` | Distributed tracing | -| `gopkg.in/yaml.v3` | YAML spec export | -| `forge.lthn.ai/core/go-ws` | WebSocket Hub (existing) | - -## Estimated Size - -| Component | LOC | -|-----------|-----| -| Engine + options | ~200 | -| Middleware | ~150 | -| Response envelope | ~80 | -| RouteGroup interface | ~30 | -| WebSocket integration | ~60 | -| Tests | ~300 | -| **Total go-api** | **~820** | - -Each subsystem's `api/` package adds ~100-200 LOC per route group. - -## Phase 1 — Implemented (20 Feb 2026) - -**Commit:** `17ae945` on Forge (`core/go-api`) - -| Component | Status | Tests | -|-----------|--------|-------| -| Response envelope (OK, Fail, Paginated) | Done | 9 | -| RouteGroup + StreamGroup interfaces | Done | 4 | -| Engine (New, Register, Handler, Serve) | Done | 9 | -| Bearer auth middleware | Done | 3 | -| Request ID middleware | Done | 2 | -| CORS middleware (gin-contrib/cors) | Done | 3 | -| WebSocket endpoint | Done | 3 | -| Swagger UI (gin-swagger) | Done | 2 | -| Health endpoint | Done | 1 | -| **Total** | **~840 LOC** | **36** | - -**Integration proof:** go-ml/api/ registers 3 endpoints with 12 tests (`0c23858`). - -## Phase 2 Wave 1 — Implemented (20 Feb 2026) - -**Commits:** `6bb7195..daae6f7` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | -|-----------|--------|------------|-------| -| Authentik (forward auth + OIDC) | `WithAuthentik()` | `go-oidc/v3`, `oauth2` | 14 | -| Security headers (HSTS, CSP, etc.) | `WithSecure()` | `gin-contrib/secure` | 8 | -| Structured request logging | `WithSlog()` | `gin-contrib/slog` | 6 | -| Per-request timeouts | `WithTimeout()` | `gin-contrib/timeout` | 5 | -| Gzip compression | `WithGzip()` | `gin-contrib/gzip` | 5 | -| Static file serving | `WithStatic()` | `gin-contrib/static` | 5 | -| **Wave 1 Total** | | | **43** | - -**Cumulative:** 76 tests (36 Phase 1 + 43 Wave 1 - 3 shared), all passing. - -## Phase 2 Wave 2 — Implemented (20 Feb 2026) - -**Commits:** `64a8b16..67dcc83` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| Brotli compression | `WithBrotli()` | `andybalholm/brotli` | 5 | Custom middleware; `gin-contrib/brotli` is empty stub | -| Response caching | `WithCache()` | none (in-memory) | 5 | Custom middleware; `gin-contrib/cache` is per-handler, not global | -| Server-side sessions | `WithSessions()` | `gin-contrib/sessions` | 5 | Cookie store, configurable name + secret | -| Casbin authorisation | `WithAuthz()` | `gin-contrib/authz`, `casbin/v2` | 5 | Subject via Basic Auth; RBAC policy model | -| **Wave 2 Total** | | | **20** | | - -**Cumulative:** 102 passing tests (2 integration skipped), all green. - -## Phase 2 Wave 3 — Implemented (20 Feb 2026) - -**Commits:** `7b3f99e..d517fa2` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| HTTP signature verification | `WithHTTPSign()` | `gin-contrib/httpsign` | 5 | HMAC-SHA256; extensible via httpsign.Option | -| Server-Sent Events | `WithSSE()` | none (custom SSEBroker) | 6 | Channel filtering, multi-client broadcast, GET /events | -| Reverse proxy detection | `WithLocation()` | `gin-contrib/location/v2` | 5 | X-Forwarded-Host/Proto parsing | -| Locale detection | `WithI18n()` | `golang.org/x/text/language` | 5 | Accept-Language parsing, message lookup, GetLocale/GetMessage | -| GraphQL endpoint | `WithGraphQL()` | `99designs/gqlgen` | 5 | /graphql + optional /graphql/playground | -| **Wave 3 Total** | | | **26** | | - -**Cumulative:** 128 passing tests (2 integration skipped), all green. - -## Phase 2 Wave 4 — Implemented (21 Feb 2026) - -**Commits:** `32b3680..8ba1716` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| Runtime profiling | `WithPprof()` | `gin-contrib/pprof` | 5 | /debug/pprof/* endpoints, flag-based mount | -| Runtime metrics | `WithExpvar()` | `gin-contrib/expvar` | 5 | /debug/vars endpoint, flag-based mount | -| Distributed tracing | `WithTracing()` | `otelgin` + OpenTelemetry SDK | 5 | W3C traceparent propagation, span attributes | -| **Wave 4 Total** | | | **15** | | - -**Cumulative:** 143 passing tests (2 integration skipped), all green. - -**Phase 2 complete.** All 4 waves implemented. Every planned plugin has a `With*()` option and tests. - -## Phase 3 — OpenAPI Spec Generation + SDK Codegen (21 Feb 2026) - -**Architecture:** Runtime OpenAPI generation via SpecBuilder (NOT swaggo annotations). Routes are dynamic via RouteGroup, Response[T] generics break swaggo, and MCP tools carry JSON Schema at runtime. A `ToolBridge` converts tool descriptors into RouteGroup + OpenAPI metadata. A `SpecBuilder` constructs the full OpenAPI 3.1 spec. SDK codegen wraps `openapi-generator-cli`. - -### Wave 1: go-api (Tasks 1-5) - -**Commits:** `465bd60..1910aec` on Forge (`core/go-api`) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| DescribableGroup interface | `group.go` | 5 | Opt-in OpenAPI metadata for RouteGroups | -| ToolBridge | `bridge.go` | 6 | Tool descriptors → POST endpoints + DescribableGroup | -| SpecBuilder | `openapi.go` | 6 | OpenAPI 3.1 JSON with Response[T] envelope wrapping | -| Swagger refactor | `swagger.go` | 5 | Replaced hardcoded empty spec with SpecBuilder | -| Spec export | `export.go` | 5 | JSON + YAML export to file/writer | -| SDK codegen | `codegen.go` | 5 | 11-language wrapper for openapi-generator-cli | -| **Wave 1 Total** | | **32** | | - -### Wave 2: go-ai MCP bridge (Tasks 6-7) - -**Commits:** `2107eda..c37e1cf` on Forge (`core/go-ai`) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| Tool registry | `mcp/registry.go` | 5 | Generic `addToolRecorded[In,Out]` captures types in closures | -| BridgeToAPI | `mcp/bridge.go` | 5 | MCP tools → go-api ToolBridge, 10MB body limit, error classification | -| **Wave 2 Total** | | **10** | | - -### Wave 3: CLI commands (Tasks 8-9) - -**Commit:** `d6eec4d` on Forge (`core/cli` dev branch) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| `core api spec` | `cmd/api/cmd_spec.go` | 2 | JSON/YAML export, --output/--format flags | -| `core api sdk` | `cmd/api/cmd_sdk.go` | 2 | --lang (required), --output, --spec, --package flags | -| **Wave 3 Total** | | **4** | | - -**Cumulative go-api:** 176 passing tests. **Phase 3 complete.** - -### Known Limitations - -- **Subsystem tools excluded from bridge:** Subsystems call `mcp.AddTool` directly, bypassing `addToolRecorded`. Only the 10 built-in MCP tools appear in the REST bridge. Future: pass `*Service` to `RegisterTools` instead of `*mcp.Server`. -- **Flat schema only:** `structSchema` reflection handles flat structs but does not recurse into nested structs. Adequate for current tool inputs. -- **CLI spec produces empty bridge:** `core api spec` currently generates a spec with only `/health`. Full MCP integration requires wiring the MCP service into the CLI command. - -## Phase 2 — Gin Plugin Roadmap (Complete) - -All plugins drop in as `With*()` options on the Engine. No architecture changes needed. - -### Security & Auth - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~**Authentik**~~ | ~~`WithAuthentik()`~~ | ~~OIDC + forward auth integration.~~ | ~~**Done**~~ | -| ~~gin-contrib/secure~~ | ~~`WithSecure()`~~ | ~~Security headers: HSTS, X-Frame-Options, X-Content-Type-Options, CSP.~~ | ~~**Done**~~ | -| ~~gin-contrib/sessions~~ | ~~`WithSessions()`~~ | ~~Server-side sessions (cookie store). Web session management alongside Authentik tokens.~~ | ~~**Done**~~ | -| ~~gin-contrib/authz~~ | ~~`WithAuthz()`~~ | ~~Casbin-based authorisation. Policy-driven access control via RBAC.~~ | ~~**Done**~~ | -| ~~gin-contrib/httpsign~~ | ~~`WithHTTPSign()`~~ | ~~HTTP signature verification. HMAC-SHA256 with extensible options.~~ | ~~**Done**~~ | - -### Performance & Reliability - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/cache~~ | ~~`WithCache()`~~ | ~~Response caching (in-memory). GET response caching with TTL, lazy eviction.~~ | ~~**Done**~~ | -| ~~gin-contrib/timeout~~ | ~~`WithTimeout()`~~ | ~~Per-request timeouts.~~ | ~~**Done**~~ | -| ~~gin-contrib/gzip~~ | ~~`WithGzip()`~~ | ~~Gzip response compression.~~ | ~~**Done**~~ | -| ~~gin-contrib/brotli~~ | ~~`WithBrotli()`~~ | ~~Brotli compression via `andybalholm/brotli`. Custom middleware (gin-contrib stub empty).~~ | ~~**Done**~~ | - -### Observability - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/slog~~ | ~~`WithSlog()`~~ | ~~Structured request logging via slog.~~ | ~~**Done**~~ | -| ~~gin-contrib/pprof~~ | ~~`WithPprof()`~~ | ~~Runtime profiling endpoints at /debug/pprof/. Flag-based mount.~~ | ~~**Done**~~ | -| ~~gin-contrib/expvar~~ | ~~`WithExpvar()`~~ | ~~Go runtime metrics at /debug/vars. Flag-based mount.~~ | ~~**Done**~~ | -| ~~otelgin~~ | ~~`WithTracing()`~~ | ~~OpenTelemetry distributed tracing. W3C traceparent propagation.~~ | ~~**Done**~~ | - -### Content & Streaming - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/static~~ | ~~`WithStatic()`~~ | ~~Serve static files.~~ | ~~**Done**~~ | -| ~~gin-contrib/sse~~ | ~~`WithSSE()`~~ | ~~Server-Sent Events. Custom SSEBroker with channel filtering, GET /events.~~ | ~~**Done**~~ | -| ~~gin-contrib/location~~ | ~~`WithLocation()`~~ | ~~Auto-detect scheme/host from X-Forwarded-* headers.~~ | ~~**Done**~~ | - -### Query Layer - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~99designs/gqlgen~~ | ~~`WithGraphQL()`~~ | ~~GraphQL endpoint at `/graphql` + optional playground. Accepts gqlgen ExecutableSchema.~~ | ~~**Done**~~ | - -The GraphQL schema can be generated from the same Go Input/Output structs that define the REST endpoints. gqlgen produces an `http.Handler` that mounts directly on Gin. Subsystems opt-in via: - -```go -// Subsystems that want GraphQL implement this alongside RouteGroup -type ResolverGroup interface { - // RegisterResolvers adds query/mutation resolvers to the GraphQL schema - RegisterResolvers(schema *graphql.Schema) -} -``` - -This means a subsystem like go-ml exposes: -- **REST:** `POST /v1/ml/generate` (existing) -- **GraphQL:** `mutation { mlGenerate(prompt: "...", backend: "mlx") { response, model } }` (same handler) -- **MCP:** `ml_generate` tool (existing) - -Four protocols, one set of handlers. - -### Ecosystem Integration - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/i18n~~ | ~~`WithI18n()`~~ | ~~Locale detection via Accept-Language. Custom middleware using `golang.org/x/text/language`.~~ | ~~**Done**~~ | -| [gin-contrib/graceful](https://github.com/gin-contrib/graceful) | — | Already implemented in Engine.Serve(). Could swap to this for more robust lifecycle management if needed. | — | -| [gin-contrib/requestid](https://github.com/gin-contrib/requestid) | — | Already implemented. Theirs uses UUID, ours uses hex. Could swap for standards compliance. | — | - -### Implementation Order - -**Wave 1 (gateway hardening):** ~~Authentik, secure, slog, timeout, gzip, static~~ **DONE** (20 Feb 2026) -**Wave 2 (performance + auth):** ~~cache, sessions, authz, brotli~~ **DONE** (20 Feb 2026) -**Wave 3 (network + streaming):** ~~httpsign, sse, location, i18n, gqlgen~~ **DONE** (20 Feb 2026) -**Wave 4 (observability):** ~~pprof, expvar, tracing~~ **DONE** (21 Feb 2026) - -Each wave adds `With*()` options + tests. No breaking changes — existing code continues to work without any new options enabled. - -## Authentik Integration - -[Authentik](https://goauthentik.io/) is the identity provider and edge auth proxy. It handles user registration, login, MFA, social auth, SAML, and OIDC — so go-api doesn't have to. - -### Two Integration Modes - -**1. Forward Auth (web traffic)** - -Traefik sits in front of go-api. For web routes, Traefik's `forwardAuth` middleware checks with Authentik before passing the request through. Authentik handles login flows, session cookies, and consent. go-api receives pre-authenticated requests with identity headers. - -``` -Browser → Traefik → Authentik (forward auth) → go-api - ↓ - Login page (if unauthenticated) -``` - -go-api reads trusted headers set by Authentik: -``` -X-Authentik-Username: alice -X-Authentik-Groups: admins,developers -X-Authentik-Email: alice@example.com -X-Authentik-Uid: -X-Authentik-Jwt: -``` - -**2. OIDC Token Validation (API traffic)** - -API clients (SDKs, CLI tools, network peers) authenticate directly with Authentik's OAuth2 token endpoint, then send the JWT to go-api. go-api validates the JWT using Authentik's OIDC discovery endpoint (`.well-known/openid-configuration`). - -``` -SDK client → Authentik (token endpoint) → receives JWT -SDK client → go-api (Authorization: Bearer ) → validates via OIDC -``` - -### Implementation in go-api - -```go -engine := api.New( - api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.lthn.ai/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, // Trust X-Authentik-* headers from Traefik - }), -) -``` - -`WithAuthentik()` adds middleware that: -1. Checks for `X-Authentik-Jwt` header (forward auth mode) — validates signature, extracts claims -2. Falls back to `Authorization: Bearer ` header (direct OIDC mode) — validates via JWKS -3. Populates `c.Set("user", AuthentikUser{...})` in the Gin context for handlers to use -4. Skips /health, /swagger, and any public paths - -```go -// In any handler: -func (r *Routes) ListItems(c *gin.Context) { - user := api.GetUser(c) // Returns *AuthentikUser or nil - if user == nil { - c.JSON(401, api.Fail("unauthorised", "Authentication required")) - return - } - // user.Username, user.Groups, user.Email, user.UID available -} -``` - -### Auth Layers - -``` -Authentik (identity) → WHO is this? (user, groups, email) - ↓ -go-api middleware → IS their token valid? (JWT verification) - ↓ -Casbin authz (optional) → CAN they do this? (role → endpoint policies) - ↓ -Handler → DOES this (business logic) -``` - -Phase 1 bearer auth continues to work alongside Authentik — useful for service-to-service tokens, CI/CD, and development. `WithBearerAuth` and `WithAuthentik` can coexist. - -### Authentik Deployment - -Authentik runs as a Docker service alongside go-api, fronted by Traefik: -- **auth.lthn.ai** — Authentik UI + OIDC endpoints (production) -- **auth.leth.in** — Authentik for devnet/testnet -- Traefik routes `/outpost.goauthentik.io/` to Authentik's embedded outpost for forward auth - -### Dependencies - -| Package | Purpose | -|---------|---------| -| `github.com/coreos/go-oidc/v3` | OIDC discovery + JWT validation | -| `golang.org/x/oauth2` | OAuth2 token exchange (for server-side flows) | - -Both are standard Go libraries with no heavy dependencies. - -## Non-Goals - -- gRPC gateway -- Built-in user registration/login (Authentik handles this) -- API versioning beyond /v1/ prefix - -## Success Criteria - -### Phase 1 (Done) - -1. ~~`core api serve` starts a Gin server with registered subsystem routes~~ -2. ~~WebSocket subscriptions work alongside REST~~ -3. ~~Swagger UI accessible at `/swagger/`~~ -4. ~~All endpoints return consistent Response envelope~~ -5. ~~Bearer token auth protects all routes~~ -6. ~~First subsystem integration (go-ml/api/) proves the pattern~~ - -### Phase 2 (Done) - -7. ~~Security headers, compression, and caching active in production~~ -8. ~~Session-based auth alongside bearer tokens~~ -9. ~~HTTP signature verification for Lethean network peers~~ -10. ~~Static file serving for docs site and SDK downloads~~ -11. ~~GraphQL endpoint at `/graphql` with playground~~ - -### Phase 3 (Done) - -12. ~~`core api spec` emits valid OpenAPI 3.1 JSON via runtime SpecBuilder~~ -13. ~~`core api sdk` generates SDKs for 11 languages via openapi-generator-cli~~ -14. ~~MCP tools bridged to REST endpoints via ToolBridge + BridgeToAPI~~ -15. ~~OpenAPI spec includes Response[T] envelope wrapping~~ -16. ~~Spec export to file in JSON and YAML formats~~ diff --git a/docs/plans/completed/2026-02-20-go-api-plan-original.md b/docs/plans/completed/2026-02-20-go-api-plan-original.md deleted file mode 100644 index 11d164d..0000000 --- a/docs/plans/completed/2026-02-20-go-api-plan-original.md +++ /dev/null @@ -1,1503 +0,0 @@ -# go-api Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build `forge.lthn.ai/core/go-api`, a Gin-based REST framework with OpenAPI generation that subsystems plug into via a RouteGroup interface. - -**Architecture:** go-api provides the HTTP engine, middleware stack, response envelope, and OpenAPI tooling. Each ecosystem package (go-ml, go-rag, etc.) imports go-api and registers its own route group. WebSocket support via go-ws Hub runs alongside REST. - -**Tech Stack:** Go 1.25, Gin, swaggo/swag, gin-swagger, gin-contrib/cors, go-ws - -**Design doc:** `docs/plans/2026-02-20-go-api-design.md` - -**Repo location:** `/Users/snider/Code/go-api` (module: `forge.lthn.ai/core/go-api`) - -**Licence:** EUPL-1.2 - -**Convention:** UK English in comments and user-facing strings. Test naming: `_Good`, `_Bad`, `_Ugly`. - ---- - -### Task 1: Scaffold Repository - -**Files:** -- Create: `/Users/snider/Code/go-api/go.mod` -- Create: `/Users/snider/Code/go-api/response.go` -- Create: `/Users/snider/Code/go-api/response_test.go` -- Create: `/Users/snider/Code/go-api/LICENCE` - -**Step 1: Create repo and go.mod** - -```bash -mkdir -p /Users/snider/Code/go-api -cd /Users/snider/Code/go-api -git init -``` - -Create `go.mod`: -``` -module forge.lthn.ai/core/go-api - -go 1.25.5 - -require github.com/gin-gonic/gin v1.10.0 -``` - -Then run: -```bash -go mod tidy -``` - -**Step 2: Create LICENCE file** - -Copy the EUPL-1.2 licence text. Use the same LICENCE file as other ecosystem repos: -```bash -cp /Users/snider/Code/go-ai/LICENCE /Users/snider/Code/go-api/LICENCE -``` - -**Step 3: Commit scaffold** - -```bash -git add go.mod go.sum LICENCE -git commit -m "chore: scaffold go-api module with Gin dependency" -``` - ---- - -### Task 2: Response Envelope (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/response.go` -- Create: `/Users/snider/Code/go-api/response_test.go` - -**Step 1: Write the failing tests** - -Create `response_test.go`: -```go -package api_test - -import ( - "encoding/json" - "testing" - - api "forge.lthn.ai/core/go-api" -) - -func TestOK_Good(t *testing.T) { - type Payload struct { - Name string `json:"name"` - } - resp := api.OK(Payload{Name: "test"}) - - if !resp.Success { - t.Fatal("expected Success to be true") - } - if resp.Data.Name != "test" { - t.Fatalf("expected Data.Name = test, got %s", resp.Data.Name) - } - if resp.Error != nil { - t.Fatal("expected Error to be nil") - } -} - -func TestFail_Good(t *testing.T) { - resp := api.Fail("not_found", "Resource not found") - - if resp.Success { - t.Fatal("expected Success to be false") - } - if resp.Error == nil { - t.Fatal("expected Error to be non-nil") - } - if resp.Error.Code != "not_found" { - t.Fatalf("expected Code = not_found, got %s", resp.Error.Code) - } - if resp.Error.Message != "Resource not found" { - t.Fatalf("expected Message = Resource not found, got %s", resp.Error.Message) - } -} - -func TestFailWithDetails_Good(t *testing.T) { - details := map[string]string{"field": "email"} - resp := api.FailWithDetails("validation_error", "Invalid input", details) - - if resp.Error.Details == nil { - t.Fatal("expected Details to be non-nil") - } -} - -func TestPaginated_Good(t *testing.T) { - items := []string{"a", "b", "c"} - resp := api.Paginated(items, 1, 10, 42) - - if !resp.Success { - t.Fatal("expected Success to be true") - } - if resp.Meta == nil { - t.Fatal("expected Meta to be non-nil") - } - if resp.Meta.Page != 1 { - t.Fatalf("expected Page = 1, got %d", resp.Meta.Page) - } - if resp.Meta.PerPage != 10 { - t.Fatalf("expected PerPage = 10, got %d", resp.Meta.PerPage) - } - if resp.Meta.Total != 42 { - t.Fatalf("expected Total = 42, got %d", resp.Meta.Total) - } -} - -func TestOK_JSON_Good(t *testing.T) { - resp := api.OK("hello") - data, err := json.Marshal(resp) - if err != nil { - t.Fatalf("marshal failed: %v", err) - } - - var raw map[string]any - if err := json.Unmarshal(data, &raw); err != nil { - t.Fatalf("unmarshal failed: %v", err) - } - - if raw["success"] != true { - t.Fatal("expected success = true in JSON") - } - if raw["data"] != "hello" { - t.Fatalf("expected data = hello, got %v", raw["data"]) - } - // error and meta should be omitted - if _, ok := raw["error"]; ok { - t.Fatal("expected error to be omitted from JSON") - } - if _, ok := raw["meta"]; ok { - t.Fatal("expected meta to be omitted from JSON") - } -} - -func TestFail_JSON_Good(t *testing.T) { - resp := api.Fail("err", "msg") - data, err := json.Marshal(resp) - if err != nil { - t.Fatalf("marshal failed: %v", err) - } - - var raw map[string]any - if err := json.Unmarshal(data, &raw); err != nil { - t.Fatalf("unmarshal failed: %v", err) - } - - if raw["success"] != false { - t.Fatal("expected success = false in JSON") - } - // data should be omitted - if _, ok := raw["data"]; ok { - t.Fatal("expected data to be omitted from JSON") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.OK`, `api.Fail`, etc. not defined. - -**Step 3: Implement response.go** - -Create `response.go`: -```go -// Package api provides a Gin-based REST framework with OpenAPI generation. -// Subsystems implement RouteGroup to register their own endpoints. -package api - -// Response is the standard envelope for all API responses. -type Response[T any] struct { - Success bool `json:"success"` - Data T `json:"data,omitempty"` - Error *Error `json:"error,omitempty"` - Meta *Meta `json:"meta,omitempty"` -} - -// Error describes a failed API request. -type Error struct { - Code string `json:"code"` - Message string `json:"message"` - Details any `json:"details,omitempty"` -} - -// Meta carries pagination and request metadata. -type Meta struct { - RequestID string `json:"request_id,omitempty"` - Duration string `json:"duration,omitempty"` - Page int `json:"page,omitempty"` - PerPage int `json:"per_page,omitempty"` - Total int `json:"total,omitempty"` -} - -// OK returns a successful response wrapping data. -func OK[T any](data T) Response[T] { - return Response[T]{Success: true, Data: data} -} - -// Fail returns an error response with code and message. -func Fail(code, message string) Response[any] { - return Response[any]{ - Success: false, - Error: &Error{Code: code, Message: message}, - } -} - -// FailWithDetails returns an error response with additional detail payload. -func FailWithDetails(code, message string, details any) Response[any] { - return Response[any]{ - Success: false, - Error: &Error{Code: code, Message: message, Details: details}, - } -} - -// Paginated returns a successful response with pagination metadata. -func Paginated[T any](data T, page, perPage, total int) Response[T] { - return Response[T]{ - Success: true, - Data: data, - Meta: &Meta{Page: page, PerPage: perPage, Total: total}, - } -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add response.go response_test.go -git commit -m "feat: add response envelope with OK, Fail, Paginated helpers" -``` - ---- - -### Task 3: RouteGroup Interface - -**Files:** -- Create: `/Users/snider/Code/go-api/group.go` -- Create: `/Users/snider/Code/go-api/group_test.go` - -**Step 1: Write the failing test** - -Create `group_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -// stubGroup is a minimal RouteGroup for testing. -type stubGroup struct{} - -func (s *stubGroup) Name() string { return "stub" } -func (s *stubGroup) BasePath() string { return "/v1/stub" } - -func (s *stubGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/ping", func(c *gin.Context) { - c.JSON(200, api.OK("pong")) - }) -} - -// stubStreamGroup implements both RouteGroup and StreamGroup. -type stubStreamGroup struct { - stubGroup -} - -func (s *stubStreamGroup) Channels() []string { - return []string{"stub.events", "stub.updates"} -} - -func TestRouteGroup_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - g := gin.New() - group := &stubGroup{} - - rg := g.Group(group.BasePath()) - group.RegisterRoutes(rg) - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - g.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} - -func TestStreamGroup_Good(t *testing.T) { - group := &stubStreamGroup{} - - // Verify it satisfies StreamGroup - var sg api.StreamGroup = group - channels := sg.Channels() - - if len(channels) != 2 { - t.Fatalf("expected 2 channels, got %d", len(channels)) - } - if channels[0] != "stub.events" { - t.Fatalf("expected stub.events, got %s", channels[0]) - } -} - -func TestRouteGroupName_Good(t *testing.T) { - group := &stubGroup{} - - var rg api.RouteGroup = group - if rg.Name() != "stub" { - t.Fatalf("expected name stub, got %s", rg.Name()) - } - if rg.BasePath() != "/v1/stub" { - t.Fatalf("expected basepath /v1/stub, got %s", rg.BasePath()) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.RouteGroup`, `api.StreamGroup` not defined. - -**Step 3: Implement group.go** - -Create `group.go`: -```go -package api - -import "github.com/gin-gonic/gin" - -// RouteGroup registers API routes onto a Gin router group. -// Subsystems implement this to expose their REST endpoints. -type RouteGroup interface { - // Name returns the route group identifier (e.g. "ml", "rag", "tasks"). - Name() string - // BasePath returns the URL prefix (e.g. "/v1/ml"). - BasePath() string - // RegisterRoutes adds handlers to the provided router group. - RegisterRoutes(rg *gin.RouterGroup) -} - -// StreamGroup optionally declares WebSocket channels a subsystem publishes to. -// Subsystems implementing both RouteGroup and StreamGroup expose both REST -// endpoints and real-time event channels. -type StreamGroup interface { - // Channels returns the WebSocket channel names this group publishes to. - Channels() []string -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: All tests PASS (previous 6 + new 3). - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add group.go group_test.go -git commit -m "feat: add RouteGroup and StreamGroup interfaces" -``` - ---- - -### Task 4: Engine + Options (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/api.go` -- Create: `/Users/snider/Code/go-api/options.go` -- Create: `/Users/snider/Code/go-api/api_test.go` - -**Step 1: Write the failing tests** - -Create `api_test.go`: -```go -package api_test - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" - "time" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestNew_Good(t *testing.T) { - engine, err := api.New() - if err != nil { - t.Fatalf("New() failed: %v", err) - } - if engine == nil { - t.Fatal("expected non-nil engine") - } -} - -func TestNewWithAddr_Good(t *testing.T) { - engine, err := api.New(api.WithAddr(":9090")) - if err != nil { - t.Fatalf("New() failed: %v", err) - } - if engine.Addr() != ":9090" { - t.Fatalf("expected addr :9090, got %s", engine.Addr()) - } -} - -func TestDefaultAddr_Good(t *testing.T) { - engine, _ := api.New() - if engine.Addr() != ":8080" { - t.Fatalf("expected default addr :8080, got %s", engine.Addr()) - } -} - -func TestRegister_Good(t *testing.T) { - engine, _ := api.New() - group := &stubGroup{} - - engine.Register(group) - - if len(engine.Groups()) != 1 { - t.Fatalf("expected 1 group, got %d", len(engine.Groups())) - } - if engine.Groups()[0].Name() != "stub" { - t.Fatalf("expected group name stub, got %s", engine.Groups()[0].Name()) - } -} - -func TestRegisterMultiple_Good(t *testing.T) { - engine, _ := api.New() - engine.Register(&stubGroup{}) - engine.Register(&stubStreamGroup{}) - - if len(engine.Groups()) != 2 { - t.Fatalf("expected 2 groups, got %d", len(engine.Groups())) - } -} - -func TestHandler_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - engine.Register(&stubGroup{}) - - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp map[string]any - json.Unmarshal(w.Body.Bytes(), &resp) - if resp["success"] != true { - t.Fatal("expected success = true") - } -} - -func TestHealthEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} - -func TestServeAndShutdown_Good(t *testing.T) { - engine, _ := api.New(api.WithAddr(":0")) - engine.Register(&stubGroup{}) - - ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) - defer cancel() - - errCh := make(chan error, 1) - go func() { - errCh <- engine.Serve(ctx) - }() - - // Wait for context cancellation to trigger shutdown - <-ctx.Done() - - select { - case err := <-errCh: - if err != nil && err != http.ErrServerClosed && err != context.DeadlineExceeded { - t.Fatalf("Serve() returned unexpected error: %v", err) - } - case <-time.After(2 * time.Second): - t.Fatal("Serve() did not return after context cancellation") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.New`, `api.WithAddr`, `api.Engine` not defined. - -**Step 3: Implement options.go** - -Create `options.go`: -```go -package api - -// Option configures the Engine. -type Option func(*Engine) error - -// WithAddr sets the listen address (default ":8080"). -func WithAddr(addr string) Option { - return func(e *Engine) error { - e.addr = addr - return nil - } -} -``` - -**Step 4: Implement api.go** - -Create `api.go`: -```go -package api - -import ( - "context" - "fmt" - "log/slog" - "net/http" - - "github.com/gin-gonic/gin" -) - -// Engine is the central REST API server. -// Register RouteGroups to add endpoints, then call Serve to start. -type Engine struct { - gin *gin.Engine - addr string - groups []RouteGroup - logger *slog.Logger - built bool -} - -// New creates an Engine with the given options. -func New(opts ...Option) (*Engine, error) { - e := &Engine{ - addr: ":8080", - logger: slog.Default(), - } - - for _, opt := range opts { - if err := opt(e); err != nil { - return nil, fmt.Errorf("apply option: %w", err) - } - } - - return e, nil -} - -// Addr returns the configured listen address. -func (e *Engine) Addr() string { - return e.addr -} - -// Groups returns all registered route groups. -func (e *Engine) Groups() []RouteGroup { - return e.groups -} - -// Register adds a RouteGroup to the engine. -// Routes are mounted when Handler() or Serve() is called. -func (e *Engine) Register(group RouteGroup) { - e.groups = append(e.groups, group) - e.built = false -} - -// build constructs the Gin engine with all registered groups. -func (e *Engine) build() { - if e.built && e.gin != nil { - return - } - - e.gin = gin.New() - e.gin.Use(gin.Recovery()) - - // Health endpoint - e.gin.GET("/health", func(c *gin.Context) { - c.JSON(200, OK("healthy")) - }) - - // Mount each route group - for _, group := range e.groups { - rg := e.gin.Group(group.BasePath()) - group.RegisterRoutes(rg) - e.logger.Info("registered route group", "name", group.Name(), "path", group.BasePath()) - } - - e.built = true -} - -// Handler returns the http.Handler for testing or custom server usage. -func (e *Engine) Handler() http.Handler { - e.build() - return e.gin -} - -// Serve starts the HTTP server and blocks until the context is cancelled. -// Performs graceful shutdown on context cancellation. -func (e *Engine) Serve(ctx context.Context) error { - e.build() - - srv := &http.Server{ - Addr: e.addr, - Handler: e.gin, - } - - errCh := make(chan error, 1) - go func() { - if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed { - errCh <- err - } - close(errCh) - }() - - <-ctx.Done() - - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5_000_000_000) // 5s - defer cancel() - - if err := srv.Shutdown(shutdownCtx); err != nil { - return fmt.Errorf("shutdown: %w", err) - } - - if err, ok := <-errCh; ok { - return err - } - - return nil -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add api.go options.go api_test.go -git commit -m "feat: add Engine with Register, Handler, Serve, and graceful shutdown" -``` - ---- - -### Task 5: Middleware (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/middleware.go` -- Create: `/Users/snider/Code/go-api/middleware_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add middleware options - -**Step 1: Write the failing tests** - -Create `middleware_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestBearerAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Request without token → 401 - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 without token, got %d", w.Code) - } - - // Request with correct token → 200 - w = httptest.NewRecorder() - req, _ = http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("Authorization", "Bearer secret-token") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 with correct token, got %d", w.Code) - } -} - -func TestBearerAuth_Bad(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Wrong token → 401 - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("Authorization", "Bearer wrong-token") - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 with wrong token, got %d", w.Code) - } -} - -func TestHealthBypassesAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - handler := engine.Handler() - - // Health endpoint should not require auth - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for /health without auth, got %d", w.Code) - } -} - -func TestRequestID_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithRequestID()) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - rid := w.Header().Get("X-Request-ID") - if rid == "" { - t.Fatal("expected X-Request-ID header to be set") - } -} - -func TestRequestIDPreserved_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithRequestID()) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("X-Request-ID", "my-custom-id") - handler.ServeHTTP(w, req) - - rid := w.Header().Get("X-Request-ID") - if rid != "my-custom-id" { - t.Fatalf("expected X-Request-ID = my-custom-id, got %s", rid) - } -} - -func TestCORS_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithCORS("https://example.com")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Preflight request - w := httptest.NewRecorder() - req, _ := http.NewRequest("OPTIONS", "/v1/stub/ping", nil) - req.Header.Set("Origin", "https://example.com") - req.Header.Set("Access-Control-Request-Method", "POST") - handler.ServeHTTP(w, req) - - origin := w.Header().Get("Access-Control-Allow-Origin") - if origin != "https://example.com" { - t.Fatalf("expected CORS origin https://example.com, got %s", origin) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `WithBearerAuth`, `WithRequestID`, `WithCORS` not defined. - -**Step 3: Implement middleware.go** - -Create `middleware.go`: -```go -package api - -import ( - "crypto/rand" - "encoding/hex" - "strings" - - "github.com/gin-gonic/gin" -) - -// bearerAuthMiddleware validates Bearer tokens. -// Skips paths listed in skip (e.g. /health, /swagger). -func bearerAuthMiddleware(token string, skip []string) gin.HandlerFunc { - return func(c *gin.Context) { - path := c.Request.URL.Path - for _, s := range skip { - if strings.HasPrefix(path, s) { - c.Next() - return - } - } - - header := c.GetHeader("Authorization") - if header == "" { - c.JSON(401, Fail("unauthorised", "Missing Authorization header")) - c.Abort() - return - } - - parts := strings.SplitN(header, " ", 2) - if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") || parts[1] != token { - c.JSON(401, Fail("unauthorised", "Invalid bearer token")) - c.Abort() - return - } - - c.Next() - } -} - -// requestIDMiddleware sets X-Request-ID on every response. -// If the client sends one, it is preserved; otherwise a random ID is generated. -func requestIDMiddleware() gin.HandlerFunc { - return func(c *gin.Context) { - rid := c.GetHeader("X-Request-ID") - if rid == "" { - b := make([]byte, 16) - rand.Read(b) - rid = hex.EncodeToString(b) - } - c.Header("X-Request-ID", rid) - c.Set("request_id", rid) - c.Next() - } -} -``` - -**Step 4: Add middleware options to options.go** - -Append to `options.go`: -```go -import "github.com/gin-contrib/cors" - -// WithBearerAuth adds bearer token authentication middleware. -// The /health and /swagger paths are excluded from authentication. -func WithBearerAuth(token string) Option { - return func(e *Engine) error { - e.middlewares = append(e.middlewares, bearerAuthMiddleware(token, []string{"/health", "/swagger"})) - return nil - } -} - -// WithRequestID adds a middleware that sets X-Request-ID on every response. -func WithRequestID() Option { - return func(e *Engine) error { - e.middlewares = append(e.middlewares, requestIDMiddleware()) - return nil - } -} - -// WithCORS configures Cross-Origin Resource Sharing. -// Pass "*" to allow all origins, or specific origins. -func WithCORS(allowOrigins ...string) Option { - return func(e *Engine) error { - config := cors.DefaultConfig() - if len(allowOrigins) == 1 && allowOrigins[0] == "*" { - config.AllowAllOrigins = true - } else { - config.AllowOrigins = allowOrigins - } - config.AllowMethods = []string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"} - config.AllowHeaders = []string{"Authorization", "Content-Type", "X-Request-ID"} - e.middlewares = append(e.middlewares, cors.New(config)) - return nil - } -} -``` - -Update `Engine` struct in `api.go` to include `middlewares []gin.HandlerFunc` field, and apply them in `build()`: -```go -// Add to Engine struct: -middlewares []gin.HandlerFunc - -// In build(), after gin.New() and gin.Recovery(), before health endpoint: -for _, mw := range e.middlewares { - e.gin.Use(mw) -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add middleware.go middleware_test.go options.go api.go -git commit -m "feat: add bearer auth, request ID, and CORS middleware" -``` - ---- - -### Task 6: WebSocket Integration (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/websocket.go` -- Create: `/Users/snider/Code/go-api/websocket_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add WithWSHub -- Modify: `/Users/snider/Code/go-api/api.go` — mount /ws route - -**Step 1: Write the failing test** - -Create `websocket_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "strings" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" - "github.com/gorilla/websocket" -) - -func TestWSEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithWSHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - upgrader := websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return true }} - conn, err := upgrader.Upgrade(w, r, nil) - if err != nil { - return - } - defer conn.Close() - conn.WriteMessage(websocket.TextMessage, []byte("hello")) - }))) - - srv := httptest.NewServer(engine.Handler()) - defer srv.Close() - - wsURL := "ws" + strings.TrimPrefix(srv.URL, "http") + "/ws" - conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil) - if err != nil { - t.Fatalf("dial failed: %v", err) - } - defer conn.Close() - - _, msg, err := conn.ReadMessage() - if err != nil { - t.Fatalf("read failed: %v", err) - } - if string(msg) != "hello" { - t.Fatalf("expected hello, got %s", string(msg)) - } -} - -func TestNoWSHandler_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - // /ws should 404 when no handler configured - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/ws", nil) - handler.ServeHTTP(w, req) - - if w.Code != 404 { - t.Fatalf("expected 404 without WS handler, got %d", w.Code) - } -} - -func TestChannelListing_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - engine.Register(&stubStreamGroup{}) - - channels := engine.Channels() - if len(channels) != 2 { - t.Fatalf("expected 2 channels, got %d", len(channels)) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors. - -**Step 3: Implement websocket.go + option + engine changes** - -Create `websocket.go`: -```go -package api - -import ( - "net/http" - - "github.com/gin-gonic/gin" -) - -// wrapWSHandler adapts a standard http.Handler to a Gin handler for the /ws route. -func wrapWSHandler(h http.Handler) gin.HandlerFunc { - return func(c *gin.Context) { - h.ServeHTTP(c.Writer, c.Request) - } -} -``` - -Add to `options.go`: -```go -// WithWSHandler registers a WebSocket handler at GET /ws. -// Typically this wraps a go-ws Hub.Handler(). -func WithWSHandler(h http.Handler) Option { - return func(e *Engine) error { - e.wsHandler = h - return nil - } -} -``` - -Add to `Engine` struct in `api.go`: -```go -wsHandler http.Handler -``` - -Add to `build()` after mounting route groups: -```go -// WebSocket endpoint -if e.wsHandler != nil { - e.gin.GET("/ws", wrapWSHandler(e.wsHandler)) -} -``` - -Add `Channels()` method to `Engine`: -```go -// Channels returns all WebSocket channel names from registered StreamGroups. -func (e *Engine) Channels() []string { - var channels []string - for _, g := range e.groups { - if sg, ok := g.(StreamGroup); ok { - channels = append(channels, sg.Channels()...) - } - } - return channels -} -``` - -**Step 4: Run go mod tidy to pick up gorilla/websocket** - -```bash -cd /Users/snider/Code/go-api -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add websocket.go websocket_test.go options.go api.go go.mod go.sum -git commit -m "feat: add WebSocket endpoint and channel listing from StreamGroups" -``` - ---- - -### Task 7: Swagger/OpenAPI Integration - -**Files:** -- Create: `/Users/snider/Code/go-api/swagger.go` -- Create: `/Users/snider/Code/go-api/swagger_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add WithSwagger -- Modify: `/Users/snider/Code/go-api/api.go` — mount swagger routes - -**Step 1: Write the failing test** - -Create `swagger_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestSwaggerEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithSwagger("Core API", "REST API for the Lethean ecosystem", "0.1.0")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Swagger JSON endpoint - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/swagger/doc.json", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for swagger doc.json, got %d", w.Code) - } - - body := w.Body.String() - if len(body) == 0 { - t.Fatal("expected non-empty swagger doc") - } -} - -func TestSwaggerDisabledByDefault_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/swagger/doc.json", nil) - handler.ServeHTTP(w, req) - - if w.Code != 404 { - t.Fatalf("expected 404 when swagger disabled, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors. - -**Step 3: Implement swagger.go + option** - -Create `swagger.go`: -```go -package api - -import ( - "github.com/gin-gonic/gin" - swaggerFiles "github.com/swaggo/files" - ginSwagger "github.com/swaggo/gin-swagger" - "github.com/swaggo/swag" -) - -// swaggerSpec holds a minimal OpenAPI spec for runtime serving. -type swaggerSpec struct { - title string - description string - version string -} - -func (s *swaggerSpec) ReadDoc() string { - // Minimal OpenAPI 3.0 document — swaggo generates the full one at build time. - // This serves as the runtime fallback and base template. - return `{ - "swagger": "2.0", - "info": { - "title": "` + s.title + `", - "description": "` + s.description + `", - "version": "` + s.version + `" - }, - "basePath": "/", - "paths": {} -}` -} - -// registerSwagger mounts the swagger UI and doc.json endpoint. -func registerSwagger(g *gin.Engine, title, description, version string) { - spec := &swaggerSpec{title: title, description: description, version: version} - swag.Register(swag.Name, spec) - - g.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler)) -} -``` - -Add to `options.go`: -```go -// WithSwagger enables the Swagger UI at /swagger/. -func WithSwagger(title, description, version string) Option { - return func(e *Engine) error { - e.swaggerTitle = title - e.swaggerDesc = description - e.swaggerVersion = version - e.swaggerEnabled = true - return nil - } -} -``` - -Add fields to `Engine` struct: -```go -swaggerEnabled bool -swaggerTitle string -swaggerDesc string -swaggerVersion string -``` - -Add to `build()` after WebSocket: -```go -// Swagger UI -if e.swaggerEnabled { - registerSwagger(e.gin, e.swaggerTitle, e.swaggerDesc, e.swaggerVersion) -} -``` - -**Step 4: Run go mod tidy** - -```bash -cd /Users/snider/Code/go-api -go get github.com/swaggo/gin-swagger github.com/swaggo/files github.com/swaggo/swag -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add swagger.go swagger_test.go options.go api.go go.mod go.sum -git commit -m "feat: add Swagger UI endpoint with runtime spec serving" -``` - ---- - -### Task 8: CLAUDE.md + README.md - -**Files:** -- Create: `/Users/snider/Code/go-api/CLAUDE.md` -- Create: `/Users/snider/Code/go-api/README.md` - -**Step 1: Write CLAUDE.md** - -```markdown -# CLAUDE.md - -This file provides guidance to Claude Code when working with the go-api repository. - -## Project Overview - -**go-api** is the REST framework for the Lethean Go ecosystem. It provides a Gin-based HTTP engine with middleware, response envelopes, WebSocket integration, and OpenAPI generation. Subsystems implement the `RouteGroup` interface to register their own endpoints. - -- **Module path**: `forge.lthn.ai/core/go-api` -- **Language**: Go 1.25 -- **Licence**: EUPL-1.2 - -## Build & Test Commands - -```bash -go test ./... # Run all tests -go test -run TestName ./... # Run a single test -go test -v -race ./... # Verbose with race detector -go build ./... # Build (library — no main package) -go vet ./... # Vet -``` - -## Coding Standards - -- **UK English** in comments and user-facing strings (colour, organisation, unauthorised) -- **Conventional commits**: `type(scope): description` -- **Co-Author**: `Co-Authored-By: Virgil ` -- **Error handling**: Return wrapped errors with context, never panic -- **Test naming**: `_Good` (happy path), `_Bad` (expected errors), `_Ugly` (panics/edge cases) -- **Licence**: EUPL-1.2 -``` - -**Step 2: Write README.md** - -Brief README with quick start and links to design doc. - -**Step 3: Commit** - -```bash -cd /Users/snider/Code/go-api -git add CLAUDE.md README.md -git commit -m "docs: add CLAUDE.md and README.md" -``` - ---- - -### Task 9: Create Forge Repo + Push - -**Step 1: Create repo on Forge** - -```bash -curl -s -X POST "https://forge.lthn.ai/api/v1/orgs/core/repos" \ - -H "Authorization: token 375068d101922dd1cf269e8b8cb77a0f99d1b486" \ - -H "Content-Type: application/json" \ - -d '{"name":"go-api","description":"REST framework + OpenAPI SDK generation for the Lethean Go ecosystem","default_branch":"main","auto_init":false,"license":"EUPL-1.2"}' -``` - -**Step 2: Add remote and push** - -```bash -cd /Users/snider/Code/go-api -git remote add forge ssh://git@forge.lthn.ai:2223/core/go-api.git -git branch -M main -git push -u forge main -``` - -**Step 3: Verify on Forge** - -```bash -curl -s "https://forge.lthn.ai/api/v1/repos/core/go-api" \ - -H "Authorization: token 375068d101922dd1cf269e8b8cb77a0f99d1b486" | jq .name -``` - -Expected: `"go-api"` - ---- - -### Task 10: Integration Test — First Subsystem (go-ml/api) - -This task validates the framework by building the first real subsystem integration. It lives in go-ml, not go-api. - -**Files:** -- Create: `/Users/snider/Code/go-ml/api/routes.go` -- Create: `/Users/snider/Code/go-ml/api/routes_test.go` - -**Step 1: Write the failing test in go-ml** - -Create `api/routes_test.go` in go-ml that: -1. Creates a `Routes` with a mock `ml.Service` -2. Registers it on an `api.Engine` -3. Sends `POST /v1/ml/backends` and asserts a 200 response with the response envelope - -**Step 2: Implement api/routes.go** - -Implement `Routes` struct that wraps `*ml.Service` and exposes: -- `POST /v1/ml/generate` -- `POST /v1/ml/score` -- `GET /v1/ml/backends` -- `GET /v1/ml/status` - -Each handler uses `c.ShouldBindJSON()` for input and `api.OK()` / `api.Fail()` for responses. - -**Step 3: Run tests** - -```bash -cd /Users/snider/Code/go-ml -go test ./api/... -v -``` - -**Step 4: Commit in go-ml** - -```bash -cd /Users/snider/Code/go-ml -git add api/ -git commit -m "feat(api): add REST route group for ML endpoints via go-api" -``` - ---- - -## Dependency Summary - -``` -Task 1 (scaffold) → Task 2 (response) → Task 3 (group) → Task 4 (engine) - → Task 5 (middleware) → Task 6 (websocket) → Task 7 (swagger) - → Task 8 (docs) → Task 9 (forge) → Task 10 (integration) -``` - -All tasks are sequential — each builds on the previous. - -## Estimated Timeline - -- Tasks 1-7: Core go-api package (~820 LOC) -- Task 8: Documentation -- Task 9: Forge deployment -- Task 10: First subsystem integration proof diff --git a/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md b/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md deleted file mode 100644 index eaf886f..0000000 --- a/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md +++ /dev/null @@ -1,128 +0,0 @@ -# CLI Meta-Package Restructure — Design - -**Goal:** Transform `core/cli` from a 35K LOC monolith into a thin assembly repo that ships variant binaries. Domain repos own their commands. `go/pkg/cli` is the only import any domain package needs for CLI concerns. - -**Architecture:** Commands register as framework services via `cli.WithCommands()`, passed to `cli.Main()`. Command code lives in the domain repos that own the business logic. The cli repo is a thin `main.go` that wires them together. - -**Tech Stack:** go/pkg/cli (wraps cobra + charmbracelet), Core framework lifecycle, Taskfile - ---- - -## 1. CLI SDK — The Single Import - -`forge.lthn.ai/core/go/pkg/cli` is the **only** import domain packages use for CLI concerns. It wraps cobra, charmbracelet, and stdlib behind a stable API. If the underlying libraries change, only `go/pkg/cli` is touched — every domain repo is insulated. - -### Already done - -- **Cobra:** `Command` type alias, `NewCommand()`, `NewGroup()`, `NewRun()`, flag helpers (`StringFlag`, `BoolFlag`, `IntFlag`, `StringSliceFlag`), arg validators -- **Output:** `Success()`, `Error()`, `Warn()`, `Info()`, `Table`, `Section()`, `Label()`, `Task()`, `Hint()` -- **Prompts:** `Confirm()`, `Question()`, `Choose()`, `ChooseMulti()` with grammar-based action variants -- **Styles:** 17 pre-built styles, `AnsiStyle` builder, Tailwind colour constants (47 hex values) -- **Glyphs:** `:check:`, `:cross:`, `:warn:` etc. with Unicode/Emoji/ASCII themes -- **Layout:** HLCRF composite renderer (Header/Left/Content/Right/Footer) -- **Errors:** `Wrap()`, `WrapVerb()`, `ExitError`, `Is()`, `As()` -- **Logging:** `LogDebug()`, `LogInfo()`, `LogWarn()`, `LogError()`, `LogSecurity()` -- **TUI primitives:** `Spinner`, `ProgressBar`, `InteractiveList`, `TextInput`, `Viewport`, `RunTUI` -- **Command registration:** `WithCommands(name, fn)` — registers commands as framework services - -### Stubbed for later (interface exists, returns simple fallback) - -- `Form(fields []FormField) (map[string]string, error)` — multi-field form (backed by huh later) -- `FilePicker(opts ...FilePickerOption) (string, error)` — file browser -- `Tabs(items []TabItem) error` — tabbed content panes - -### Rule - -Domain packages import `forge.lthn.ai/core/go/pkg/cli` and **nothing else** for CLI concerns. No `cobra`, no `lipgloss`, no `bubbletea`. - ---- - -## 2. Command Registration — Framework Lifecycle - -Commands register through the Core framework's service lifecycle, not through global state or `init()` functions. - -### The contract - -Each domain repo exports an `Add*Commands(root *cli.Command)` function. The CLI binary wires it in via `cli.WithCommands()`: - -```go -// go-ai/cmd/daemon/cmd.go -package daemon - -import "forge.lthn.ai/core/go/pkg/cli" - -// AddDaemonCommand adds the 'daemon' command group to the root. -func AddDaemonCommand(root *cli.Command) { - daemonCmd := cli.NewGroup("daemon", "Manage the core daemon", "") - root.AddCommand(daemonCmd) - // subcommands... -} -``` - -No `init()`. No blank imports. No `cli.RegisterCommands()`. - -### How it works - -`cli.WithCommands(name, fn)` wraps the registration function as a framework service implementing `Startable`. During `Core.ServiceStartup()`, the service's `OnStartup()` casts `Core.App` to `*cobra.Command` and calls the registration function. Core services (i18n, log, workspace) start first since they're registered before command services. - -```go -// cli/main.go -func main() { - cli.Main( - cli.WithCommands("config", config.AddConfigCommands), - cli.WithCommands("doctor", doctor.AddDoctorCommands), - // ... - ) -} -``` - -### Migration status (completed) - -| Source | Destination | Status | -|--------|-------------|--------| -| `cmd/dev, setup, qa, docs, gitcmd, monitor` | `go-devops/cmd/` | Done | -| `cmd/lab` | `go-ai/cmd/` | Done | -| `cmd/workspace` | `go-agentic/cmd/` | Done | -| `cmd/go` | `core/go/cmd/gocmd` | Done | -| `cmd/vanity-import, community` | `go-devops/cmd/` | Done | -| `cmd/updater` | `go-update` | Done (own repo) | -| `cmd/daemon, mcpcmd, security` | `go-ai/cmd/` | Done | -| `cmd/crypt` | `go-crypt/cmd/` | Done | -| `cmd/rag` | `go-rag/cmd/` | Done | -| `cmd/unifi` | `go-netops/cmd/` | Done | -| `cmd/api` | `go-api/cmd/` | Done | -| `cmd/collect, forge, gitea` | `go-scm/cmd/` | Done | -| `cmd/deploy, prod, vm` | `go-devops/cmd/` | Done | - -### Stays in cli/ (meta/framework commands) - -`config`, `doctor`, `help`, `module`, `pkgcmd`, `plugin`, `session` - ---- - -## 3. Variant Binaries (future) - -The cli/ repo can produce variant binaries by creating multiple `main.go` files that wire different sets of commands. - -``` -cli/ -├── main.go # Current — meta commands only -├── cmd/core-full/main.go # Full CLI — all ecosystem commands -├── cmd/core-ci/main.go # CI agent dispatch + SCM -├── cmd/core-mlx/main.go # ML inference subprocess -└── cmd/core-ops/main.go # DevOps + infra management -``` - -Each variant calls `cli.Main()` with its specific `cli.WithCommands()` set. No blank imports needed. - -### Why variants matter - -- `core-mlx` ships to the homelab as a ~10MB binary, not 50MB with devops/forge/netops -- `core-ci` deploys to agent machines without ML or CGO dependencies -- Adding a new variant = one new `main.go` with the right `WithCommands` calls - ---- - -## 4. Current State - -cli/ has 7 meta packages, one `main.go`, and zero business logic. Everything else lives in the domain repos that own it. Total cli/ LOC is ~2K. diff --git a/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md b/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md deleted file mode 100644 index c2efef1..0000000 --- a/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md +++ /dev/null @@ -1,1724 +0,0 @@ -# CLI SDK Expansion (Phase 0) Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Extend `go/pkg/cli` with charmbracelet TUI primitives (Spinner, ProgressBar, List, TextInput, Viewport) so domain repos never import anything but `forge.lthn.ai/core/go/pkg/cli` for CLI concerns. - -**Architecture:** Each TUI primitive gets its own file in `pkg/cli/`. Charmbracelet libraries (bubbletea, bubbles, lipgloss) are imported only inside `pkg/cli/` — the public API uses our own types. Stubs for future features (Form, FilePicker, Tabs) define the interface but fall back to simple bufio implementations until charm backends are wired in later. - -**Tech Stack:** `charmbracelet/bubbletea` (app loop), `charmbracelet/bubbles` (spinner, progress, list, textinput, viewport), `charmbracelet/lipgloss` (styling — replaces our ANSI builder long-term) - ---- - -## Context - -`go/pkg/cli` currently provides: -- Cobra wrappers: `Command`, `NewCommand()`, `NewGroup()`, flag helpers -- Output: `Success()`, `Error()`, `Table`, `Section()`, `Label()` -- Prompts: `Confirm()`, `Question()`, `Choose()`, `ChooseMulti()` (all bufio-based) -- Styles: `AnsiStyle` builder with 17 pre-built styles, 47 Tailwind colour constants -- Glyphs: `:check:`, `:cross:` etc. with theme switching -- Layout: HLCRF composite renderer - -Zero charmbracelet dependencies exist today. All styling is pure ANSI escape codes. - -The 34 files in `cli/cmd/*` that import `github.com/spf13/cobra` directly need `cli.*` equivalents. This plan does NOT migrate those files — it builds the SDK surface they'll need. Migration happens in Phase 1+. - -## Critical Files - -All changes are in `/Users/snider/Code/host-uk/core/pkg/cli/`: - -- `spinner.go` + `spinner_test.go` — Async spinner -- `progress.go` + `progress_test.go` — Progress bar -- `list.go` + `list_test.go` — Interactive scrollable list -- `textinput.go` + `textinput_test.go` — Styled text input -- `viewport.go` + `viewport_test.go` — Scrollable content pane -- `tui.go` + `tui_test.go` — RunTUI escape hatch + Model interface -- `stubs.go` + `stubs_test.go` — Form, FilePicker, Tabs interfaces (simple fallback) - ---- - -### Task 1: Add charmbracelet dependencies - -**Files:** -- Modify: `/Users/snider/Code/host-uk/core/go.mod` - -**Step 1: Add bubbletea, bubbles, and lipgloss** - -Run: -```bash -cd /Users/snider/Code/host-uk/core && go get github.com/charmbracelet/bubbletea/v2@latest github.com/charmbracelet/bubbles/v2@latest github.com/charmbracelet/lipgloss/v2@latest -``` - -**Step 2: Verify module resolves** - -Run: `cd /Users/snider/Code/host-uk/core && go mod tidy` -Expected: Clean, no errors. - -**Step 3: Verify existing tests still pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./pkg/cli/...` -Expected: All existing tests pass (no behaviour changed). - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add go.mod go.sum && git commit -m "chore(cli): add charmbracelet dependencies (bubbletea, bubbles, lipgloss) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: Spinner - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/spinner.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/spinner_test.go` - -A non-blocking spinner that runs in a goroutine. The caller gets a handle to update the message, mark it done, or mark it failed. Uses `bubbles/spinner` internally. - -**Step 1: Write the tests** - -```go -// spinner_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestSpinner_Good_CreateAndStop(t *testing.T) { - s := NewSpinner("Loading...") - require.NotNil(t, s) - assert.Equal(t, "Loading...", s.Message()) - s.Stop() -} - -func TestSpinner_Good_UpdateMessage(t *testing.T) { - s := NewSpinner("Step 1") - s.Update("Step 2") - assert.Equal(t, "Step 2", s.Message()) - s.Stop() -} - -func TestSpinner_Good_Done(t *testing.T) { - s := NewSpinner("Building") - s.Done("Build complete") - // After Done, spinner is stopped — calling Stop again is safe - s.Stop() -} - -func TestSpinner_Good_Fail(t *testing.T) { - s := NewSpinner("Checking") - s.Fail("Check failed") - s.Stop() -} - -func TestSpinner_Good_DoubleStop(t *testing.T) { - s := NewSpinner("Loading") - s.Stop() - s.Stop() // Should not panic -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestSpinner ./pkg/cli/...` -Expected: FAIL — `NewSpinner` undefined. - -**Step 3: Write the implementation** - -```go -// spinner.go -package cli - -import ( - "fmt" - "sync" - "time" -) - -// SpinnerHandle controls a running spinner. -type SpinnerHandle struct { - mu sync.Mutex - message string - done bool - ticker *time.Ticker - stopCh chan struct{} -} - -// NewSpinner starts an async spinner with the given message. -// Call Stop(), Done(), or Fail() to stop it. -func NewSpinner(message string) *SpinnerHandle { - s := &SpinnerHandle{ - message: message, - ticker: time.NewTicker(100 * time.Millisecond), - stopCh: make(chan struct{}), - } - - frames := []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"} - if !ColorEnabled() { - frames = []string{"|", "/", "-", "\\"} - } - - go func() { - i := 0 - for { - select { - case <-s.stopCh: - return - case <-s.ticker.C: - s.mu.Lock() - if !s.done { - fmt.Printf("\033[2K\r%s %s", DimStyle.Render(frames[i%len(frames)]), s.message) - } - s.mu.Unlock() - i++ - } - } - }() - - return s -} - -// Message returns the current spinner message. -func (s *SpinnerHandle) Message() string { - s.mu.Lock() - defer s.mu.Unlock() - return s.message -} - -// Update changes the spinner message. -func (s *SpinnerHandle) Update(message string) { - s.mu.Lock() - defer s.mu.Unlock() - s.message = message -} - -// Stop stops the spinner silently (clears the line). -func (s *SpinnerHandle) Stop() { - s.mu.Lock() - defer s.mu.Unlock() - if s.done { - return - } - s.done = true - s.ticker.Stop() - close(s.stopCh) - fmt.Print("\033[2K\r") -} - -// Done stops the spinner with a success message. -func (s *SpinnerHandle) Done(message string) { - s.mu.Lock() - alreadyDone := s.done - s.done = true - s.mu.Unlock() - - if alreadyDone { - return - } - s.ticker.Stop() - close(s.stopCh) - fmt.Printf("\033[2K\r%s\n", SuccessStyle.Render(Glyph(":check:")+" "+message)) -} - -// Fail stops the spinner with an error message. -func (s *SpinnerHandle) Fail(message string) { - s.mu.Lock() - alreadyDone := s.done - s.done = true - s.mu.Unlock() - - if alreadyDone { - return - } - s.ticker.Stop() - close(s.stopCh) - fmt.Printf("\033[2K\r%s\n", ErrorStyle.Render(Glyph(":cross:")+" "+message)) -} -``` - -Note: This initial implementation uses a goroutine + ticker rather than bubbletea, keeping it simple and non-blocking. The bubbletea spinner can replace the internals later without changing the public API. - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestSpinner ./pkg/cli/... -v` -Expected: All 5 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/spinner.go pkg/cli/spinner_test.go && git commit -m "feat(cli): add Spinner with async handle (Update, Done, Fail) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: ProgressBar - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/progressbar.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/progressbar_test.go` - -A progress bar that renders inline. Shows percentage, bar, and optional message. - -**Step 1: Write the tests** - -```go -// progressbar_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestProgressBar_Good_Create(t *testing.T) { - pb := NewProgressBar(100) - require.NotNil(t, pb) - assert.Equal(t, 0, pb.Current()) - assert.Equal(t, 100, pb.Total()) -} - -func TestProgressBar_Good_Increment(t *testing.T) { - pb := NewProgressBar(10) - pb.Increment() - assert.Equal(t, 1, pb.Current()) - pb.Increment() - assert.Equal(t, 2, pb.Current()) -} - -func TestProgressBar_Good_SetMessage(t *testing.T) { - pb := NewProgressBar(10) - pb.SetMessage("Processing file.go") - assert.Equal(t, "Processing file.go", pb.message) -} - -func TestProgressBar_Good_Set(t *testing.T) { - pb := NewProgressBar(100) - pb.Set(50) - assert.Equal(t, 50, pb.Current()) -} - -func TestProgressBar_Good_Done(t *testing.T) { - pb := NewProgressBar(5) - for i := 0; i < 5; i++ { - pb.Increment() - } - pb.Done() - // After Done, Current == Total - assert.Equal(t, 5, pb.Current()) -} - -func TestProgressBar_Bad_ExceedsTotal(t *testing.T) { - pb := NewProgressBar(2) - pb.Increment() - pb.Increment() - pb.Increment() // Should clamp to total - assert.Equal(t, 2, pb.Current()) -} - -func TestProgressBar_Good_Render(t *testing.T) { - pb := NewProgressBar(10) - pb.Set(5) - rendered := pb.String() - assert.Contains(t, rendered, "50%") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestProgressBar ./pkg/cli/... -v` -Expected: FAIL — `NewProgressBar` undefined. - -**Step 3: Write the implementation** - -```go -// progressbar.go -package cli - -import ( - "fmt" - "strings" - "sync" -) - -// ProgressHandle controls a progress bar. -type ProgressHandle struct { - mu sync.Mutex - current int - total int - message string - width int -} - -// NewProgressBar creates a new progress bar with the given total. -func NewProgressBar(total int) *ProgressHandle { - return &ProgressHandle{ - total: total, - width: 30, - } -} - -// Current returns the current progress value. -func (p *ProgressHandle) Current() int { - p.mu.Lock() - defer p.mu.Unlock() - return p.current -} - -// Total returns the total value. -func (p *ProgressHandle) Total() int { - return p.total -} - -// Increment advances the progress by 1. -func (p *ProgressHandle) Increment() { - p.mu.Lock() - defer p.mu.Unlock() - if p.current < p.total { - p.current++ - } - p.render() -} - -// Set sets the progress to a specific value. -func (p *ProgressHandle) Set(n int) { - p.mu.Lock() - defer p.mu.Unlock() - if n > p.total { - n = p.total - } - if n < 0 { - n = 0 - } - p.current = n - p.render() -} - -// SetMessage sets the message displayed alongside the bar. -func (p *ProgressHandle) SetMessage(msg string) { - p.mu.Lock() - defer p.mu.Unlock() - p.message = msg - p.render() -} - -// Done completes the progress bar and moves to a new line. -func (p *ProgressHandle) Done() { - p.mu.Lock() - defer p.mu.Unlock() - p.current = p.total - p.render() - fmt.Println() -} - -// String returns the rendered progress bar without ANSI cursor control. -func (p *ProgressHandle) String() string { - pct := 0 - if p.total > 0 { - pct = (p.current * 100) / p.total - } - - filled := (p.width * p.current) / p.total - if filled > p.width { - filled = p.width - } - empty := p.width - filled - - bar := "[" + strings.Repeat("█", filled) + strings.Repeat("░", empty) + "]" - - if p.message != "" { - return fmt.Sprintf("%s %3d%% %s", bar, pct, p.message) - } - return fmt.Sprintf("%s %3d%%", bar, pct) -} - -// render outputs the progress bar, overwriting the current line. -func (p *ProgressHandle) render() { - fmt.Printf("\033[2K\r%s", p.String()) -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestProgressBar ./pkg/cli/... -v` -Expected: All 7 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/progressbar.go pkg/cli/progressbar_test.go && git commit -m "feat(cli): add ProgressBar with Increment, Set, SetMessage, Done - -Co-Authored-By: Virgil " -``` - ---- - -### Task 4: TUI runner (RunTUI + Model interface) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/tui.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/tui_test.go` - -The escape hatch for complex interactive UIs. Wraps `bubbletea.Program` behind our own `Model` interface so domain packages never import bubbletea directly. - -**Step 1: Write the tests** - -```go -// tui_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -// testModel is a minimal Model that quits immediately. -type testModel struct { - initCalled bool - updateCalled bool - viewCalled bool -} - -func (m *testModel) Init() Cmd { - m.initCalled = true - return Quit -} - -func (m *testModel) Update(msg Msg) (Model, Cmd) { - m.updateCalled = true - return m, nil -} - -func (m *testModel) View() string { - m.viewCalled = true - return "test view" -} - -func TestModel_Good_InterfaceSatisfied(t *testing.T) { - var m Model = &testModel{} - assert.NotNil(t, m) -} - -func TestQuitCmd_Good_ReturnsQuitMsg(t *testing.T) { - cmd := Quit - assert.NotNil(t, cmd) -} - -func TestKeyMsg_Good_String(t *testing.T) { - k := KeyMsg{Type: KeyEnter} - assert.Equal(t, KeyEnter, k.Type) -} - -func TestKeyTypes_Good_Constants(t *testing.T) { - // Verify key type constants exist - assert.NotEmpty(t, string(KeyEnter)) - assert.NotEmpty(t, string(KeyEsc)) - assert.NotEmpty(t, string(KeyCtrlC)) - assert.NotEmpty(t, string(KeyUp)) - assert.NotEmpty(t, string(KeyDown)) - assert.NotEmpty(t, string(KeyTab)) - assert.NotEmpty(t, string(KeyBackspace)) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestModel ./pkg/cli/... -v && go test -run TestQuit ./pkg/cli/... -v && go test -run TestKey ./pkg/cli/... -v` -Expected: FAIL — types undefined. - -**Step 3: Write the implementation** - -```go -// tui.go -package cli - -import ( - tea "github.com/charmbracelet/bubbletea/v2" -) - -// Model is the interface for interactive TUI applications. -// It mirrors bubbletea's Model but uses our own types so domain -// packages never import bubbletea directly. -type Model interface { - // Init returns an initial command to run. - Init() Cmd - - // Update handles a message and returns the updated model and command. - Update(msg Msg) (Model, Cmd) - - // View returns the string representation of the UI. - View() string -} - -// Msg is a message passed to Update. Can be any type. -type Msg = tea.Msg - -// Cmd is a function that returns a message. Nil means no command. -type Cmd = tea.Cmd - -// Quit is a command that tells the TUI to exit. -var Quit = tea.Quit - -// KeyMsg represents a key press event. -type KeyMsg = tea.KeyMsg - -// KeyType represents the type of key pressed. -type KeyType = tea.KeyType - -// Key type constants. -const ( - KeyEnter KeyType = tea.KeyEnter - KeyEsc KeyType = tea.KeyEscape - KeyCtrlC KeyType = tea.KeyCtrlC - KeyUp KeyType = tea.KeyUp - KeyDown KeyType = tea.KeyDown - KeyLeft KeyType = tea.KeyLeft - KeyRight KeyType = tea.KeyRight - KeyTab KeyType = tea.KeyTab - KeyBackspace KeyType = tea.KeyBackspace - KeySpace KeyType = tea.KeySpace - KeyHome KeyType = tea.KeyHome - KeyEnd KeyType = tea.KeyEnd - KeyPgUp KeyType = tea.KeyPgUp - KeyPgDown KeyType = tea.KeyPgDown - KeyDelete KeyType = tea.KeyDelete -) - -// adapter wraps our Model interface into a bubbletea.Model. -type adapter struct { - inner Model -} - -func (a adapter) Init() tea.Cmd { - return a.inner.Init() -} - -func (a adapter) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - m, cmd := a.inner.Update(msg) - return adapter{inner: m}, cmd -} - -func (a adapter) View() string { - return a.inner.View() -} - -// RunTUI runs an interactive TUI application using the provided Model. -// This is the escape hatch for complex interactive UIs that need the -// full bubbletea event loop. For simple spinners, progress bars, and -// lists, use the dedicated helpers instead. -// -// err := cli.RunTUI(&myModel{items: items}) -func RunTUI(m Model) error { - p := tea.NewProgram(adapter{inner: m}) - _, err := p.Run() - return err -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestModel|TestQuit|TestKey" ./pkg/cli/... -v` -Expected: All 4 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/tui.go pkg/cli/tui_test.go && git commit -m "feat(cli): add RunTUI escape hatch with Model/Msg/Cmd/KeyMsg types - -Wraps bubbletea behind our own interface so domain packages -never import charmbracelet directly. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 5: Interactive List - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/list.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/list_test.go` - -An interactive scrollable list for terminal selection. Uses our `RunTUI` internally. Falls back to numbered `Select()` when stdin is not a terminal. - -**Step 1: Write the tests** - -```go -// list_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestListModel_Good_Create(t *testing.T) { - items := []string{"alpha", "beta", "gamma"} - m := newListModel(items, "Pick one:") - assert.Equal(t, 3, len(m.items)) - assert.Equal(t, 0, m.cursor) - assert.Equal(t, "Pick one:", m.title) -} - -func TestListModel_Good_MoveDown(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - assert.Equal(t, 1, m.cursor) - m.moveDown() - assert.Equal(t, 2, m.cursor) -} - -func TestListModel_Good_MoveUp(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - m.moveDown() - m.moveUp() - assert.Equal(t, 1, m.cursor) -} - -func TestListModel_Good_WrapAround(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveUp() // Should wrap to bottom - assert.Equal(t, 2, m.cursor) -} - -func TestListModel_Good_View(t *testing.T) { - m := newListModel([]string{"alpha", "beta"}, "Choose:") - view := m.View() - assert.Contains(t, view, "Choose:") - assert.Contains(t, view, "alpha") - assert.Contains(t, view, "beta") -} - -func TestListModel_Good_Selected(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - m.selected = true - assert.Equal(t, "b", m.items[m.cursor]) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestListModel ./pkg/cli/... -v` -Expected: FAIL — `newListModel` undefined. - -**Step 3: Write the implementation** - -```go -// list.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// listModel is the internal bubbletea model for interactive list selection. -type listModel struct { - items []string - cursor int - title string - selected bool - quitted bool -} - -func newListModel(items []string, title string) *listModel { - return &listModel{ - items: items, - title: title, - } -} - -func (m *listModel) moveDown() { - m.cursor++ - if m.cursor >= len(m.items) { - m.cursor = 0 - } -} - -func (m *listModel) moveUp() { - m.cursor-- - if m.cursor < 0 { - m.cursor = len(m.items) - 1 - } -} - -func (m *listModel) Init() tea.Cmd { - return nil -} - -func (m *listModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyUp, tea.KeyShiftTab: - m.moveUp() - case tea.KeyDown, tea.KeyTab: - m.moveDown() - case tea.KeyEnter: - m.selected = true - return m, tea.Quit - case tea.KeyEscape, tea.KeyCtrlC: - m.quitted = true - return m, tea.Quit - default: - // Handle j/k vim-style navigation - if msg.String() == "j" { - m.moveDown() - } else if msg.String() == "k" { - m.moveUp() - } - } - } - return m, nil -} - -func (m *listModel) View() string { - var sb strings.Builder - - if m.title != "" { - sb.WriteString(BoldStyle.Render(m.title) + "\n\n") - } - - for i, item := range m.items { - cursor := " " - style := DimStyle - if i == m.cursor { - cursor = AccentStyle.Render(Glyph(":pointer:")) + " " - style = BoldStyle - } - sb.WriteString(fmt.Sprintf("%s%s\n", cursor, style.Render(item))) - } - - sb.WriteString("\n" + DimStyle.Render("↑/↓ navigate • enter select • esc cancel")) - - return sb.String() -} - -// ListOption configures List behaviour. -type ListOption func(*listConfig) - -type listConfig struct { - height int -} - -// WithHeight sets the visible height of the list (number of items shown). -func WithHeight(n int) ListOption { - return func(c *listConfig) { - c.height = n - } -} - -// InteractiveList presents an interactive scrollable list and returns the -// selected item's index and value. Returns -1 and empty string if cancelled. -// -// Falls back to numbered Select() when stdin is not a terminal (e.g. piped input). -// -// idx, value := cli.InteractiveList("Pick a repo:", repos) -func InteractiveList(title string, items []string, opts ...ListOption) (int, string) { - if len(items) == 0 { - return -1, "" - } - - // Fall back to simple Select if not a terminal - if !term.IsTerminal(int(StdinFd())) { - result, err := Select(title, items) - if err != nil { - return -1, "" - } - for i, item := range items { - if item == result { - return i, result - } - } - return -1, "" - } - - m := newListModel(items, title) - p := tea.NewProgram(m) - finalModel, err := p.Run() - if err != nil { - return -1, "" - } - - final := finalModel.(*listModel) - if final.quitted || !final.selected { - return -1, "" - } - return final.cursor, final.items[final.cursor] -} - -// StdinFd returns the file descriptor for stdin. -// Extracted for testing. -func StdinFd() uintptr { - return uintptr(0) // stdin -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestListModel ./pkg/cli/... -v` -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/list.go pkg/cli/list_test.go && git commit -m "feat(cli): add InteractiveList with keyboard navigation and terminal fallback - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: TextInput - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/textinput.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/textinput_test.go` - -A styled single-line text input with placeholder, validation, and optional masking (for passwords). Falls back to `Question()` when stdin is not a terminal. - -**Step 1: Write the tests** - -```go -// textinput_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestTextInputModel_Good_Create(t *testing.T) { - m := newTextInputModel("Enter name:", "") - assert.Equal(t, "Enter name:", m.title) - assert.Equal(t, "", m.value) -} - -func TestTextInputModel_Good_WithPlaceholder(t *testing.T) { - m := newTextInputModel("Name:", "John") - assert.Equal(t, "John", m.placeholder) -} - -func TestTextInputModel_Good_TypeCharacters(t *testing.T) { - m := newTextInputModel("Name:", "") - m.insertChar('H') - m.insertChar('i') - assert.Equal(t, "Hi", m.value) -} - -func TestTextInputModel_Good_Backspace(t *testing.T) { - m := newTextInputModel("Name:", "") - m.insertChar('A') - m.insertChar('B') - m.backspace() - assert.Equal(t, "A", m.value) -} - -func TestTextInputModel_Good_BackspaceEmpty(t *testing.T) { - m := newTextInputModel("Name:", "") - m.backspace() // Should not panic - assert.Equal(t, "", m.value) -} - -func TestTextInputModel_Good_Masked(t *testing.T) { - m := newTextInputModel("Password:", "") - m.masked = true - m.insertChar('s') - m.insertChar('e') - m.insertChar('c') - assert.Equal(t, "sec", m.value) // Internal value is real - view := m.View() - assert.NotContains(t, view, "sec") // Display is masked - assert.Contains(t, view, "***") -} - -func TestTextInputModel_Good_View(t *testing.T) { - m := newTextInputModel("Enter:", "") - m.insertChar('X') - view := m.View() - assert.Contains(t, view, "Enter:") - assert.Contains(t, view, "X") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestTextInputModel ./pkg/cli/... -v` -Expected: FAIL — `newTextInputModel` undefined. - -**Step 3: Write the implementation** - -```go -// textinput.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// textInputModel is the internal bubbletea model for text input. -type textInputModel struct { - title string - placeholder string - value string - masked bool - submitted bool - cancelled bool - cursorPos int - validator func(string) error - err error -} - -func newTextInputModel(title, placeholder string) *textInputModel { - return &textInputModel{ - title: title, - placeholder: placeholder, - } -} - -func (m *textInputModel) insertChar(ch rune) { - m.value = m.value[:m.cursorPos] + string(ch) + m.value[m.cursorPos:] - m.cursorPos++ -} - -func (m *textInputModel) backspace() { - if m.cursorPos > 0 { - m.value = m.value[:m.cursorPos-1] + m.value[m.cursorPos:] - m.cursorPos-- - } -} - -func (m *textInputModel) Init() tea.Cmd { - return nil -} - -func (m *textInputModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyEnter: - if m.validator != nil { - if err := m.validator(m.value); err != nil { - m.err = err - return m, nil - } - } - if m.value == "" && m.placeholder != "" { - m.value = m.placeholder - } - m.submitted = true - return m, tea.Quit - case tea.KeyEscape, tea.KeyCtrlC: - m.cancelled = true - return m, tea.Quit - case tea.KeyBackspace: - m.backspace() - m.err = nil - case tea.KeyLeft: - if m.cursorPos > 0 { - m.cursorPos-- - } - case tea.KeyRight: - if m.cursorPos < len(m.value) { - m.cursorPos++ - } - default: - if msg.Text != "" { - for _, ch := range msg.Text { - m.insertChar(ch) - } - m.err = nil - } - } - } - return m, nil -} - -func (m *textInputModel) View() string { - var sb strings.Builder - - sb.WriteString(BoldStyle.Render(m.title) + "\n\n") - - display := m.value - if m.masked { - display = strings.Repeat("*", len(m.value)) - } - - if display == "" && m.placeholder != "" { - sb.WriteString(DimStyle.Render(m.placeholder)) - } else { - sb.WriteString(display) - } - sb.WriteString(AccentStyle.Render("█")) // Cursor - - if m.err != nil { - sb.WriteString("\n" + ErrorStyle.Render(fmt.Sprintf(" %s", m.err))) - } - - sb.WriteString("\n\n" + DimStyle.Render("enter submit • esc cancel")) - - return sb.String() -} - -// TextInputOption configures TextInput behaviour. -type TextInputOption func(*textInputConfig) - -type textInputConfig struct { - placeholder string - masked bool - validator func(string) error -} - -// WithPlaceholder sets placeholder text shown when input is empty. -func WithPlaceholder(text string) TextInputOption { - return func(c *textInputConfig) { - c.placeholder = text - } -} - -// WithMask hides input characters (for passwords). -func WithMask() TextInputOption { - return func(c *textInputConfig) { - c.masked = true - } -} - -// WithInputValidator adds a validation function for the input. -func WithInputValidator(fn func(string) error) TextInputOption { - return func(c *textInputConfig) { - c.validator = fn - } -} - -// TextInput presents a styled text input prompt and returns the entered value. -// Returns empty string if cancelled. -// -// Falls back to Question() when stdin is not a terminal. -// -// name, err := cli.TextInput("Enter your name:", WithPlaceholder("Anonymous")) -// pass, err := cli.TextInput("Password:", WithMask()) -func TextInput(title string, opts ...TextInputOption) (string, error) { - cfg := &textInputConfig{} - for _, opt := range opts { - opt(cfg) - } - - // Fall back to simple Question if not a terminal - if !term.IsTerminal(int(StdinFd())) { - var qopts []QuestionOption - if cfg.placeholder != "" { - qopts = append(qopts, WithDefault(cfg.placeholder)) - } - if cfg.validator != nil { - qopts = append(qopts, WithValidator(cfg.validator)) - } - return Question(title, qopts...), nil - } - - m := newTextInputModel(title, cfg.placeholder) - m.masked = cfg.masked - m.validator = cfg.validator - - p := tea.NewProgram(m) - finalModel, err := p.Run() - if err != nil { - return "", err - } - - final := finalModel.(*textInputModel) - if final.cancelled { - return "", nil - } - return final.value, nil -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestTextInputModel ./pkg/cli/... -v` -Expected: All 7 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/textinput.go pkg/cli/textinput_test.go && git commit -m "feat(cli): add TextInput with placeholder, masking, validation - -Co-Authored-By: Virgil " -``` - ---- - -### Task 7: Viewport (scrollable content) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/viewport.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/viewport_test.go` - -A scrollable content pane for displaying long output (logs, diffs, docs). Uses bubbletea internally. - -**Step 1: Write the tests** - -```go -// viewport_test.go -package cli - -import ( - "strings" - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestViewportModel_Good_Create(t *testing.T) { - content := "line 1\nline 2\nline 3" - m := newViewportModel(content, "Title", 5) - assert.Equal(t, "Title", m.title) - assert.Equal(t, 3, len(m.lines)) - assert.Equal(t, 0, m.offset) -} - -func TestViewportModel_Good_ScrollDown(t *testing.T) { - lines := make([]string, 20) - for i := range lines { - lines[i] = strings.Repeat("x", 10) - } - m := newViewportModel(strings.Join(lines, "\n"), "", 5) - m.scrollDown() - assert.Equal(t, 1, m.offset) -} - -func TestViewportModel_Good_ScrollUp(t *testing.T) { - lines := make([]string, 20) - for i := range lines { - lines[i] = strings.Repeat("x", 10) - } - m := newViewportModel(strings.Join(lines, "\n"), "", 5) - m.scrollDown() - m.scrollDown() - m.scrollUp() - assert.Equal(t, 1, m.offset) -} - -func TestViewportModel_Good_NoScrollPastTop(t *testing.T) { - m := newViewportModel("a\nb\nc", "", 5) - m.scrollUp() // Already at top - assert.Equal(t, 0, m.offset) -} - -func TestViewportModel_Good_NoScrollPastBottom(t *testing.T) { - m := newViewportModel("a\nb\nc", "", 5) - for i := 0; i < 10; i++ { - m.scrollDown() - } - // Should clamp — can't scroll past content - assert.GreaterOrEqual(t, m.offset, 0) -} - -func TestViewportModel_Good_View(t *testing.T) { - m := newViewportModel("line 1\nline 2", "My Title", 10) - view := m.View() - assert.Contains(t, view, "My Title") - assert.Contains(t, view, "line 1") - assert.Contains(t, view, "line 2") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestViewportModel ./pkg/cli/... -v` -Expected: FAIL — `newViewportModel` undefined. - -**Step 3: Write the implementation** - -```go -// viewport.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// viewportModel is the internal bubbletea model for scrollable content. -type viewportModel struct { - title string - lines []string - offset int - height int - quitted bool -} - -func newViewportModel(content, title string, height int) *viewportModel { - lines := strings.Split(content, "\n") - return &viewportModel{ - title: title, - lines: lines, - height: height, - } -} - -func (m *viewportModel) scrollDown() { - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - if m.offset < maxOffset { - m.offset++ - } -} - -func (m *viewportModel) scrollUp() { - if m.offset > 0 { - m.offset-- - } -} - -func (m *viewportModel) Init() tea.Cmd { - return nil -} - -func (m *viewportModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyUp: - m.scrollUp() - case tea.KeyDown: - m.scrollDown() - case tea.KeyPgUp: - for i := 0; i < m.height; i++ { - m.scrollUp() - } - case tea.KeyPgDown: - for i := 0; i < m.height; i++ { - m.scrollDown() - } - case tea.KeyHome: - m.offset = 0 - case tea.KeyEnd: - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - m.offset = maxOffset - case tea.KeyEscape, tea.KeyCtrlC: - m.quitted = true - return m, tea.Quit - default: - switch msg.String() { - case "q": - m.quitted = true - return m, tea.Quit - case "j": - m.scrollDown() - case "k": - m.scrollUp() - case "g": - m.offset = 0 - case "G": - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - m.offset = maxOffset - } - } - } - return m, nil -} - -func (m *viewportModel) View() string { - var sb strings.Builder - - if m.title != "" { - sb.WriteString(BoldStyle.Render(m.title) + "\n") - sb.WriteString(DimStyle.Render(strings.Repeat("─", len(m.title))) + "\n") - } - - // Visible window - end := m.offset + m.height - if end > len(m.lines) { - end = len(m.lines) - } - for _, line := range m.lines[m.offset:end] { - sb.WriteString(line + "\n") - } - - // Scroll indicator - total := len(m.lines) - if total > m.height { - pct := (m.offset * 100) / (total - m.height) - sb.WriteString(DimStyle.Render(fmt.Sprintf("\n%d%% (%d/%d lines)", pct, m.offset+m.height, total))) - } - - sb.WriteString("\n" + DimStyle.Render("↑/↓ scroll • PgUp/PgDn page • q quit")) - - return sb.String() -} - -// ViewportOption configures Viewport behaviour. -type ViewportOption func(*viewportConfig) - -type viewportConfig struct { - title string - height int -} - -// WithViewportTitle sets the title shown above the viewport. -func WithViewportTitle(title string) ViewportOption { - return func(c *viewportConfig) { - c.title = title - } -} - -// WithViewportHeight sets the visible height in lines. -func WithViewportHeight(n int) ViewportOption { - return func(c *viewportConfig) { - c.height = n - } -} - -// Viewport displays scrollable content in the terminal. -// Falls back to printing the full content when stdin is not a terminal. -// -// cli.Viewport(longContent, WithViewportTitle("Build Log"), WithViewportHeight(20)) -func Viewport(content string, opts ...ViewportOption) error { - cfg := &viewportConfig{ - height: 20, - } - for _, opt := range opts { - opt(cfg) - } - - // Fall back to plain output if not a terminal - if !term.IsTerminal(int(StdinFd())) { - if cfg.title != "" { - fmt.Println(BoldStyle.Render(cfg.title)) - fmt.Println(DimStyle.Render(strings.Repeat("─", len(cfg.title)))) - } - fmt.Println(content) - return nil - } - - m := newViewportModel(content, cfg.title, cfg.height) - p := tea.NewProgram(m) - _, err := p.Run() - return err -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestViewportModel ./pkg/cli/... -v` -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/viewport.go pkg/cli/viewport_test.go && git commit -m "feat(cli): add Viewport for scrollable content (logs, diffs, docs) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: Future stubs (Form, FilePicker, Tabs) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/stubs.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/stubs_test.go` - -Interface definitions for features we'll build later. Simple fallback implementations so the API is usable today. - -**Step 1: Write the tests** - -```go -// stubs_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestFormField_Good_Types(t *testing.T) { - fields := []FormField{ - {Label: "Name", Key: "name", Type: FieldText}, - {Label: "Password", Key: "pass", Type: FieldPassword}, - {Label: "Accept", Key: "ok", Type: FieldConfirm}, - } - assert.Equal(t, 3, len(fields)) - assert.Equal(t, FieldText, fields[0].Type) - assert.Equal(t, FieldPassword, fields[1].Type) - assert.Equal(t, FieldConfirm, fields[2].Type) -} - -func TestFieldType_Good_Constants(t *testing.T) { - assert.Equal(t, FieldType("text"), FieldText) - assert.Equal(t, FieldType("password"), FieldPassword) - assert.Equal(t, FieldType("confirm"), FieldConfirm) - assert.Equal(t, FieldType("select"), FieldSelect) -} - -func TestTabItem_Good_Structure(t *testing.T) { - tabs := []TabItem{ - {Title: "Overview", Content: "overview content"}, - {Title: "Details", Content: "detail content"}, - } - assert.Equal(t, 2, len(tabs)) - assert.Equal(t, "Overview", tabs[0].Title) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestFormField|TestFieldType|TestTabItem" ./pkg/cli/... -v` -Expected: FAIL — types undefined. - -**Step 3: Write the implementation** - -```go -// stubs.go -package cli - -// ────────────────────────────────────────────────────────────────────────────── -// Form (stubbed — simple fallback, will use charmbracelet/huh later) -// ────────────────────────────────────────────────────────────────────────────── - -// FieldType defines the type of a form field. -type FieldType string - -const ( - FieldText FieldType = "text" - FieldPassword FieldType = "password" - FieldConfirm FieldType = "confirm" - FieldSelect FieldType = "select" -) - -// FormField describes a single field in a form. -type FormField struct { - Label string - Key string - Type FieldType - Default string - Placeholder string - Options []string // For FieldSelect - Required bool - Validator func(string) error -} - -// Form presents a multi-field form and returns the values keyed by FormField.Key. -// Currently falls back to sequential Question()/Confirm()/Select() calls. -// Will be replaced with charmbracelet/huh interactive form later. -// -// results, err := cli.Form([]cli.FormField{ -// {Label: "Name", Key: "name", Type: cli.FieldText, Required: true}, -// {Label: "Password", Key: "pass", Type: cli.FieldPassword}, -// {Label: "Accept terms?", Key: "terms", Type: cli.FieldConfirm}, -// }) -func Form(fields []FormField) (map[string]string, error) { - results := make(map[string]string, len(fields)) - - for _, f := range fields { - switch f.Type { - case FieldPassword: - val := Question(f.Label+":", WithDefault(f.Default)) - results[f.Key] = val - case FieldConfirm: - if Confirm(f.Label) { - results[f.Key] = "true" - } else { - results[f.Key] = "false" - } - case FieldSelect: - val, err := Select(f.Label, f.Options) - if err != nil { - return nil, err - } - results[f.Key] = val - default: // FieldText - var opts []QuestionOption - if f.Default != "" { - opts = append(opts, WithDefault(f.Default)) - } - if f.Required { - opts = append(opts, RequiredInput()) - } - if f.Validator != nil { - opts = append(opts, WithValidator(f.Validator)) - } - results[f.Key] = Question(f.Label+":", opts...) - } - } - - return results, nil -} - -// ────────────────────────────────────────────────────────────────────────────── -// FilePicker (stubbed — will use charmbracelet/filepicker later) -// ────────────────────────────────────────────────────────────────────────────── - -// FilePickerOption configures FilePicker behaviour. -type FilePickerOption func(*filePickerConfig) - -type filePickerConfig struct { - dir string - extensions []string -} - -// InDirectory sets the starting directory for the file picker. -func InDirectory(dir string) FilePickerOption { - return func(c *filePickerConfig) { - c.dir = dir - } -} - -// WithExtensions filters to specific file extensions (e.g. ".go", ".yaml"). -func WithExtensions(exts ...string) FilePickerOption { - return func(c *filePickerConfig) { - c.extensions = exts - } -} - -// FilePicker presents a file browser and returns the selected path. -// Currently falls back to a text prompt. Will be replaced with an -// interactive file browser later. -// -// path, err := cli.FilePicker(cli.InDirectory("."), cli.WithExtensions(".go")) -func FilePicker(opts ...FilePickerOption) (string, error) { - cfg := &filePickerConfig{dir: "."} - for _, opt := range opts { - opt(cfg) - } - - hint := "File path" - if cfg.dir != "." { - hint += " (from " + cfg.dir + ")" - } - return Question(hint + ":"), nil -} - -// ────────────────────────────────────────────────────────────────────────────── -// Tabs (stubbed — will use bubbletea model later) -// ────────────────────────────────────────────────────────────────────────────── - -// TabItem describes a tab with a title and content. -type TabItem struct { - Title string - Content string -} - -// Tabs displays tabbed content. Currently prints all tabs sequentially. -// Will be replaced with an interactive tab switcher later. -// -// cli.Tabs([]cli.TabItem{ -// {Title: "Overview", Content: summaryText}, -// {Title: "Details", Content: detailText}, -// }) -func Tabs(items []TabItem) error { - for i, tab := range items { - if i > 0 { - Blank() - } - Section(tab.Title) - Println("%s", tab.Content) - } - return nil -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestFormField|TestFieldType|TestTabItem" ./pkg/cli/... -v` -Expected: All 3 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/stubs.go pkg/cli/stubs_test.go && git commit -m "feat(cli): stub Form, FilePicker, Tabs with simple fallbacks - -Interfaces defined for future charmbracelet/huh upgrade. -Current implementations use sequential prompts. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Run full test suite and verify - -**Step 1: Run all cli package tests** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./pkg/cli/... -v -count=1` -Expected: All tests pass (existing + new). - -**Step 2: Run full module tests** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./... 2>&1 | tail -30` -Expected: No regressions. - -**Step 3: Verify no charmbracelet imports leaked outside pkg/cli** - -Run: `cd /Users/snider/Code/host-uk/core && grep -r "charmbracelet" --include="*.go" . | grep -v pkg/cli/ | grep -v vendor/` -Expected: No output (charmbracelet only imported inside pkg/cli/). - ---- - -## Verification - -After all tasks: - -1. `go test ./pkg/cli/... -v` — all pass (existing + ~34 new tests) -2. `go test ./...` — no regressions across the module -3. `grep -r "charmbracelet" --include="*.go" . | grep -v pkg/cli/` — empty (no leaks) -4. New public API surface: - - `NewSpinner(msg)` → `*SpinnerHandle` (Update, Done, Fail, Stop) - - `NewProgressBar(total)` → `*ProgressHandle` (Increment, Set, SetMessage, Done) - - `InteractiveList(title, items)` → `(int, string)` - - `TextInput(title, opts...)` → `(string, error)` - - `Viewport(content, opts...)` → `error` - - `RunTUI(model)` → `error` (escape hatch) - - `Form(fields)` → `(map[string]string, error)` (stub) - - `FilePicker(opts...)` → `(string, error)` (stub) - - `Tabs(items)` → `error` (stub) - - `Model`, `Msg`, `Cmd`, `KeyMsg`, `KeyType` + key constants - -## Dependency Sequencing - -``` -Task 1 (add deps) ← Task 2 (Spinner) -Task 1 ← Task 3 (ProgressBar) -Task 1 ← Task 4 (TUI runner) ← Task 5 (List) -Task 4 ← Task 6 (TextInput) -Task 4 ← Task 7 (Viewport) -Task 1 ← Task 8 (Stubs) -Tasks 2-8 ← Task 9 (Verification) -``` - -Tasks 2, 3, and 8 are independent of each other (can run in parallel after Task 1). Tasks 5, 6, 7 depend on Task 4 (RunTUI) but are independent of each other. diff --git a/docs/plans/completed/2026-02-21-go-forge-design.md b/docs/plans/completed/2026-02-21-go-forge-design.md deleted file mode 100644 index b718629..0000000 --- a/docs/plans/completed/2026-02-21-go-forge-design.md +++ /dev/null @@ -1,286 +0,0 @@ -# go-forge Design Document - -## Overview - -**go-forge** is a full-coverage Go client for the Forgejo API (450 endpoints, 284 paths, 229 types). It uses a generic `Resource[T, C, U]` pattern for CRUD operations (91% of endpoints) and hand-written methods for 39 unique action endpoints. Types are generated from Forgejo's `swagger.v1.json` spec. - -**Module path:** `forge.lthn.ai/core/go-forge` - -**Origin:** Extracted from `go-scm/forge/` (45 methods covering 10% of API), expanded to full coverage. - -## Architecture - -``` -forge.lthn.ai/core/go-forge -├── client.go # HTTP client: auth, headers, rate limiting, context.Context -├── pagination.go # Generic paginated request helper -├── resource.go # Resource[T, C, U] generic CRUD (List/Get/Create/Update/Delete) -├── errors.go # Typed error handling (APIError, NotFound, Forbidden, etc.) -├── forge.go # Top-level Forge client aggregating all services -│ -├── types/ # Generated from swagger.v1.json -│ ├── generate.go # //go:generate directive -│ ├── repo.go # Repository, CreateRepoOption, EditRepoOption -│ ├── issue.go # Issue, CreateIssueOption, EditIssueOption -│ ├── pr.go # PullRequest, CreatePullRequestOption -│ ├── user.go # User, CreateUserOption -│ ├── org.go # Organisation, CreateOrgOption -│ ├── team.go # Team, CreateTeamOption -│ ├── label.go # Label, CreateLabelOption -│ ├── release.go # Release, CreateReleaseOption -│ ├── branch.go # Branch, BranchProtection -│ ├── milestone.go # Milestone, CreateMilestoneOption -│ ├── hook.go # Hook, CreateHookOption -│ ├── key.go # DeployKey, PublicKey, GPGKey -│ ├── notification.go # NotificationThread, NotificationSubject -│ ├── package.go # Package, PackageFile -│ ├── action.go # ActionRunner, ActionSecret, ActionVariable -│ ├── commit.go # Commit, CommitStatus, CombinedStatus -│ ├── content.go # ContentsResponse, FileOptions -│ ├── wiki.go # WikiPage, WikiPageMetaData -│ ├── review.go # PullReview, PullReviewComment -│ ├── reaction.go # Reaction -│ ├── topic.go # TopicResponse -│ ├── misc.go # Markdown, License, GitignoreTemplate, NodeInfo -│ ├── admin.go # Cron, QuotaGroup, QuotaRule -│ ├── activity.go # Activity, Feed -│ └── common.go # Shared types: Permission, ExternalTracker, etc. -│ -├── repos.go # RepoService: CRUD + fork, mirror, transfer, template -├── issues.go # IssueService: CRUD + pin, deadline, reactions, stopwatch -├── pulls.go # PullService: CRUD + merge, update, reviews, dismiss -├── orgs.go # OrgService: CRUD + members, avatar, block, hooks -├── users.go # UserService: CRUD + keys, followers, starred, settings -├── teams.go # TeamService: CRUD + members, repos -├── admin.go # AdminService: users, orgs, cron, runners, quota, unadopted -├── branches.go # BranchService: CRUD + protection rules -├── releases.go # ReleaseService: CRUD + assets -├── labels.go # LabelService: repo + org + issue labels -├── webhooks.go # WebhookService: CRUD + test hook -├── notifications.go # NotificationService: list, mark read -├── packages.go # PackageService: list, get, delete -├── actions.go # ActionsService: runners, secrets, variables, workflow dispatch -├── contents.go # ContentService: file read/write/delete via API -├── wiki.go # WikiService: pages -├── commits.go # CommitService: status, notes, diff -├── misc.go # MiscService: markdown, licenses, gitignore, nodeinfo -│ -├── config.go # URL/token resolution: env → config file → flags -│ -├── cmd/forgegen/ # Code generator: swagger.v1.json → types/*.go -│ ├── main.go -│ ├── parser.go # Parse OpenAPI 2.0 definitions -│ ├── generator.go # Render Go source files -│ └── templates/ # Go text/template files for codegen -│ -└── testdata/ - └── swagger.v1.json # Pinned spec for testing + generation -``` - -## Key Design Decisions - -### 1. Generic Resource[T, C, U] - -Three type parameters: T (resource type), C (create options), U (update options). - -```go -type Resource[T any, C any, U any] struct { - client *Client - path string // e.g. "/api/v1/repos/{owner}/{repo}/issues" -} - -func (r *Resource[T, C, U]) List(ctx context.Context, params Params, opts ListOptions) ([]T, error) -func (r *Resource[T, C, U]) Get(ctx context.Context, params Params, id string) (*T, error) -func (r *Resource[T, C, U]) Create(ctx context.Context, params Params, body *C) (*T, error) -func (r *Resource[T, C, U]) Update(ctx context.Context, params Params, id string, body *U) (*T, error) -func (r *Resource[T, C, U]) Delete(ctx context.Context, params Params, id string) error -``` - -`Params` is `map[string]string` resolving path variables: `{"owner": "core", "repo": "go-forge"}`. - -This covers 411 of 450 endpoints (91%). - -### 2. Service Structs Embed Resource - -```go -type IssueService struct { - Resource[types.Issue, types.CreateIssueOption, types.EditIssueOption] -} - -// CRUD comes free. Actions are hand-written: -func (s *IssueService) Pin(ctx context.Context, owner, repo string, index int64) error -func (s *IssueService) SetDeadline(ctx context.Context, owner, repo string, index int64, deadline *time.Time) error -``` - -### 3. Top-Level Forge Client - -```go -type Forge struct { - client *Client - Repos *RepoService - Issues *IssueService - Pulls *PullService - Orgs *OrgService - Users *UserService - Teams *TeamService - Admin *AdminService - Branches *BranchService - Releases *ReleaseService - Labels *LabelService - Webhooks *WebhookService - Notifications *NotificationService - Packages *PackageService - Actions *ActionsService - Contents *ContentService - Wiki *WikiService - Commits *CommitService - Misc *MiscService -} - -func NewForge(url, token string, opts ...Option) *Forge -``` - -### 4. Codegen from swagger.v1.json - -The `cmd/forgegen/` tool reads the OpenAPI 2.0 spec and generates: -- Go struct definitions with JSON tags and doc comments -- Enum constants -- Type mapping (OpenAPI → Go) - -229 type definitions → ~25 grouped Go files in `types/`. - -Type mapping rules: -| OpenAPI | Go | -|---------|-----| -| `string` | `string` | -| `string` + `date-time` | `time.Time` | -| `integer` + `int64` | `int64` | -| `integer` | `int` | -| `boolean` | `bool` | -| `array` of T | `[]T` | -| `$ref` | `*T` (pointer) | -| nullable | pointer type | -| `binary` | `[]byte` | - -### 5. HTTP Client - -```go -type Client struct { - baseURL string - token string - httpClient *http.Client - userAgent string -} - -func New(url, token string, opts ...Option) *Client - -func (c *Client) Get(ctx context.Context, path string, out any) error -func (c *Client) Post(ctx context.Context, path string, body, out any) error -func (c *Client) Patch(ctx context.Context, path string, body, out any) error -func (c *Client) Put(ctx context.Context, path string, body, out any) error -func (c *Client) Delete(ctx context.Context, path string) error -``` - -Options: `WithHTTPClient`, `WithUserAgent`, `WithRateLimit`, `WithLogger`. - -### 6. Pagination - -Forgejo uses `page` + `limit` query params and `X-Total-Count` response header. - -```go -type ListOptions struct { - Page int - Limit int // default 50, max configurable -} - -type PagedResult[T any] struct { - Items []T - TotalCount int - Page int - HasMore bool -} - -// ListAll fetches all pages automatically. -func (r *Resource[T, C, U]) ListAll(ctx context.Context, params Params) ([]T, error) -``` - -### 7. Error Handling - -```go -type APIError struct { - StatusCode int - Message string - URL string -} - -func IsNotFound(err error) bool -func IsForbidden(err error) bool -func IsConflict(err error) bool -``` - -### 8. Config Resolution (from go-scm/forge) - -Priority: flags → environment → config file. - -```go -func NewFromConfig(flagURL, flagToken string) (*Forge, error) -func ResolveConfig(flagURL, flagToken string) (url, token string, err error) -func SaveConfig(url, token string) error -``` - -Env vars: `FORGE_URL`, `FORGE_TOKEN`. Config file: `~/.config/forge/config.json`. - -## API Coverage - -| Category | Endpoints | CRUD | Actions | -|----------|-----------|------|---------| -| Repository | 175 | 165 | 10 (fork, mirror, transfer, template, avatar, diffpatch) | -| User | 74 | 70 | 4 (avatar, GPG verify) | -| Issue | 67 | 57 | 10 (pin, deadline, reactions, stopwatch, labels) | -| Organisation | 63 | 59 | 4 (avatar, block/unblock) | -| Admin | 39 | 35 | 4 (cron run, rename, adopt, quota set) | -| Miscellaneous | 12 | 7 | 5 (markdown render, markup, nodeinfo) | -| Notification | 7 | 7 | 0 | -| ActivityPub | 6 | 3 | 3 (inbox POST) | -| Package | 4 | 4 | 0 | -| Settings | 4 | 4 | 0 | -| **Total** | **450** | **411** | **39** | - -## Integration Points - -### go-api - -Services implement `DescribableGroup` from go-api Phase 3, enabling: -- REST endpoint generation via ToolBridge -- Auto-generated OpenAPI spec -- Multi-language SDK codegen - -### go-scm - -go-scm/forge/ becomes a thin adapter importing go-forge types. Existing go-scm users are unaffected — the multi-provider abstraction layer stays. - -### go-ai/mcp - -The MCP subsystem can register go-forge operations as MCP tools, giving AI agents full Forgejo API access. - -## 39 Unique Action Methods - -These require hand-written implementation: - -**Repository:** migrate, fork, generate (template), transfer, accept/reject transfer, mirror sync, push mirror sync, avatar, diffpatch, contents (multi-file modify) - -**Pull Requests:** merge, update (rebase), submit review, dismiss/undismiss review - -**Issues:** pin, set deadline, add reaction, start/stop stopwatch, add issue labels - -**Comments:** add reaction - -**Admin:** run cron task, adopt unadopted, rename user, set quota groups - -**Misc:** render markdown, render raw markdown, render markup, GPG key verify - -**ActivityPub:** inbox POST (actor, repo, user) - -**Actions:** dispatch workflow - -**Git:** set note on commit, test webhook diff --git a/docs/plans/completed/2026-02-21-go-forge-plan.md b/docs/plans/completed/2026-02-21-go-forge-plan.md deleted file mode 100644 index c6b8240..0000000 --- a/docs/plans/completed/2026-02-21-go-forge-plan.md +++ /dev/null @@ -1,2549 +0,0 @@ -# go-forge Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build a full-coverage Go client for the Forgejo API (450 endpoints) using a generic Resource[T,C,U] pattern and types generated from swagger.v1.json. - -**Architecture:** A code generator (`cmd/forgegen/`) parses Forgejo's Swagger 2.0 spec and emits typed Go structs. A generic `Resource[T,C,U]` provides List/Get/Create/Update/Delete for 411 CRUD endpoints. 18 service structs embed the generic resource and add 39 hand-written action methods. An HTTP client handles auth, pagination, rate limiting, and context.Context. - -**Tech Stack:** Go 1.25, `net/http`, `text/template`, generics, Swagger 2.0 (JSON) - ---- - -## Context - -**This is a NEW repo** at `forge.lthn.ai/core/go-forge`. Create it locally at `/Users/snider/Code/go-forge`. - -**Extracted from:** `/Users/snider/Code/go-scm/forge/` (45 methods covering 10% of API). The config resolution pattern (env → file → flags) comes from there. - -**Swagger spec:** Download from `https://forge.lthn.ai/swagger.v1.json` — Swagger 2.0 format, 229 type definitions, 450 operations across 284 paths. Pin it at `testdata/swagger.v1.json`. - -**Forgejo version:** 10.0.3 (Gitea 1.22.0 compatible) - -**Dependencies:** None (pure `net/http`). Config uses `forge.lthn.ai/core/go` for `pkg/config` and `pkg/log` — same as go-scm. - -**Key insight:** 91% of endpoints are generic CRUD (List/Get/Create/Update/Delete). The generic `Resource[T,C,U]` pattern means each service is a struct definition + path constant + optional action methods. The code generator handles 229 type definitions. - -**Test command:** `go test ./...` from the repo root. - -**The forge remote for this repo will be:** `ssh://git@forge.lthn.ai:2223/core/go-forge.git` - ---- - -## Wave 1: Foundation (Tasks 1-6) - -### Task 1: Repo scaffolding + go.mod - -**Files:** -- Create: `go.mod` -- Create: `go.sum` (auto-generated) -- Create: `doc.go` -- Create: `testdata/swagger.v1.json` (downloaded) - -**Step 1: Create directory and initialise module** - -```bash -mkdir -p /Users/snider/Code/go-forge/testdata -cd /Users/snider/Code/go-forge -git init -go mod init forge.lthn.ai/core/go-forge -``` - -**Step 2: Download and pin swagger spec** - -```bash -curl -s https://forge.lthn.ai/swagger.v1.json > testdata/swagger.v1.json -``` - -Verify: `python3 -c "import json; d=json.load(open('testdata/swagger.v1.json')); print(f'{len(d[\"definitions\"])} types, {len(d[\"paths\"])} paths')"` -Expected: `229 types, 284 paths` - -**Step 3: Write doc.go** - -```go -// Package forge provides a full-coverage Go client for the Forgejo API. -// -// Usage: -// -// f := forge.NewForge("https://forge.lthn.ai", "your-token") -// repos, err := f.Repos.List(ctx, forge.Params{"org": "core"}, forge.DefaultList) -// -// Types are generated from Forgejo's swagger.v1.json spec via cmd/forgegen/. -// Run `go generate ./types/...` to regenerate after a Forgejo upgrade. -package forge -``` - -**Step 4: Commit** - -```bash -git add -A -git commit -m "feat: scaffold go-forge repo with pinned swagger spec - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: HTTP Client - -**Files:** -- Create: `client.go` -- Create: `client_test.go` - -**Step 1: Write client tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" -) - -func TestClient_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodGet { - t.Errorf("expected GET, got %s", r.Method) - } - if r.Header.Get("Authorization") != "token test-token" { - t.Errorf("missing auth header") - } - if r.URL.Path != "/api/v1/user" { - t.Errorf("wrong path: %s", r.URL.Path) - } - json.NewEncoder(w).Encode(map[string]string{"login": "virgil"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - var out map[string]string - err := c.Get(context.Background(), "/api/v1/user", &out) - if err != nil { - t.Fatal(err) - } - if out["login"] != "virgil" { - t.Errorf("got login=%q", out["login"]) - } -} - -func TestClient_Good_Post(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - var body map[string]string - json.NewDecoder(r.Body).Decode(&body) - if body["name"] != "test-repo" { - t.Errorf("wrong body: %v", body) - } - w.WriteHeader(http.StatusCreated) - json.NewEncoder(w).Encode(map[string]any{"id": 1, "name": "test-repo"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - body := map[string]string{"name": "test-repo"} - var out map[string]any - err := c.Post(context.Background(), "/api/v1/orgs/core/repos", body, &out) - if err != nil { - t.Fatal(err) - } - if out["name"] != "test-repo" { - t.Errorf("got name=%v", out["name"]) - } -} - -func TestClient_Good_Delete(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodDelete { - t.Errorf("expected DELETE, got %s", r.Method) - } - w.WriteHeader(http.StatusNoContent) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Delete(context.Background(), "/api/v1/repos/core/test") - if err != nil { - t.Fatal(err) - } -} - -func TestClient_Bad_ServerError(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(http.StatusInternalServerError) - json.NewEncoder(w).Encode(map[string]string{"message": "internal error"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Get(context.Background(), "/api/v1/user", nil) - if err == nil { - t.Fatal("expected error") - } - var apiErr *APIError - if !errors.As(err, &apiErr) { - t.Fatalf("expected APIError, got %T", err) - } - if apiErr.StatusCode != 500 { - t.Errorf("got status=%d", apiErr.StatusCode) - } -} - -func TestClient_Bad_NotFound(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(http.StatusNotFound) - json.NewEncoder(w).Encode(map[string]string{"message": "not found"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Get(context.Background(), "/api/v1/repos/x/y", nil) - if !IsNotFound(err) { - t.Fatalf("expected not found, got %v", err) - } -} - -func TestClient_Good_ContextCancellation(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - <-r.Context().Done() - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - ctx, cancel := context.WithCancel(context.Background()) - cancel() // cancel immediately - err := c.Get(ctx, "/api/v1/user", nil) - if err == nil { - t.Fatal("expected error from cancelled context") - } -} - -func TestClient_Good_Options(t *testing.T) { - c := NewClient("https://forge.lthn.ai", "tok", - WithUserAgent("go-forge/1.0"), - ) - if c.userAgent != "go-forge/1.0" { - t.Errorf("got user agent=%q", c.userAgent) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestClient` -Expected: Compilation errors (types don't exist yet) - -**Step 3: Write client.go** - -```go -package forge - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "io" - "net/http" - "strings" -) - -// APIError represents an error response from the Forgejo API. -type APIError struct { - StatusCode int - Message string - URL string -} - -func (e *APIError) Error() string { - return fmt.Sprintf("forge: %s %d: %s", e.URL, e.StatusCode, e.Message) -} - -// IsNotFound returns true if the error is a 404 response. -func IsNotFound(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusNotFound -} - -// IsForbidden returns true if the error is a 403 response. -func IsForbidden(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusForbidden -} - -// IsConflict returns true if the error is a 409 response. -func IsConflict(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusConflict -} - -// Option configures the Client. -type Option func(*Client) - -// WithHTTPClient sets a custom http.Client. -func WithHTTPClient(hc *http.Client) Option { - return func(c *Client) { c.httpClient = hc } -} - -// WithUserAgent sets the User-Agent header. -func WithUserAgent(ua string) Option { - return func(c *Client) { c.userAgent = ua } -} - -// Client is a low-level HTTP client for the Forgejo API. -type Client struct { - baseURL string - token string - httpClient *http.Client - userAgent string -} - -// NewClient creates a new Forgejo API client. -func NewClient(url, token string, opts ...Option) *Client { - c := &Client{ - baseURL: strings.TrimRight(url, "/"), - token: token, - httpClient: http.DefaultClient, - userAgent: "go-forge/0.1", - } - for _, opt := range opts { - opt(c) - } - return c -} - -// Get performs a GET request. -func (c *Client) Get(ctx context.Context, path string, out any) error { - return c.do(ctx, http.MethodGet, path, nil, out) -} - -// Post performs a POST request. -func (c *Client) Post(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPost, path, body, out) -} - -// Patch performs a PATCH request. -func (c *Client) Patch(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPatch, path, body, out) -} - -// Put performs a PUT request. -func (c *Client) Put(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPut, path, body, out) -} - -// Delete performs a DELETE request. -func (c *Client) Delete(ctx context.Context, path string) error { - return c.do(ctx, http.MethodDelete, path, nil, nil) -} - -func (c *Client) do(ctx context.Context, method, path string, body, out any) error { - url := c.baseURL + path - - var bodyReader io.Reader - if body != nil { - data, err := json.Marshal(body) - if err != nil { - return fmt.Errorf("forge: marshal body: %w", err) - } - bodyReader = bytes.NewReader(data) - } - - req, err := http.NewRequestWithContext(ctx, method, url, bodyReader) - if err != nil { - return fmt.Errorf("forge: create request: %w", err) - } - - req.Header.Set("Authorization", "token "+c.token) - req.Header.Set("Accept", "application/json") - if body != nil { - req.Header.Set("Content-Type", "application/json") - } - if c.userAgent != "" { - req.Header.Set("User-Agent", c.userAgent) - } - - resp, err := c.httpClient.Do(req) - if err != nil { - return fmt.Errorf("forge: request %s %s: %w", method, path, err) - } - defer resp.Body.Close() - - if resp.StatusCode >= 400 { - return c.parseError(resp, path) - } - - if out != nil && resp.StatusCode != http.StatusNoContent { - if err := json.NewDecoder(resp.Body).Decode(out); err != nil { - return fmt.Errorf("forge: decode response: %w", err) - } - } - - return nil -} - -func (c *Client) parseError(resp *http.Response, path string) error { - var errBody struct { - Message string `json:"message"` - } - _ = json.NewDecoder(resp.Body).Decode(&errBody) - return &APIError{ - StatusCode: resp.StatusCode, - Message: errBody.Message, - URL: path, - } -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestClient` -Expected: All 7 tests PASS - -**Step 5: Commit** - -```bash -git add client.go client_test.go -git commit -m "feat: HTTP client with auth, context, error handling - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: Pagination - -**Files:** -- Create: `pagination.go` -- Create: `pagination_test.go` - -**Step 1: Write pagination tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "fmt" - "net/http" - "net/http/httptest" - "strconv" - "testing" -) - -func TestPagination_Good_SinglePage(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "2") - json.NewEncoder(w).Encode([]map[string]int{{"id": 1}, {"id": 2}}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 2 { - t.Errorf("got %d items", len(result)) - } -} - -func TestPagination_Good_MultiPage(t *testing.T) { - page := 0 - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - page++ - w.Header().Set("X-Total-Count", "100") - items := make([]map[string]int, 50) - for i := range items { - items[i] = map[string]int{"id": (page-1)*50 + i + 1} - } - json.NewEncoder(w).Encode(items) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 100 { - t.Errorf("got %d items, want 100", len(result)) - } -} - -func TestPagination_Good_EmptyResult(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "0") - json.NewEncoder(w).Encode([]map[string]int{}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 0 { - t.Errorf("got %d items", len(result)) - } -} - -func TestListPage_Good_QueryParams(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - p := r.URL.Query().Get("page") - l := r.URL.Query().Get("limit") - s := r.URL.Query().Get("state") - if p != "2" || l != "25" || s != "open" { - t.Errorf("wrong params: page=%s limit=%s state=%s", p, l, s) - } - w.Header().Set("X-Total-Count", "50") - json.NewEncoder(w).Encode([]map[string]int{}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - _, err := ListPage[map[string]int](context.Background(), c, "/api/v1/repos", - map[string]string{"state": "open"}, ListOptions{Page: 2, Limit: 25}) - if err != nil { - t.Fatal(err) - } -} - -func TestPagination_Bad_ServerError(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(500) - json.NewEncoder(w).Encode(map[string]string{"message": "fail"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - _, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err == nil { - t.Fatal("expected error") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestPagination -run TestListPage` -Expected: Compilation errors - -**Step 3: Write pagination.go** - -```go -package forge - -import ( - "context" - "encoding/json" - "fmt" - "net/http" - "net/url" - "strconv" -) - -// ListOptions controls pagination. -type ListOptions struct { - Page int // 1-based page number - Limit int // items per page (default 50) -} - -// DefaultList returns sensible default pagination. -var DefaultList = ListOptions{Page: 1, Limit: 50} - -// PagedResult holds a single page of results with metadata. -type PagedResult[T any] struct { - Items []T - TotalCount int - Page int - HasMore bool -} - -// ListPage fetches a single page of results. -// Extra query params can be passed via the query map. -func ListPage[T any](ctx context.Context, c *Client, path string, query map[string]string, opts ListOptions) (*PagedResult[T], error) { - if opts.Page < 1 { - opts.Page = 1 - } - if opts.Limit < 1 { - opts.Limit = 50 - } - - u, err := url.Parse(c.baseURL + path) - if err != nil { - return nil, fmt.Errorf("forge: parse url: %w", err) - } - - q := u.Query() - q.Set("page", strconv.Itoa(opts.Page)) - q.Set("limit", strconv.Itoa(opts.Limit)) - for k, v := range query { - q.Set(k, v) - } - u.RawQuery = q.Encode() - - req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil) - if err != nil { - return nil, fmt.Errorf("forge: create request: %w", err) - } - - req.Header.Set("Authorization", "token "+c.token) - req.Header.Set("Accept", "application/json") - if c.userAgent != "" { - req.Header.Set("User-Agent", c.userAgent) - } - - resp, err := c.httpClient.Do(req) - if err != nil { - return nil, fmt.Errorf("forge: request GET %s: %w", path, err) - } - defer resp.Body.Close() - - if resp.StatusCode >= 400 { - return nil, c.parseError(resp, path) - } - - var items []T - if err := json.NewDecoder(resp.Body).Decode(&items); err != nil { - return nil, fmt.Errorf("forge: decode response: %w", err) - } - - totalCount, _ := strconv.Atoi(resp.Header.Get("X-Total-Count")) - - return &PagedResult[T]{ - Items: items, - TotalCount: totalCount, - Page: opts.Page, - HasMore: len(items) >= opts.Limit && opts.Page*opts.Limit < totalCount, - }, nil -} - -// ListAll fetches all pages of results. -func ListAll[T any](ctx context.Context, c *Client, path string, query map[string]string) ([]T, error) { - var all []T - page := 1 - - for { - result, err := ListPage[T](ctx, c, path, query, ListOptions{Page: page, Limit: 50}) - if err != nil { - return nil, err - } - all = append(all, result.Items...) - if !result.HasMore { - break - } - page++ - } - - return all, nil -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestPagination|TestListPage"` -Expected: All 5 tests PASS - -**Step 5: Commit** - -```bash -git add pagination.go pagination_test.go -git commit -m "feat: generic pagination with ListAll and ListPage - -Co-Authored-By: Virgil " -``` - ---- - -### Task 4: Params and path resolution - -**Files:** -- Create: `params.go` -- Create: `params_test.go` - -**Step 1: Write tests** - -```go -package forge - -import "testing" - -func TestResolvePath_Good_Simple(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}", Params{"owner": "core", "repo": "go-forge"}) - want := "/api/v1/repos/core/go-forge" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} - -func TestResolvePath_Good_NoParams(t *testing.T) { - got := ResolvePath("/api/v1/user", nil) - if got != "/api/v1/user" { - t.Errorf("got %q", got) - } -} - -func TestResolvePath_Good_WithID(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}/issues/{index}", Params{ - "owner": "core", "repo": "go-forge", "index": "42", - }) - want := "/api/v1/repos/core/go-forge/issues/42" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} - -func TestResolvePath_Good_URLEncoding(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}", Params{"owner": "my org", "repo": "my repo"}) - want := "/api/v1/repos/my%20org/my%20repo" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolvePath` -Expected: Compilation errors - -**Step 3: Write params.go** - -```go -package forge - -import ( - "net/url" - "strings" -) - -// Params maps path variable names to values. -// Example: Params{"owner": "core", "repo": "go-forge"} -type Params map[string]string - -// ResolvePath substitutes {placeholders} in path with values from params. -func ResolvePath(path string, params Params) string { - for k, v := range params { - path = strings.ReplaceAll(path, "{"+k+"}", url.PathEscape(v)) - } - return path -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolvePath` -Expected: All 4 tests PASS - -**Step 5: Commit** - -```bash -git add params.go params_test.go -git commit -m "feat: path parameter resolution with URL encoding - -Co-Authored-By: Virgil " -``` - ---- - -### Task 5: Generic Resource[T, C, U] - -**Files:** -- Create: `resource.go` -- Create: `resource_test.go` - -**Step 1: Write resource tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" -) - -// Test types -type testItem struct { - ID int `json:"id"` - Name string `json:"name"` -} - -type testCreate struct { - Name string `json:"name"` -} - -type testUpdate struct { - Name *string `json:"name,omitempty"` -} - -func TestResource_Good_List(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/v1/orgs/core/repos" { - t.Errorf("wrong path: %s", r.URL.Path) - } - w.Header().Set("X-Total-Count", "2") - json.NewEncoder(w).Encode([]testItem{{1, "a"}, {2, "b"}}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/orgs/{org}/repos") - - items, err := res.List(context.Background(), Params{"org": "core"}, DefaultList) - if err != nil { - t.Fatal(err) - } - if len(items.Items) != 2 { - t.Errorf("got %d items", len(items.Items)) - } -} - -func TestResource_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/v1/repos/core/go-forge" { - t.Errorf("wrong path: %s", r.URL.Path) - } - json.NewEncoder(w).Encode(testItem{1, "go-forge"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - item, err := res.Get(context.Background(), Params{"owner": "core", "repo": "go-forge"}) - if err != nil { - t.Fatal(err) - } - if item.Name != "go-forge" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Create(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - var body testCreate - json.NewDecoder(r.Body).Decode(&body) - w.WriteHeader(http.StatusCreated) - json.NewEncoder(w).Encode(testItem{1, body.Name}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/orgs/{org}/repos") - - item, err := res.Create(context.Background(), Params{"org": "core"}, &testCreate{Name: "new-repo"}) - if err != nil { - t.Fatal(err) - } - if item.Name != "new-repo" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Update(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPatch { - t.Errorf("expected PATCH, got %s", r.Method) - } - json.NewEncoder(w).Encode(testItem{1, "updated"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - name := "updated" - item, err := res.Update(context.Background(), Params{"owner": "core", "repo": "old"}, &testUpdate{Name: &name}) - if err != nil { - t.Fatal(err) - } - if item.Name != "updated" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Delete(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodDelete { - t.Errorf("expected DELETE, got %s", r.Method) - } - w.WriteHeader(http.StatusNoContent) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - err := res.Delete(context.Background(), Params{"owner": "core", "repo": "old"}) - if err != nil { - t.Fatal(err) - } -} - -func TestResource_Good_ListAll(t *testing.T) { - page := 0 - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - page++ - w.Header().Set("X-Total-Count", "3") - if page == 1 { - json.NewEncoder(w).Encode([]testItem{{1, "a"}, {2, "b"}}) - } else { - json.NewEncoder(w).Encode([]testItem{{3, "c"}}) - } - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos") - - items, err := res.ListAll(context.Background(), nil) - if err != nil { - t.Fatal(err) - } - if len(items) != 3 { - t.Errorf("got %d items, want 3", len(items)) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResource` -Expected: Compilation errors - -**Step 3: Write resource.go** - -```go -package forge - -import "context" - -// Resource provides generic CRUD operations for a Forgejo API resource. -// T is the resource type, C is the create options type, U is the update options type. -type Resource[T any, C any, U any] struct { - client *Client - path string -} - -// NewResource creates a new Resource for the given path pattern. -// The path may contain {placeholders} that are resolved via Params. -func NewResource[T any, C any, U any](c *Client, path string) *Resource[T, C, U] { - return &Resource[T, C, U]{client: c, path: path} -} - -// List returns a single page of resources. -func (r *Resource[T, C, U]) List(ctx context.Context, params Params, opts ListOptions) (*PagedResult[T], error) { - return ListPage[T](ctx, r.client, ResolvePath(r.path, params), nil, opts) -} - -// ListAll returns all resources across all pages. -func (r *Resource[T, C, U]) ListAll(ctx context.Context, params Params) ([]T, error) { - return ListAll[T](ctx, r.client, ResolvePath(r.path, params), nil) -} - -// Get returns a single resource by appending id to the path. -func (r *Resource[T, C, U]) Get(ctx context.Context, params Params) (*T, error) { - var out T - if err := r.client.Get(ctx, ResolvePath(r.path, params), &out); err != nil { - return nil, err - } - return &out, nil -} - -// Create creates a new resource. -func (r *Resource[T, C, U]) Create(ctx context.Context, params Params, body *C) (*T, error) { - var out T - if err := r.client.Post(ctx, ResolvePath(r.path, params), body, &out); err != nil { - return nil, err - } - return &out, nil -} - -// Update modifies an existing resource. -func (r *Resource[T, C, U]) Update(ctx context.Context, params Params, body *U) (*T, error) { - var out T - if err := r.client.Patch(ctx, ResolvePath(r.path, params), body, &out); err != nil { - return nil, err - } - return &out, nil -} - -// Delete removes a resource. -func (r *Resource[T, C, U]) Delete(ctx context.Context, params Params) error { - return r.client.Delete(ctx, ResolvePath(r.path, params)) -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResource` -Expected: All 6 tests PASS - -**Step 5: Commit** - -```bash -git add resource.go resource_test.go -git commit -m "feat: generic Resource[T,C,U] for CRUD operations - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: Config resolution (extracted from go-scm) - -**Files:** -- Create: `config.go` -- Create: `config_test.go` - -**Step 1: Write config tests** - -```go -package forge - -import ( - "os" - "testing" -) - -func TestResolveConfig_Good_EnvOverrides(t *testing.T) { - t.Setenv("FORGE_URL", "https://forge.example.com") - t.Setenv("FORGE_TOKEN", "env-token") - - url, token, err := ResolveConfig("", "") - if err != nil { - t.Fatal(err) - } - if url != "https://forge.example.com" { - t.Errorf("got url=%q", url) - } - if token != "env-token" { - t.Errorf("got token=%q", token) - } -} - -func TestResolveConfig_Good_FlagOverridesEnv(t *testing.T) { - t.Setenv("FORGE_URL", "https://env.example.com") - t.Setenv("FORGE_TOKEN", "env-token") - - url, token, err := ResolveConfig("https://flag.example.com", "flag-token") - if err != nil { - t.Fatal(err) - } - if url != "https://flag.example.com" { - t.Errorf("got url=%q", url) - } - if token != "flag-token" { - t.Errorf("got token=%q", token) - } -} - -func TestResolveConfig_Good_DefaultURL(t *testing.T) { - // Clear env vars to test defaults - os.Unsetenv("FORGE_URL") - os.Unsetenv("FORGE_TOKEN") - - url, _, err := ResolveConfig("", "") - if err != nil { - t.Fatal(err) - } - if url != DefaultURL { - t.Errorf("got url=%q, want %q", url, DefaultURL) - } -} - -func TestNewForgeFromConfig_Bad_NoToken(t *testing.T) { - os.Unsetenv("FORGE_URL") - os.Unsetenv("FORGE_TOKEN") - - _, err := NewForgeFromConfig("", "") - if err == nil { - t.Fatal("expected error for missing token") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolveConfig -run TestNewForgeFromConfig` -Expected: Compilation errors - -**Step 3: Write config.go** - -```go -package forge - -import ( - "fmt" - "os" -) - -const ( - // DefaultURL is used when no URL is configured. - DefaultURL = "http://localhost:3000" -) - -// ResolveConfig resolves Forge URL and token from multiple sources. -// Priority (highest to lowest): flags → environment → defaults. -func ResolveConfig(flagURL, flagToken string) (url, token string, err error) { - // Environment variables - url = os.Getenv("FORGE_URL") - token = os.Getenv("FORGE_TOKEN") - - // Flag overrides - if flagURL != "" { - url = flagURL - } - if flagToken != "" { - token = flagToken - } - - // Default URL - if url == "" { - url = DefaultURL - } - - return url, token, nil -} - -// NewForgeFromConfig creates a Forge client using resolved configuration. -func NewForgeFromConfig(flagURL, flagToken string, opts ...Option) (*Forge, error) { - url, token, err := ResolveConfig(flagURL, flagToken) - if err != nil { - return nil, err - } - if token == "" { - return nil, fmt.Errorf("forge: no API token configured (set FORGE_TOKEN or pass --token)") - } - return NewForge(url, token, opts...), nil -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestResolveConfig|TestNewForgeFromConfig"` -Expected: All 4 tests PASS (Note: `NewForge` doesn't exist yet — if this fails, create a stub `NewForge` function that just returns `&Forge{client: NewClient(url, token, opts...)}`) - -**Step 5: Commit** - -```bash -git add config.go config_test.go -git commit -m "feat: config resolution from env vars and flags - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 2: Code Generator (Tasks 7-9) - -### Task 7: Swagger spec parser - -**Files:** -- Create: `cmd/forgegen/main.go` -- Create: `cmd/forgegen/parser.go` -- Create: `cmd/forgegen/parser_test.go` - -The parser reads swagger.v1.json and extracts type definitions into an intermediate representation. - -**Step 1: Write parser tests** - -```go -package main - -import ( - "os" - "testing" -) - -func TestParser_Good_LoadSpec(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - if spec.Swagger != "2.0" { - t.Errorf("got swagger=%q", spec.Swagger) - } - if len(spec.Definitions) < 200 { - t.Errorf("got %d definitions, expected 200+", len(spec.Definitions)) - } -} - -func TestParser_Good_ExtractTypes(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - if len(types) < 200 { - t.Errorf("got %d types", len(types)) - } - - // Check a known type - repo, ok := types["Repository"] - if !ok { - t.Fatal("Repository type not found") - } - if len(repo.Fields) < 50 { - t.Errorf("Repository has %d fields, expected 50+", len(repo.Fields)) - } -} - -func TestParser_Good_FieldTypes(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - repo := types["Repository"] - - // Check specific field mappings - for _, f := range repo.Fields { - switch f.JSONName { - case "id": - if f.GoType != "int64" { - t.Errorf("id: got %q, want int64", f.GoType) - } - case "name": - if f.GoType != "string" { - t.Errorf("name: got %q, want string", f.GoType) - } - case "private": - if f.GoType != "bool" { - t.Errorf("private: got %q, want bool", f.GoType) - } - case "created_at": - if f.GoType != "time.Time" { - t.Errorf("created_at: got %q, want time.Time", f.GoType) - } - case "owner": - if f.GoType != "*User" { - t.Errorf("owner: got %q, want *User", f.GoType) - } - } - } -} - -func TestParser_Good_DetectCreateEditPairs(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - pairs := DetectCRUDPairs(spec) - // Should find Repository, Issue, PullRequest, etc. - if len(pairs) < 10 { - t.Errorf("got %d pairs, expected 10+", len(pairs)) - } - - found := false - for _, p := range pairs { - if p.Base == "Repository" { - found = true - if p.Create != "CreateRepoOption" { - t.Errorf("repo create=%q", p.Create) - } - } - } - if !found { - t.Fatal("Repository pair not found") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestParser` -Expected: Compilation errors - -**Step 3: Write parser.go** - -```go -package main - -import ( - "encoding/json" - "fmt" - "os" - "sort" - "strings" -) - -// Spec represents a Swagger 2.0 specification. -type Spec struct { - Swagger string `json:"swagger"` - Info SpecInfo `json:"info"` - Definitions map[string]SchemaDefinition `json:"definitions"` - Paths map[string]map[string]any `json:"paths"` -} - -type SpecInfo struct { - Title string `json:"title"` - Version string `json:"version"` -} - -// SchemaDefinition represents a type definition in the spec. -type SchemaDefinition struct { - Description string `json:"description"` - Type string `json:"type"` - Properties map[string]SchemaProperty `json:"properties"` - Required []string `json:"required"` - Enum []any `json:"enum"` - XGoName string `json:"x-go-name"` -} - -// SchemaProperty represents a field in a type definition. -type SchemaProperty struct { - Type string `json:"type"` - Format string `json:"format"` - Description string `json:"description"` - Ref string `json:"$ref"` - Items *SchemaProperty `json:"items"` - Enum []any `json:"enum"` - XGoName string `json:"x-go-name"` -} - -// GoType represents a Go type extracted from the spec. -type GoType struct { - Name string - Description string - Fields []GoField - IsEnum bool - EnumValues []string -} - -// GoField represents a field in a Go struct. -type GoField struct { - GoName string - GoType string - JSONName string - Comment string - Required bool -} - -// CRUDPair maps a base type to its Create and Edit option types. -type CRUDPair struct { - Base string // e.g. "Repository" - Create string // e.g. "CreateRepoOption" - Edit string // e.g. "EditRepoOption" -} - -// LoadSpec reads and parses a Swagger 2.0 JSON file. -func LoadSpec(path string) (*Spec, error) { - data, err := os.ReadFile(path) - if err != nil { - return nil, fmt.Errorf("read spec: %w", err) - } - var spec Spec - if err := json.Unmarshal(data, &spec); err != nil { - return nil, fmt.Errorf("parse spec: %w", err) - } - return &spec, nil -} - -// ExtractTypes converts spec definitions to Go types. -func ExtractTypes(spec *Spec) map[string]*GoType { - result := make(map[string]*GoType) - - for name, def := range spec.Definitions { - gt := &GoType{ - Name: name, - Description: def.Description, - } - - if len(def.Enum) > 0 { - gt.IsEnum = true - for _, v := range def.Enum { - gt.EnumValues = append(gt.EnumValues, fmt.Sprintf("%v", v)) - } - sort.Strings(gt.EnumValues) - result[name] = gt - continue - } - - required := make(map[string]bool) - for _, r := range def.Required { - required[r] = true - } - - for fieldName, prop := range def.Properties { - goName := prop.XGoName - if goName == "" { - goName = pascalCase(fieldName) - } - - gf := GoField{ - GoName: goName, - GoType: resolveGoType(prop), - JSONName: fieldName, - Comment: prop.Description, - Required: required[fieldName], - } - gt.Fields = append(gt.Fields, gf) - } - - // Sort fields alphabetically for stable output - sort.Slice(gt.Fields, func(i, j int) bool { - return gt.Fields[i].GoName < gt.Fields[j].GoName - }) - - result[name] = gt - } - - return result -} - -// DetectCRUDPairs finds Create/Edit option pairs. -func DetectCRUDPairs(spec *Spec) []CRUDPair { - var pairs []CRUDPair - - for name := range spec.Definitions { - if !strings.HasPrefix(name, "Create") || !strings.HasSuffix(name, "Option") { - continue - } - - // CreateXxxOption → Xxx → EditXxxOption - inner := strings.TrimPrefix(name, "Create") - inner = strings.TrimSuffix(inner, "Option") - - editName := "Edit" + inner + "Option" - - pair := CRUDPair{ - Base: inner, - Create: name, - } - - if _, ok := spec.Definitions[editName]; ok { - pair.Edit = editName - } - - pairs = append(pairs, pair) - } - - sort.Slice(pairs, func(i, j int) bool { - return pairs[i].Base < pairs[j].Base - }) - - return pairs -} - -func resolveGoType(prop SchemaProperty) string { - if prop.Ref != "" { - parts := strings.Split(prop.Ref, "/") - return "*" + parts[len(parts)-1] - } - - switch prop.Type { - case "string": - switch prop.Format { - case "date-time": - return "time.Time" - case "binary": - return "[]byte" - default: - return "string" - } - case "integer": - switch prop.Format { - case "int64": - return "int64" - case "int32": - return "int32" - default: - return "int" - } - case "number": - switch prop.Format { - case "float": - return "float32" - default: - return "float64" - } - case "boolean": - return "bool" - case "array": - if prop.Items != nil { - itemType := resolveGoType(*prop.Items) - return "[]" + itemType - } - return "[]any" - case "object": - return "map[string]any" - default: - if prop.Type == "" && prop.Ref == "" { - return "any" - } - return "any" - } -} - -func pascalCase(s string) string { - parts := strings.FieldsFunc(s, func(r rune) bool { - return r == '_' || r == '-' - }) - for i, p := range parts { - if len(p) == 0 { - continue - } - // Handle common acronyms - upper := strings.ToUpper(p) - switch upper { - case "ID", "URL", "HTML", "SSH", "HTTP", "HTTPS", "API", "URI", "GPG", "IP", "CSS", "JS": - parts[i] = upper - default: - parts[i] = strings.ToUpper(p[:1]) + p[1:] - } - } - return strings.Join(parts, "") -} -``` - -**Step 4: Write main.go stub** - -```go -package main - -import ( - "flag" - "fmt" - "os" -) - -func main() { - specPath := flag.String("spec", "testdata/swagger.v1.json", "path to swagger.v1.json") - outDir := flag.String("out", "types", "output directory for generated types") - flag.Parse() - - spec, err := LoadSpec(*specPath) - if err != nil { - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - os.Exit(1) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - fmt.Printf("Loaded %d types, %d CRUD pairs\n", len(types), len(pairs)) - fmt.Printf("Output dir: %s\n", *outDir) - - // Generation happens in Task 8 - if err := Generate(types, pairs, *outDir); err != nil { - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - os.Exit(1) - } -} -``` - -**Step 5: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestParser` -Expected: All 4 tests PASS (Note: `Generate` doesn't exist yet — add a stub: `func Generate(...) error { return nil }`) - -**Step 6: Commit** - -```bash -git add cmd/forgegen/ -git commit -m "feat: swagger spec parser for type extraction - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: Code generator — Go source emission - -**Files:** -- Create: `cmd/forgegen/generator.go` -- Create: `cmd/forgegen/generator_test.go` - -**Step 1: Write generator tests** - -```go -package main - -import ( - "os" - "path/filepath" - "strings" - "testing" -) - -func TestGenerate_Good_CreatesFiles(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Should create at least one .go file - entries, _ := os.ReadDir(outDir) - goFiles := 0 - for _, e := range entries { - if strings.HasSuffix(e.Name(), ".go") { - goFiles++ - } - } - if goFiles == 0 { - t.Fatal("no .go files generated") - } -} - -func TestGenerate_Good_ValidGoSyntax(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Read a generated file and verify basic Go syntax markers - data, err := os.ReadFile(filepath.Join(outDir, "repo.go")) - if err != nil { - // Try another name - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - if strings.HasSuffix(e.Name(), ".go") { - data, err = os.ReadFile(filepath.Join(outDir, e.Name())) - break - } - } - } - if err != nil { - t.Fatal(err) - } - - content := string(data) - if !strings.Contains(content, "package types") { - t.Error("missing package declaration") - } - if !strings.Contains(content, "// Code generated") { - t.Error("missing generated comment") - } -} - -func TestGenerate_Good_RepositoryType(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Find file containing Repository type - var content string - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - data, _ := os.ReadFile(filepath.Join(outDir, e.Name())) - if strings.Contains(string(data), "type Repository struct") { - content = string(data) - break - } - } - - if content == "" { - t.Fatal("Repository type not found in any generated file") - } - - // Check essential fields exist - checks := []string{ - "`json:\"id\"`", - "`json:\"name\"`", - "`json:\"full_name\"`", - "`json:\"private\"`", - } - for _, check := range checks { - if !strings.Contains(content, check) { - t.Errorf("missing field with tag %s", check) - } - } -} - -func TestGenerate_Good_TimeImport(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Files with time.Time fields should import "time" - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - data, _ := os.ReadFile(filepath.Join(outDir, e.Name())) - content := string(data) - if strings.Contains(content, "time.Time") && !strings.Contains(content, "\"time\"") { - t.Errorf("file %s uses time.Time but doesn't import time", e.Name()) - } - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestGenerate` -Expected: Failures (Generate is stub) - -**Step 3: Write generator.go** - -The generator groups types by logical domain and writes one `.go` file per group. Type grouping uses name prefixes and the CRUD pairs. - -```go -package main - -import ( - "fmt" - "os" - "path/filepath" - "sort" - "strings" - "text/template" -) - -// typeGrouping maps types to their output file. -var typeGrouping = map[string]string{ - "Repository": "repo", - "Repo": "repo", - "Issue": "issue", - "PullRequest": "pr", - "Pull": "pr", - "User": "user", - "Organization": "org", - "Org": "org", - "Team": "team", - "Label": "label", - "Milestone": "milestone", - "Release": "release", - "Tag": "tag", - "Branch": "branch", - "Hook": "hook", - "Deploy": "key", - "PublicKey": "key", - "GPGKey": "key", - "Key": "key", - "Notification": "notification", - "Package": "package", - "Action": "action", - "Commit": "commit", - "Git": "git", - "Contents": "content", - "File": "content", - "Wiki": "wiki", - "Comment": "comment", - "Review": "review", - "Reaction": "reaction", - "Topic": "topic", - "Status": "status", - "Combined": "status", - "Cron": "admin", - "Quota": "quota", - "OAuth2": "oauth", - "AccessToken": "oauth", - "API": "error", - "Forbidden": "error", - "NotFound": "error", - "NodeInfo": "federation", - "Activity": "activity", - "Feed": "activity", - "StopWatch": "time_tracking", - "TrackedTime": "time_tracking", - "Blocked": "user", - "Email": "user", - "Settings": "settings", - "GeneralAPI": "settings", - "GeneralAttachment": "settings", - "GeneralRepo": "settings", - "GeneralUI": "settings", - "Markdown": "misc", - "Markup": "misc", - "License": "misc", - "Gitignore": "misc", - "Annotated": "git", - "Note": "git", - "ChangedFile": "git", - "ExternalTracker": "repo", - "ExternalWiki": "repo", - "InternalTracker": "repo", - "Permission": "common", - "RepoTransfer": "repo", - "PayloadCommit": "hook", - "Dispatch": "action", - "Secret": "action", - "Variable": "action", - "Push": "repo", - "Mirror": "repo", - "Attachment": "common", - "EditDeadline": "issue", - "IssueDeadline": "issue", - "IssueLabels": "issue", - "IssueMeta": "issue", - "IssueTemplate": "issue", - "StateType": "common", - "TimeStamp": "common", - "Rename": "admin", - "Unadopted": "admin", -} - -// classifyType determines which file a type belongs in. -func classifyType(name string) string { - // Direct match - if group, ok := typeGrouping[name]; ok { - return group - } - - // Prefix match (longest first) - for prefix, group := range typeGrouping { - if strings.HasPrefix(name, prefix) { - return group - } - } - - // Try common suffixes - if strings.HasSuffix(name, "Option") || strings.HasSuffix(name, "Options") { - // Strip Create/Edit prefix to find base - trimmed := name - trimmed = strings.TrimPrefix(trimmed, "Create") - trimmed = strings.TrimPrefix(trimmed, "Edit") - trimmed = strings.TrimPrefix(trimmed, "Delete") - trimmed = strings.TrimPrefix(trimmed, "Update") - trimmed = strings.TrimSuffix(trimmed, "Option") - trimmed = strings.TrimSuffix(trimmed, "Options") - if group, ok := typeGrouping[trimmed]; ok { - return group - } - } - - return "misc" -} - -// Generate writes Go source files for all types. -func Generate(types map[string]*GoType, pairs []CRUDPair, outDir string) error { - if err := os.MkdirAll(outDir, 0755); err != nil { - return fmt.Errorf("create output dir: %w", err) - } - - // Group types by file - groups := make(map[string][]*GoType) - for _, gt := range types { - file := classifyType(gt.Name) - groups[file] = append(groups[file], gt) - } - - // Sort types within each group - for file := range groups { - sort.Slice(groups[file], func(i, j int) bool { - return groups[file][i].Name < groups[file][j].Name - }) - } - - // Write each file - for file, fileTypes := range groups { - if err := writeFile(filepath.Join(outDir, file+".go"), fileTypes); err != nil { - return fmt.Errorf("write %s.go: %w", file, err) - } - } - - return nil -} - -var fileTmpl = template.Must(template.New("file").Parse(`// Code generated by forgegen from swagger.v1.json — DO NOT EDIT. - -package types -{{if .NeedsTime}} -import "time" -{{end}} -{{range .Types}} -{{if .Description}}// {{.Name}} — {{.Description}}{{else}}// {{.Name}} represents a Forgejo API type.{{end}} -{{if .IsEnum}}type {{.Name}} string - -const ( -{{range .EnumValues}} {{$.EnumConst .Name .}} {{$.EnumType .Name}} = "{{.}}" -{{end}}) -{{else}}type {{.Name}} struct { -{{range .Fields}} {{.GoName}} {{.GoType}} ` + "`" + `json:"{{.JSONName}}{{if not .Required}},omitempty{{end}}"` + "`" + `{{if .Comment}} // {{.Comment}}{{end}} -{{end}}} -{{end}} -{{end}}`)) - -type fileData struct { - Types []*GoType - NeedsTime bool -} - -func (fd fileData) EnumConst(typeName, value string) string { - return typeName + pascalCase(value) -} - -func (fd fileData) EnumType(typeName string) string { - return typeName -} - -func writeFile(path string, types []*GoType) error { - needsTime := false - for _, gt := range types { - for _, f := range gt.Fields { - if strings.Contains(f.GoType, "time.Time") { - needsTime = true - break - } - } - if needsTime { - break - } - } - - f, err := os.Create(path) - if err != nil { - return err - } - defer f.Close() - - return fileTmpl.Execute(f, fileData{ - Types: types, - NeedsTime: needsTime, - }) -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestGenerate` -Expected: All 4 tests PASS - -**Step 5: Commit** - -```bash -git add cmd/forgegen/generator.go cmd/forgegen/generator_test.go -git commit -m "feat: Go source code generator from Swagger types - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Generate types + verify compilation - -**Files:** -- Create: `types/` directory with generated files -- Create: `types/generate.go` (go:generate directive) - -**Step 1: Run the generator** - -```bash -cd /Users/snider/Code/go-forge -mkdir -p types -go run ./cmd/forgegen/ -spec testdata/swagger.v1.json -out types/ -``` - -**Step 2: Add go:generate directive** - -Create `types/generate.go`: -```go -package types - -//go:generate go run ../cmd/forgegen/ -spec ../testdata/swagger.v1.json -out . -``` - -**Step 3: Verify compilation** - -Run: `cd /Users/snider/Code/go-forge && go build ./types/` -Expected: Compiles without errors - -If there are compilation errors, fix the generator (`cmd/forgegen/generator.go`) and regenerate. Common issues: -- Missing imports (time) -- Duplicate field names (GoName collision) -- Invalid Go identifiers (reserved words, starting with numbers) - -**Step 4: Run all tests** - -Run: `cd /Users/snider/Code/go-forge && go test ./...` -Expected: All tests pass - -**Step 5: Commit** - -```bash -git add types/ -git commit -m "feat: generate all 229 Forgejo API types from swagger spec - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 3: Core Services (Tasks 10-13) - -Each service follows the same pattern: embed `Resource[T,C,U]`, add action methods. The first service (Task 10) is fully detailed as a template. Subsequent services follow the same structure with less repetition. - -### Task 10: Forge client + RepoService (template service) - -**Files:** -- Create: `forge.go` -- Create: `repos.go` -- Create: `forge_test.go` - -**Step 1: Write tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" - - "forge.lthn.ai/core/go-forge/types" -) - -func TestForge_Good_NewForge(t *testing.T) { - f := NewForge("https://forge.lthn.ai", "tok") - if f.Repos == nil { - t.Fatal("Repos service is nil") - } - if f.Issues == nil { - t.Fatal("Issues service is nil") - } -} - -func TestRepoService_Good_List(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "1") - json.NewEncoder(w).Encode([]types.Repository{{Name: "go-forge"}}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - result, err := f.Repos.List(context.Background(), Params{"org": "core"}, DefaultList) - if err != nil { - t.Fatal(err) - } - if len(result.Items) != 1 || result.Items[0].Name != "go-forge" { - t.Errorf("unexpected result: %+v", result) - } -} - -func TestRepoService_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - json.NewEncoder(w).Encode(types.Repository{Name: "go-forge", FullName: "core/go-forge"}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - repo, err := f.Repos.Get(context.Background(), Params{"owner": "core", "repo": "go-forge"}) - if err != nil { - t.Fatal(err) - } - if repo.Name != "go-forge" { - t.Errorf("got name=%q", repo.Name) - } -} - -func TestRepoService_Good_Fork(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - w.WriteHeader(http.StatusAccepted) - json.NewEncoder(w).Encode(types.Repository{Name: "go-forge", Fork: true}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - repo, err := f.Repos.Fork(context.Background(), "core", "go-forge", "my-org") - if err != nil { - t.Fatal(err) - } - if !repo.Fork { - t.Error("expected fork=true") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestForge|TestRepoService"` -Expected: Compilation errors - -**Step 3: Write forge.go** - -```go -package forge - -import "forge.lthn.ai/core/go-forge/types" - -// Forge is the top-level client for the Forgejo API. -type Forge struct { - client *Client - - Repos *RepoService - Issues *IssueService - Pulls *PullService - Orgs *OrgService - Users *UserService - Teams *TeamService - Admin *AdminService - Branches *BranchService - Releases *ReleaseService - Labels *LabelService - Webhooks *WebhookService - Notifications *NotificationService - Packages *PackageService - Actions *ActionsService - Contents *ContentService - Wiki *WikiService - Misc *MiscService -} - -// NewForge creates a new Forge client. -func NewForge(url, token string, opts ...Option) *Forge { - c := NewClient(url, token, opts...) - f := &Forge{client: c} - f.Repos = newRepoService(c) - // Other services initialised in their respective tasks. - // Stub them here so tests compile: - f.Issues = &IssueService{} - f.Pulls = &PullService{} - f.Orgs = &OrgService{} - f.Users = &UserService{} - f.Teams = &TeamService{} - f.Admin = &AdminService{} - f.Branches = &BranchService{} - f.Releases = &ReleaseService{} - f.Labels = &LabelService{} - f.Webhooks = &WebhookService{} - f.Notifications = &NotificationService{} - f.Packages = &PackageService{} - f.Actions = &ActionsService{} - f.Contents = &ContentService{} - f.Wiki = &WikiService{} - f.Misc = &MiscService{} - return f -} - -// Client returns the underlying HTTP client. -func (f *Forge) Client() *Client { return f.client } -``` - -**Step 4: Write repos.go** - -```go -package forge - -import ( - "context" - - "forge.lthn.ai/core/go-forge/types" -) - -// RepoService handles repository operations. -type RepoService struct { - Resource[types.Repository, types.CreateRepoOption, types.EditRepoOption] -} - -func newRepoService(c *Client) *RepoService { - return &RepoService{ - Resource: *NewResource[types.Repository, types.CreateRepoOption, types.EditRepoOption]( - c, "/api/v1/repos/{owner}/{repo}", - ), - } -} - -// ListOrgRepos returns all repositories for an organisation. -func (s *RepoService) ListOrgRepos(ctx context.Context, org string) ([]types.Repository, error) { - return ListAll[types.Repository](ctx, s.client, "/api/v1/orgs/"+org+"/repos", nil) -} - -// ListUserRepos returns all repositories for the authenticated user. -func (s *RepoService) ListUserRepos(ctx context.Context) ([]types.Repository, error) { - return ListAll[types.Repository](ctx, s.client, "/api/v1/user/repos", nil) -} - -// Fork forks a repository. If org is non-empty, forks into that organisation. -func (s *RepoService) Fork(ctx context.Context, owner, repo, org string) (*types.Repository, error) { - body := map[string]string{} - if org != "" { - body["organization"] = org - } - var out types.Repository - err := s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/forks", body, &out) - if err != nil { - return nil, err - } - return &out, nil -} - -// Migrate imports a repository from an external service. -func (s *RepoService) Migrate(ctx context.Context, opts *types.MigrateRepoOptions) (*types.Repository, error) { - var out types.Repository - err := s.client.Post(ctx, "/api/v1/repos/migrate", opts, &out) - if err != nil { - return nil, err - } - return &out, nil -} - -// Transfer initiates a repository transfer. -func (s *RepoService) Transfer(ctx context.Context, owner, repo string, opts map[string]any) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer", opts, nil) -} - -// AcceptTransfer accepts a pending repository transfer. -func (s *RepoService) AcceptTransfer(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer/accept", nil, nil) -} - -// RejectTransfer rejects a pending repository transfer. -func (s *RepoService) RejectTransfer(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer/reject", nil, nil) -} - -// MirrorSync triggers a mirror sync. -func (s *RepoService) MirrorSync(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/mirror-sync", nil, nil) -} -``` - -**Step 5: Write stub service types** so `forge.go` compiles. Create `services_stub.go`: - -```go -package forge - -// Stub service types — replaced as each service is implemented. - -type IssueService struct{} -type PullService struct{} -type OrgService struct{} -type UserService struct{} -type TeamService struct{} -type AdminService struct{} -type BranchService struct{} -type ReleaseService struct{} -type LabelService struct{} -type WebhookService struct{} -type NotificationService struct{} -type PackageService struct{} -type ActionsService struct{} -type ContentService struct{} -type WikiService struct{} -type MiscService struct{} -``` - -**Step 6: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestForge|TestRepoService"` -Expected: All tests PASS (if generated types compile — if `types.CreateRepoOption` or `types.MigrateRepoOptions` don't exist, adjust field names to match generated types) - -**Step 7: Commit** - -```bash -git add forge.go repos.go services_stub.go forge_test.go -git commit -m "feat: Forge client + RepoService with CRUD and actions - -Co-Authored-By: Virgil " -``` - ---- - -### Task 11: IssueService + PullService - -**Files:** -- Create: `issues.go` -- Create: `pulls.go` -- Create: `issues_test.go` -- Create: `pulls_test.go` -- Modify: `forge.go` (wire up services) -- Modify: `services_stub.go` (remove IssueService, PullService stubs) - -Follow the same pattern as Task 10. Key points: - -**IssueService** embeds `Resource[types.Issue, types.CreateIssueOption, types.EditIssueOption]`. -Path: `/api/v1/repos/{owner}/{repo}/issues/{index}` - -Action methods (9): -- `Pin(ctx, owner, repo, index)` — POST `.../issues/{index}/pin` -- `Unpin(ctx, owner, repo, index)` — DELETE `.../issues/{index}/pin` -- `SetDeadline(ctx, owner, repo, index, deadline)` — POST `.../issues/{index}/deadline` -- `AddReaction(ctx, owner, repo, index, reaction)` — POST `.../issues/{index}/reactions` -- `DeleteReaction(ctx, owner, repo, index, reaction)` — DELETE `.../issues/{index}/reactions` -- `StartStopwatch(ctx, owner, repo, index)` — POST `.../issues/{index}/stopwatch/start` -- `StopStopwatch(ctx, owner, repo, index)` — POST `.../issues/{index}/stopwatch/stop` -- `AddLabels(ctx, owner, repo, index, labelIDs)` — POST `.../issues/{index}/labels` -- `RemoveLabel(ctx, owner, repo, index, labelID)` — DELETE `.../issues/{index}/labels/{id}` -- `ListComments(ctx, owner, repo, index)` — GET `.../issues/{index}/comments` -- `CreateComment(ctx, owner, repo, index, body)` — POST `.../issues/{index}/comments` - -**PullService** embeds `Resource[types.PullRequest, types.CreatePullRequestOption, types.EditPullRequestOption]`. -Path: `/api/v1/repos/{owner}/{repo}/pulls/{index}` - -Action methods (6): -- `Merge(ctx, owner, repo, index, method)` — POST `.../pulls/{index}/merge` -- `Update(ctx, owner, repo, index)` — POST `.../pulls/{index}/update` -- `ListReviews(ctx, owner, repo, index)` — GET `.../pulls/{index}/reviews` -- `SubmitReview(ctx, owner, repo, index, reviewID)` — POST `.../pulls/{index}/reviews/{id}` -- `DismissReview(ctx, owner, repo, index, reviewID, msg)` — POST `.../pulls/{index}/reviews/{id}/dismissals` -- `UndismissReview(ctx, owner, repo, index, reviewID)` — POST `.../pulls/{index}/reviews/{id}/undismissals` - -Write tests for at least: List, Get, Create for each service + one action method each. - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: IssueService and PullService with actions"` - ---- - -### Task 12: OrgService + TeamService + UserService - -**Files:** -- Create: `orgs.go`, `teams.go`, `users.go` -- Create: `orgs_test.go`, `teams_test.go`, `users_test.go` -- Modify: `forge.go` (wire up) -- Modify: `services_stub.go` (remove stubs) - -**OrgService** — `Resource[types.Organization, types.CreateOrgOption, types.EditOrgOption]` -Path: `/api/v1/orgs/{org}` -Actions: ListMembers, AddMember, RemoveMember, SetAvatar, Block, Unblock - -**TeamService** — `Resource[types.Team, types.CreateTeamOption, types.EditTeamOption]` -Path: `/api/v1/teams/{id}` -Actions: ListMembers, AddMember, RemoveMember, ListRepos, AddRepo, RemoveRepo - -**UserService** — `Resource[types.User, struct{}, struct{}]` (no create/edit via this path) -Path: `/api/v1/users/{username}` -Custom: `GetCurrent(ctx)`, `ListFollowers(ctx)`, `ListStarred(ctx)`, keys, GPG keys, settings - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: OrgService, TeamService, UserService"` - ---- - -### Task 13: AdminService - -**Files:** -- Create: `admin.go` -- Create: `admin_test.go` -- Modify: `forge.go` (wire up) -- Modify: `services_stub.go` (remove stub) - -**AdminService** — No generic Resource (admin endpoints are heterogeneous). -Direct methods: -- `ListUsers(ctx)` — GET `/api/v1/admin/users` -- `CreateUser(ctx, opts)` — POST `/api/v1/admin/users` -- `EditUser(ctx, username, opts)` — PATCH `/api/v1/admin/users/{username}` -- `DeleteUser(ctx, username)` — DELETE `/api/v1/admin/users/{username}` -- `RenameUser(ctx, username, newName)` — POST `.../users/{username}/rename` -- `ListOrgs(ctx)` — GET `/api/v1/admin/orgs` -- `RunCron(ctx, task)` — POST `/api/v1/admin/cron/{task}` -- `ListCron(ctx)` — GET `/api/v1/admin/cron` -- `AdoptRepo(ctx, owner, repo)` — POST `.../unadopted/{owner}/{repo}` -- `GenerateRunnerToken(ctx)` — POST `/api/v1/admin/runners/registration-token` - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: AdminService with user, org, cron, runner operations"` - ---- - -## Wave 4: Extended Services (Tasks 14-17) - -### Task 14: BranchService + ReleaseService - -**BranchService** — `Resource[types.Branch, types.CreateBranchRepoOption, struct{}]` -Path: `/api/v1/repos/{owner}/{repo}/branches/{branch}` -Additional: BranchProtection CRUD at `.../branch_protections/{name}` - -**ReleaseService** — `Resource[types.Release, types.CreateReleaseOption, types.EditReleaseOption]` -Path: `/api/v1/repos/{owner}/{repo}/releases/{id}` -Additional: Asset upload/download at `.../releases/{id}/assets` - -### Task 15: LabelService + WebhookService + ContentService - -**LabelService** — Handles repo labels, org labels, and issue labels. -- `ListRepoLabels(ctx, owner, repo)` -- `CreateRepoLabel(ctx, owner, repo, opts)` -- `ListOrgLabels(ctx, org)` - -**WebhookService** — `Resource[types.Hook, types.CreateHookOption, types.EditHookOption]` -Actions: `TestHook(ctx, owner, repo, id)` - -**ContentService** — File read/write via API -- `GetFile(ctx, owner, repo, path)` — GET `.../contents/{path}` -- `CreateFile(ctx, owner, repo, path, opts)` — POST `.../contents/{path}` -- `UpdateFile(ctx, owner, repo, path, opts)` — PUT `.../contents/{path}` -- `DeleteFile(ctx, owner, repo, path, opts)` — DELETE `.../contents/{path}` - -### Task 16: ActionsService + NotificationService + PackageService - -**ActionsService** — runners, secrets, variables, workflow dispatch -- Repo-level: `.../repos/{owner}/{repo}/actions/{secrets,variables,runners}` -- Org-level: `.../orgs/{org}/actions/{secrets,variables,runners}` -- `DispatchWorkflow(ctx, owner, repo, workflow, opts)` - -**NotificationService** — list, mark read -- `List(ctx)` — GET `/api/v1/notifications` -- `MarkRead(ctx)` — PUT `/api/v1/notifications` -- `GetThread(ctx, id)` — GET `.../notifications/threads/{id}` - -**PackageService** — list, get, delete -- `List(ctx, owner)` — GET `/api/v1/packages/{owner}` -- `Get(ctx, owner, type, name, version)` — GET `.../packages/{owner}/{type}/{name}/{version}` - -### Task 17: WikiService + MiscService + CommitService - -**WikiService** — pages -- `ListPages(ctx, owner, repo)` -- `GetPage(ctx, owner, repo, pageName)` -- `CreatePage(ctx, owner, repo, opts)` -- `EditPage(ctx, owner, repo, pageName, opts)` -- `DeletePage(ctx, owner, repo, pageName)` - -**MiscService** — markdown, licenses, gitignore, nodeinfo -- `RenderMarkdown(ctx, text, mode)` — POST `/api/v1/markdown` -- `ListLicenses(ctx)` — GET `/api/v1/licenses` -- `ListGitignoreTemplates(ctx)` — GET `/api/v1/gitignore/templates` -- `NodeInfo(ctx)` — GET `/api/v1/nodeinfo` - -**CommitService** — status and notes -- `GetCombinedStatus(ctx, owner, repo, ref)` -- `CreateStatus(ctx, owner, repo, sha, opts)` -- `SetNote(ctx, owner, repo, sha, opts)` - -For each task in Wave 4: write tests first, implement, verify all tests pass, commit. - -Run after each task: `cd /Users/snider/Code/go-forge && go test ./... -v` - ---- - -## Wave 5: Clean Up + Services Stub Removal (Task 18) - -### Task 18: Remove stubs + final wiring - -**Files:** -- Delete: `services_stub.go` -- Modify: `forge.go` — replace all stub initialisations with real `newXxxService(c)` calls - -**Step 1: Remove services_stub.go** - -Delete the file. All service types should now be defined in their own files. - -**Step 2: Wire all services in forge.go** - -Update `NewForge()` to call `newXxxService(c)` for every service. - -**Step 3: Run all tests** - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v -count=1` -Expected: All tests pass - -**Step 4: Commit** - -```bash -git add -A -git commit -m "feat: wire all 17 services, remove stubs - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 6: Integration + Forge Repo Setup (Tasks 19-20) - -### Task 19: Create Forge repo + push - -**Step 1: Create repo on Forge** - -Use the Forgejo API or web UI to create `core/go-forge` on `forge.lthn.ai`. - -**Step 2: Add remote and push** - -```bash -cd /Users/snider/Code/go-forge -git remote add forge ssh://git@forge.lthn.ai:2223/core/go-forge.git -git push -u forge main -``` - -### Task 20: Wiki documentation (go-ai treatment) - -Create wiki pages for go-forge on Forge, matching the go-ai documentation pattern: - -1. **Home** — Overview, install, quick start -2. **Architecture** — Generic Resource[T,C,U], codegen pipeline, service pattern -3. **Services** — All 17 services with example usage -4. **Code Generation** — How to regenerate types, upgrade Forgejo version -5. **Configuration** — Env vars, config file, flags -6. **Error Handling** — APIError, IsNotFound, IsForbidden -7. **Development** — Contributing, testing, releasing - -Use the Forge wiki API: `POST /api/v1/repos/core/go-forge/wiki/new` with `{"content_base64":"...","title":"..."}`. - ---- - -## Dependency Sequencing - -``` -Task 1 (scaffold) ← Task 2 (client) ← Task 3 (pagination) ← Task 4 (params) ← Task 5 (resource) -Task 1 ← Task 7 (parser) ← Task 8 (generator) ← Task 9 (generate types) -Task 5 + Task 9 ← Task 6 (config) ← Task 10 (forge + repos) -Task 10 ← Task 11 (issues + PRs) -Task 10 ← Task 12 (orgs + teams + users) -Task 10 ← Task 13 (admin) -Task 10 ← Task 14-17 (extended services) -Task 14-17 ← Task 18 (remove stubs) -Task 18 ← Task 19 (forge push) -Task 19 ← Task 20 (wiki) -``` - -**Wave 1 (Tasks 1-6)**: Foundation — all independent once scaffolded -**Wave 2 (Tasks 7-9)**: Codegen — sequential (parser → generator → run) -**Wave 3 (Tasks 10-13)**: Core services — Task 10 first (creates Forge + stubs), then 11-13 parallel -**Wave 4 (Tasks 14-17)**: Extended services — all parallel after Task 10 -**Wave 5 (Task 18)**: Clean up — after all services done -**Wave 6 (Tasks 19-20)**: Ship — after clean up - -## Verification - -After all tasks: - -1. `cd /Users/snider/Code/go-forge && go test ./... -count=1` — all pass -2. `go build ./...` — compiles cleanly -3. `go vet ./...` — no issues -4. Verify `types/` contains generated files with `Repository`, `Issue`, `PullRequest`, etc. -5. Verify `NewForge()` creates client with all 17 services populated -6. Verify action methods exist (Fork, Merge, Pin, etc.) diff --git a/docs/plans/completed/cli-meta-package.md b/docs/plans/completed/cli-meta-package.md deleted file mode 100644 index d88672b..0000000 --- a/docs/plans/completed/cli-meta-package.md +++ /dev/null @@ -1,30 +0,0 @@ -# CLI Meta-Package Restructure — Completed - -**Completed:** 22 Feb 2026 - -## What Was Done - -`pkg/cli` was extracted from `core/go` into its own Go module at `forge.lthn.ai/core/cli`. This made the CLI SDK a first-class, independently versioned package rather than a subdirectory of the Go foundation repo. - -Following the extraction, an ecosystem-wide import path migration updated all consumers from the old path to the new one: - -- Old: `forge.lthn.ai/core/go/pkg/cli` -- New: `forge.lthn.ai/core/cli/pkg/cli` - -## Scope - -- **147+ files** updated across **10 repos** -- All repos build clean after migration - -## Repos Migrated - -`core/cli`, `core/go`, `go-devops`, `go-ai`, `go-agentic`, `go-crypt`, `go-rag`, `go-scm`, `go-api`, `go-update` - -## Key Outcomes - -- `forge.lthn.ai/core/cli/pkg/cli` is the single import for all CLI concerns across the ecosystem -- Domain repos are insulated from cobra, lipgloss, and bubbletea — only `pkg/cli` imports them -- Command registration uses the Core framework lifecycle via `cli.WithCommands()` — no `init()`, no global state -- `core/cli` is a thin assembly repo (~2K LOC) with 7 meta packages; all business logic lives in domain repos -- Variant binary pattern established: multiple `main.go` files can wire different `WithCommands` sets for targeted binaries (core-ci, core-mlx, core-ops, etc.) -- Command migration from the old `core/cli` monolith to domain repos was completed in full (13 command groups moved) diff --git a/docs/plans/completed/cli-sdk-expansion.md b/docs/plans/completed/cli-sdk-expansion.md deleted file mode 100644 index a0a84a3..0000000 --- a/docs/plans/completed/cli-sdk-expansion.md +++ /dev/null @@ -1,39 +0,0 @@ -# CLI SDK Expansion — Completion Summary - -**Completed:** 21 February 2026 -**Module:** `forge.lthn.ai/core/go/pkg/cli` (later migrated to `forge.lthn.ai/core/cli`) -**Status:** Complete — all TUI primitives shipped, then extracted to core/cli - -## What Was Built - -Extended `pkg/cli` with charmbracelet TUI primitives so domain repos only -import `core/cli` for all CLI concerns. Charmbracelet dependencies (bubbletea, -bubbles, lipgloss) are encapsulated behind our own types. - -### Components added - -| Component | File | Purpose | -|-----------|------|---------| -| RunTUI | `runtui.go` | Escape hatch with `Model`/`Msg`/`Cmd`/`KeyMsg` types | -| Spinner | `spinner.go` | Async handle with `Update()`, `Done()`, `Fail()` | -| ProgressBar | `progressbar.go` | `Increment()`, `Set()`, `SetMessage()`, `Done()` | -| InteractiveList | `list.go` | Keyboard navigation with terminal fallback | -| TextInput | `textinput.go` | Placeholder, masking, validation | -| Viewport | `viewport.go` | Scrollable content for logs, diffs, docs | -| Form (stub) | `form.go` | Interface defined, bufio fallback | -| FilePicker (stub) | `filepicker.go` | Interface defined, bufio fallback | -| Tabs (stub) | `tabs.go` | Interface defined, simple fallback | - -### Subsequent migration - -On 22 February 2026, `pkg/cli` was extracted from `core/go` into its own -module at `forge.lthn.ai/core/cli` and all imports were updated. The TUI -primitives now live in the standalone CLI module. - -### Frame upgrade (follow-on) - -The Frame layout system was upgraded to implement `tea.Model` directly on -22 February 2026 (in `core/cli`), adding bubbletea lifecycle, `KeyMap` for -configurable bindings, `Navigate()`/`Back()` for panel switching, and -lipgloss-based HLCRF rendering. This was a separate plan -(`frame-bubbletea`) that built on the SDK expansion. diff --git a/docs/plans/completed/go-api.md b/docs/plans/completed/go-api.md deleted file mode 100644 index 86278a2..0000000 --- a/docs/plans/completed/go-api.md +++ /dev/null @@ -1,57 +0,0 @@ -# go-api — Completion Summary - -**Completed:** 21 February 2026 -**Module:** `forge.lthn.ai/core/go-api` -**Status:** Phases 1–3 complete, 176 tests passing - -## What Was Built - -### Phase 1 — Core Framework (20 Feb 2026) - -Gin-based HTTP engine with extensible middleware via `With*()` options. Key components: - -- `RouteGroup` / `StreamGroup` interfaces — subsystems register their own endpoints -- `Response[T]` envelope — `OK()`, `Fail()`, `Paginated()` generics -- `Engine` — `New()`, `Register()`, `Handler()`, `Serve()` with graceful shutdown -- Bearer auth, request ID, and CORS middleware -- WebSocket endpoint wrapping a `go-ws` Hub -- Swagger UI at `/swagger/` with runtime spec serving -- `/health` endpoint always available without auth -- First integration proof in `go-ml/api/` (3 endpoints, 12 tests) - -### Phase 2 — Gin Plugin Stack (20–21 Feb 2026) - -17 middleware plugins added across four waves, all as drop-in `With*()` options: - -| Wave | Plugins | -|------|---------| -| 1 — Gateway hardening | Authentik (OIDC + forward auth), secure headers, structured slog, timeouts, gzip, static files | -| 2 — Performance + auth | Brotli compression, in-memory response cache, server-side sessions, Casbin RBAC | -| 3 — Network + streaming | HTTP signature verification, SSE broker, reverse proxy detection, i18n locale, GraphQL | -| 4 — Observability | pprof, expvar, OpenTelemetry distributed tracing | - -### Phase 3 — OpenAPI + SDK Codegen (21 Feb 2026) - -Runtime spec generation (not swaggo annotations — incompatible with dynamic RouteGroups and `Response[T]` generics): - -- `DescribableGroup` interface — opt-in OpenAPI metadata for route groups -- `ToolBridge` — converts MCP tool descriptors into `POST /{tool_name}` REST endpoints -- `SpecBuilder` — assembles full OpenAPI 3.1 JSON from registered groups at runtime -- Spec export to JSON and YAML (`core api spec`) -- SDK codegen wrapper for openapi-generator-cli, 11 languages (`core api sdk --lang go`) -- `go-ai` `mcp/registry.go` — generic `addToolRecorded[In,Out]` captures types in closures -- `go-ai` `mcp/bridge.go` — `BridgeToAPI()` populates ToolBridge from MCP tool registry -- CLI commands: `core api spec`, `core api sdk` (in `core/cli` dev branch) - -## Key Outcomes - -- **176 tests** across go-api (143), go-ai bridge (10), and CLI commands (4), all passing -- Zero internal ecosystem dependencies — subsystems import go-api, not the reverse -- Authentik (OIDC) and bearer token auth coexist; Casbin adds RBAC on top -- Four-protocol access pattern established: REST, GraphQL, WebSocket, MCP — same handlers - -## Known Limitations - -- Subsystem MCP tools registered via `mcp.AddTool` directly are excluded from the REST bridge (only the 10 built-in tools appear). Fix: pass `*Service` to `RegisterTools` instead of `*mcp.Server`. -- `structSchema` reflection handles flat structs only; nested structs are not recursed. -- `core api spec` currently emits a spec with only `/health`; full MCP wiring into the CLI command is pending. diff --git a/docs/plans/completed/mcp-integration.md b/docs/plans/completed/mcp-integration.md deleted file mode 100644 index 7edf86e..0000000 --- a/docs/plans/completed/mcp-integration.md +++ /dev/null @@ -1,37 +0,0 @@ -# MCP Integration — Completion Summary - -**Completed:** 2026-02-05 -**Plan:** `docs/plans/2026-02-05-mcp-integration.md` - -## What Was Built - -### RAG Tools (`pkg/mcp/tools_rag.go`) -Three MCP tools added to the existing `pkg/mcp` server: -- `rag_query` — semantic search against Qdrant vector DB -- `rag_ingest` — ingest a file or directory into a named collection -- `rag_collections` — list available Qdrant collections (with optional stats) - -### Metrics Tools (`pkg/mcp/tools_metrics.go`) -Two MCP tools for agent activity tracking: -- `metrics_record` — write a typed event (agent_id, repo, arbitrary data) to JSONL storage -- `metrics_query` — query events with aggregation by type, repo, and agent; supports human-friendly duration strings (7d, 24h) - -Also added `parseDuration()` helper for "Nd"/"Nh"/"Nm" duration strings. - -### `core mcp serve` Command (`internal/cmd/mcpcmd/cmd_mcp.go`) -New CLI sub-command registered via `cli.WithCommands()` (not `init()`). -- Runs `pkg/mcp` server over stdio by default -- TCP mode via `MCP_ADDR=:9000` environment variable -- `--workspace` flag to restrict file operations to a directory - -Registered in the full CLI variant. i18n strings added for all user-facing text. - -### Plugin Configuration -`.mcp.json` created for the `agentic-flows` Claude Code plugin, pointing to `core mcp serve`. Exposes all 15 tools to Claude Code agents via the `core-cli` MCP server name. - -## Key Outcomes - -- `core mcp serve` is the single entry point for all MCP tooling (file ops, RAG, metrics, language detection, process management, WebSocket, webview/CDP) -- MCP command moved to `go-ai/cmd/mcpcmd/` in final form; the plan's `internal/cmd/mcpcmd/` path reflects the pre-extraction location -- Registration pattern updated from `init()` + `RegisterCommands()` to `cli.WithCommands()` lifecycle hooks -- Services required at runtime: Qdrant (localhost:6333), Ollama with nomic-embed-text (localhost:11434) diff --git a/docs/plans/completed/qk-bone-orientation.md b/docs/plans/completed/qk-bone-orientation.md deleted file mode 100644 index 0cfcaa9..0000000 --- a/docs/plans/completed/qk-bone-orientation.md +++ /dev/null @@ -1,62 +0,0 @@ -# Q/K Bone Orientation — Completion Summary - -**Completed:** 23 February 2026 -**Repos:** go-inference, go-mlx, go-ml, LEM -**Status:** All 7 tasks complete, 14 files changed (+917 lines), all tests passing - -## What Was Built - -### go-inference — AttentionSnapshot types (Task 1) - -`AttentionSnapshot` struct and `AttentionInspector` optional interface. Backends expose attention data via type assertion — no breaking changes to `TextModel`. - -### go-mlx — KV cache extraction (Task 2) - -`InspectAttention` on `metalAdapter` runs a single prefill pass and extracts post-RoPE K vectors from each layer's KV cache. Tested against real Gemma3-1B (26 layers, 1 KV head via GQA, 256 head dim). - -### go-ml — Adapter pass-through (Task 3) - -`InspectAttention` on `InferenceAdapter` type-asserts the underlying `TextModel` to `AttentionInspector`. Returns clear error for unsupported backends. - -### LEM — Analysis engine (Task 4) - -Pure Go CPU math in `pkg/lem/attention.go`. Computes 5 BO metrics from raw K tensors: - -- **Mean Coherence** — pairwise cosine similarity of K vectors within each layer -- **Cross-Layer Alignment** — cosine similarity of mean K vectors between adjacent layers -- **Head Entropy** — normalised Shannon entropy of K vector magnitudes across positions -- **Phase-Lock Score** — fraction of head pairs above coherence threshold (0.7) -- **Joint Collapse Count** — layers where cross-alignment drops below threshold (0.5) - -Composite score: 30% coherence + 25% cross-alignment + 20% phase-lock + 15% entropy + 10% joint stability → 0-100 scale. - -### LEM — CLI command (Task 5) - -`lem score attention -model -prompt [-json]` loads a model, runs InspectAttention, and prints BO metrics. - -### LEM — Distill integration (Task 6) - -Opt-in attention scoring in the distill pipeline. Gated behind `scorer.attention: true` and `scorer.attention_min_score` in ai.yaml. Costs one extra prefill per probe. - -### LEM — Feature vectors (Task 7) - -19D full feature vector: 6D grammar + 8D heuristic + 5D attention (`mean_coherence`, `cross_alignment`, `head_entropy`, `phase_lock`, `joint_stability`). Ready for Poindexter KDTree spatial indexing. - -## Key Decisions - -- **Optional interface** — `AttentionInspector` via type assertion, not added to `TextModel` -- **Named `BOResult`** — avoids collision with `metal.AttentionResult` in go-mlx -- **Opt-in for distill** — extra prefill per probe is expensive, off by default -- **Pure Go analysis** — zero CGO deps in the analysis engine; GPU data extracted once via `.Floats()` - -## Commits - -| Repo | SHA | Message | -|------|-----|---------| -| go-inference | `0f7263f` | feat: add AttentionInspector optional interface | -| go-mlx | `c2177f7` | feat: implement AttentionInspector via KV cache extraction | -| go-ml | `45e9fed` | feat: add InspectAttention pass-through | -| LEM | `28309b2` | feat: add Q/K Bone Orientation analysis engine | -| LEM | `e333192` | feat: add 'lem score attention' CLI | -| LEM | `fbc636e` | feat: integrate attention scoring into distill pipeline | -| LEM | `b621baa` | feat: add 19D full feature vector | diff --git a/docs/skill/index.md b/docs/skill/index.md deleted file mode 100644 index 40ae3ad..0000000 --- a/docs/skill/index.md +++ /dev/null @@ -1,35 +0,0 @@ -# Claude Code Skill - -The `core` skill teaches Claude Code how to use the Core CLI effectively. - -## Installation - -```bash -curl -fsSL https://raw.githubusercontent.com/host-uk/core/main/.claude/skills/core/install.sh | bash -``` - -Or if you have the repo cloned: - -```bash -./.claude/skills/core/install.sh -``` - -## What it does - -Once installed, Claude Code will: - -- Auto-invoke when working in host-uk repositories -- Use `core` commands instead of raw `go`/`php`/`git` commands -- Follow the correct patterns for testing, building, and releasing - -## Manual invocation - -Type `/core` in Claude Code to invoke the skill manually. - -## Updating - -Re-run the install command to update to the latest version. - -## Location - -Skills are installed to `~/.claude/skills/core/SKILL.md`. diff --git a/docs/static/assets/style.css b/docs/static/assets/style.css deleted file mode 100644 index e69de29..0000000 diff --git a/docs/static/index.html b/docs/static/index.html deleted file mode 100644 index e69de29..0000000 diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md deleted file mode 100644 index 72a488d..0000000 --- a/docs/troubleshooting.md +++ /dev/null @@ -1,356 +0,0 @@ -# Troubleshooting - -Common issues and how to resolve them. - -## Installation Issues - -### "command not found: core" - -**Cause:** Go's bin directory is not in your PATH. - -**Fix:** - -```bash -# Add to ~/.bashrc or ~/.zshrc -export PATH="$PATH:$(go env GOPATH)/bin" - -# Then reload -source ~/.bashrc # or ~/.zshrc -``` - -### "go: module forge.lthn.ai/core/cli: no matching versions" - -**Cause:** Go module proxy hasn't cached the latest version yet. - -**Fix:** - -```bash -# Bypass proxy -GOPROXY=direct go install forge.lthn.ai/core/cli/cmd/core@latest -``` - ---- - -## Build Issues - -### "no Go files in..." - -**Cause:** Core couldn't find a main package to build. - -**Fix:** - -1. Check you're in the correct directory -2. Ensure `.core/build.yaml` has the correct `main` path: - -```yaml -project: - main: ./cmd/myapp # Path to main package -``` - -### "CGO_ENABLED=1 but no C compiler" - -**Cause:** Build requires CGO but no C compiler is available. - -**Fix:** - -```bash -# Option 1: Disable CGO (if not needed) -core build # Core disables CGO by default - -# Option 2: Install a C compiler -# macOS -xcode-select --install - -# Ubuntu/Debian -sudo apt install build-essential - -# Windows -# Install MinGW or use WSL -``` - -### Build succeeds but binary doesn't run - -**Cause:** Built for wrong architecture. - -**Fix:** - -```bash -# Check what you built -file dist/myapp-* - -# Build for your current platform -core build --targets $(go env GOOS)/$(go env GOARCH) -``` - ---- - -## Release Issues - -### "dry-run mode, use --we-are-go-for-launch to publish" - -**This is expected behaviour.** Core runs in dry-run mode by default for safety. - -**To actually publish:** - -```bash -core ci --we-are-go-for-launch -``` - -### "failed to create release: 401 Unauthorized" - -**Cause:** GitHub token missing or invalid. - -**Fix:** - -```bash -# Authenticate with GitHub CLI -gh auth login - -# Or set token directly -export GITHUB_TOKEN=ghp_xxxxxxxxxxxx -``` - -### "no artifacts found in dist/" - -**Cause:** You need to build before releasing. - -**Fix:** - -```bash -# Build first -core build - -# Then release -core ci --we-are-go-for-launch -``` - -### "tag already exists" - -**Cause:** Trying to release a version that's already been released. - -**Fix:** - -1. Update version in your code/config -2. Or delete the existing tag (if intentional): - -```bash -git tag -d v1.0.0 -git push origin :refs/tags/v1.0.0 -``` - ---- - -## Multi-Repo Issues - -### "repos.yaml not found" - -**Cause:** Core can't find the package registry. - -**Fix:** - -Core looks for `repos.yaml` in: -1. Current directory -2. Parent directories (walking up to root) -3. `~/Code/host-uk/repos.yaml` -4. `~/.config/core/repos.yaml` - -Either: -- Run commands from a directory with `repos.yaml` -- Use `--registry /path/to/repos.yaml` -- Run `core setup` to bootstrap a new workspace - -### "failed to clone: Permission denied (publickey)" - -**Cause:** SSH key not configured for GitHub. - -**Fix:** - -```bash -# Check SSH connection -ssh -T git@github.com - -# If that fails, add your key -ssh-add ~/.ssh/id_ed25519 - -# Or configure SSH -# See: https://docs.github.com/en/authentication/connecting-to-github-with-ssh -``` - -### "repository not found" during setup - -**Cause:** You don't have access to the repository, or it doesn't exist. - -**Fix:** - -1. Check you're authenticated: `gh auth status` -2. Verify the repo exists and you have access -3. For private repos, ensure your token has `repo` scope - ---- - -## GitHub Integration Issues - -### "gh: command not found" - -**Cause:** GitHub CLI not installed. - -**Fix:** - -```bash -# macOS -brew install gh - -# Ubuntu/Debian -sudo apt install gh - -# Windows -winget install GitHub.cli - -# Then authenticate -gh auth login -``` - -### "core dev issues" shows nothing - -**Possible causes:** - -1. No open issues exist -2. Not authenticated with GitHub -3. Not in a directory with `repos.yaml` - -**Fix:** - -```bash -# Check auth -gh auth status - -# Check you're in a workspace -ls repos.yaml - -# Show all issues including closed -core dev issues --all -``` - ---- - -## PHP Issues - -### "frankenphp: command not found" - -**Cause:** FrankenPHP not installed. - -**Fix:** - -```bash -# macOS -brew install frankenphp - -# Or use Docker -core php dev --docker -``` - -### "core php dev" exits immediately - -**Cause:** Usually a port conflict or missing dependency. - -**Fix:** - -```bash -# Check if port 8000 is in use -lsof -i :8000 - -# Try a different port -core php dev --port 9000 - -# Check logs for errors -core php logs -``` - ---- - -## Performance Issues - -### Commands are slow - -**Possible causes:** - -1. Large number of repositories -2. Network latency to GitHub -3. Go module downloads - -**Fix:** - -```bash -# For multi-repo commands, use health for quick check -core dev health # Fast summary - -# Instead of -core dev work --status # Full table (slower) - -# Pre-download Go modules -go mod download -``` - ---- - -## AI and Agentic Issues - -### "ANTHROPIC_API_KEY not set" - -**Cause:** You're trying to use `core ai` or `core dev commit` (which uses Claude for messages) without an API key. - -**Fix:** - -```bash -export ANTHROPIC_API_KEY=sk-ant-xxxxxxxxxxxx -``` - -### "failed to connect to Agentic API" - -**Cause:** Network issues or incorrect `AGENTIC_BASE_URL`. - -**Fix:** - -1. Check your internet connection -2. If using a custom endpoint, verify `AGENTIC_BASE_URL` -3. Ensure you are authenticated if required: `export AGENTIC_TOKEN=xxxx` - ---- - -## Getting More Help - -### Enable Verbose Output - -Most commands support `-v` or `--verbose`: - -```bash -core build -v -core go test -v -``` - -### Check Environment - -```bash -core doctor -``` - -This verifies all required tools are installed and configured. - -### Report Issues - -If you've found a bug: - -1. Check existing issues: https://forge.lthn.ai/core/cli/issues -2. Create a new issue with: - - Core version (`core --version`) - - OS and architecture (`go env GOOS GOARCH`) - - Command that failed - - Full error output - ---- - -## See Also - -- [Getting Started](getting-started.md) - Installation and first steps -- [Configuration](configuration.md) - Config file reference -- [doctor](cmd/doctor/) - Environment verification diff --git a/docs/user-guide.md b/docs/user-guide.md deleted file mode 100644 index 3820d9a..0000000 --- a/docs/user-guide.md +++ /dev/null @@ -1,100 +0,0 @@ -# User Guide - -This guide provides a comprehensive overview of how to use the Core CLI to manage your development workflow. - -## Key Concepts - -### Projects -A Project is a single repository containing code (Go, PHP, or Wails). Core helps you test, build, and release these projects using a consistent set of commands. - -### Workspaces -A Workspace is a collection of related projects. Core is designed to work across multiple repositories, allowing you to perform actions (like checking status or committing changes) on all of them at once. - -### Registry (`repos.yaml`) -The Registry is a configuration file that defines the repositories in your workspace. It includes information about where they are located on GitHub, their dependencies, and their purpose. - ---- - -## Daily Workflow - -### Working with a Single Project - -For a typical day-to-day development on a single project: - -1. **Verify your environment**: - ```bash - core doctor - ``` -2. **Run tests while you work**: - ```bash - core go test - ``` -3. **Keep code clean**: - ```bash - core go fmt --fix - core go lint - ``` -4. **Build and preview**: - ```bash - core build - ``` - -### Working with Multiple Repositories - -If you are working across many repositories in a workspace: - -1. **Check status of all repos**: - ```bash - core dev work --status - ``` -2. **Sync all changes**: - ```bash - core dev pull --all - ``` -3. **Commit and push everything**: - ```bash - core dev work - ``` - ---- - -## Building and Releasing - -Core separates the building of artifacts from the releasing of those artifacts. - -### 1. Build -The `core build` command detects your project type and builds binaries for your configured targets. Artifacts are placed in the `dist/` directory. - -### 2. Preview Release -Use `core ci` to see a summary of what would be included in a release (changelog, artifacts, etc.). This is a dry-run by default. - -### 3. Publish Release -When you are ready to publish to GitHub: -```bash -core ci --we-are-go-for-launch -``` - ---- - -## PHP and Laravel Development - -Core provides a unified development server for Laravel projects that orchestrates several services: - -```bash -core php dev -``` -This starts FrankenPHP, Vite, Horizon, Reverb, and Redis as configured in your `.core/php.yaml`. - ---- - -## Common Workflows - -For detailed examples of common end-to-end workflows, see the [Workflows](workflows.md) page. - ---- - -## Getting More Help - -- Use the `--help` flag with any command: `core build --help` -- Check the [FAQ](faq.md) for common questions. -- If you run into trouble, see the [Troubleshooting Guide](troubleshooting.md). diff --git a/docs/workflows.md b/docs/workflows.md deleted file mode 100644 index c0aca12..0000000 --- a/docs/workflows.md +++ /dev/null @@ -1,334 +0,0 @@ -# Workflows - -Common end-to-end workflows for Core CLI. - -## Go Project: Build and Release - -Complete workflow from code to GitHub release. - -```bash -# 1. Run tests -core go test - -# 2. Check coverage (Statement and Branch) -core go cov --threshold 40 --branch-threshold 35 - -# 3. Format and lint -core go fmt --fix -core go lint - -# 4. Build for all platforms -core build --targets linux/amd64,linux/arm64,darwin/arm64,windows/amd64 - -# 5. Preview release (dry-run) -core ci - -# 6. Publish -core ci --we-are-go-for-launch -``` - -**Output structure:** - -``` -dist/ -├── myapp-darwin-arm64.tar.gz -├── myapp-linux-amd64.tar.gz -├── myapp-linux-arm64.tar.gz -├── myapp-windows-amd64.zip -└── CHECKSUMS.txt -``` - ---- - -## PHP Project: Development to Deployment - -Local development through to production deployment. - -```bash -# 1. Start development environment -core php dev - -# 2. Run tests (in another terminal) -core php test --parallel - -# 3. Check code quality -core php fmt --fix -core php analyse - -# 4. Deploy to staging -core php deploy --staging --wait - -# 5. Verify staging -# (manual testing) - -# 6. Deploy to production -core php deploy --wait - -# 7. Monitor -core php deploy:status -``` - -**Rollback if needed:** - -```bash -core php deploy:rollback -``` - ---- - -## Multi-Repo: Daily Workflow - -Working across the host-uk monorepo. - -### Morning: Sync Everything - -```bash -# Quick health check -core dev health - -# Pull all repos that are behind -core dev pull --all - -# Check for issues assigned to you -core dev issues --assignee @me -``` - -### During Development - -```bash -# Work on code... - -# Check status across all repos -core dev work --status - -# Commit changes (Claude-assisted messages) -core dev commit - -# Push when ready -core dev push -``` - -### End of Day - -```bash -# Full workflow: status → commit → push -core dev work - -# Check CI status -core dev ci - -# Review any failed builds -core dev ci --failed -``` - ---- - -## New Developer: Environment Setup - -First-time setup for a new team member. - -```bash -# 1. Verify prerequisites -core doctor - -# 2. Create workspace directory -mkdir ~/Code/host-uk && cd ~/Code/host-uk - -# 3. Bootstrap workspace (interactive) -core setup - -# 4. Select packages in wizard -# Use arrow keys, space to select, enter to confirm - -# 5. Verify setup -core dev health - -# 6. Start working -core dev work --status -``` - ---- - -## CI Pipeline: Automated Build - -Example GitHub Actions workflow. - -```yaml -# .github/workflows/release.yml -name: Release - -on: - push: - tags: - - 'v*' - -jobs: - release: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-go@v5 - with: - go-version: '1.23' - - - name: Install Core - run: go install forge.lthn.ai/core/cli/cmd/core@latest - - - name: Build - run: core build --ci - - - name: Release - run: core ci --we-are-go-for-launch - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} -``` - ---- - -## SDK Generation: API Client Updates - -Generate SDK clients when API changes. - -```bash -# 1. Validate OpenAPI spec -core sdk validate - -# 2. Check for breaking changes -core sdk diff --base v1.0.0 - -# 3. Generate SDKs -core build sdk - -# 4. Review generated code -git diff - -# 5. Commit if satisfied -git add -A && git commit -m "chore: regenerate SDK clients" -``` - ---- - -## Dependency Update: Cross-Repo Change - -When updating a shared package like `core-php`. - -```bash -# 1. Make changes in core-php -cd ~/Code/host-uk/core-php -# ... edit code ... - -# 2. Run tests -core go test # or core php test - -# 3. Check what depends on core-php -core dev impact core-php - -# Output: -# core-tenant (direct) -# core-admin (via core-tenant) -# core-api (direct) -# ... - -# 4. Commit core-php changes -core dev commit - -# 5. Update dependent packages -cd ~/Code/host-uk -for pkg in core-tenant core-admin core-api; do - cd $pkg - composer update host-uk/core-php - core php test - cd .. -done - -# 6. Commit all updates -core dev work -``` - ---- - -## Hotfix: Emergency Production Fix - -Fast path for critical fixes. - -```bash -# 1. Create hotfix branch -git checkout -b hotfix/critical-bug main - -# 2. Make fix -# ... edit code ... - -# 3. Test -core go test --run TestCriticalPath - -# 4. Build -core build - -# 5. Preview release -core ci --prerelease - -# 6. Publish hotfix -core ci --we-are-go-for-launch --prerelease - -# 7. Merge back to main -git checkout main -git merge hotfix/critical-bug -git push -``` - ---- - -## Documentation: Sync Across Repos - -Keep documentation synchronised. - -```bash -# 1. List all docs -core docs list - -# 2. Sync to central location -core docs sync --output ./docs-site - -# 3. Review changes -git diff docs-site/ - -# 4. Commit -git add docs-site/ -git commit -m "docs: sync from packages" -``` - ---- - -## Troubleshooting: Failed Build - -When a build fails. - -```bash -# 1. Check environment -core doctor - -# 2. Clean previous artifacts -rm -rf dist/ - -# 3. Verbose build -core build -v - -# 4. If Go-specific issues -core go mod tidy -core go mod verify - -# 5. Check for test failures -core go test -v - -# 6. Review configuration -cat .core/build.yaml -``` - ---- - -## See Also - -- [Getting Started](getting-started.md) - First-time setup -- [Troubleshooting](troubleshooting.md) - When things go wrong -- [Configuration](configuration.md) - Config file reference