diff --git a/docs/plans/2026-02-20-authentik-traefik-plan.md b/docs/plans/2026-02-20-authentik-traefik-plan.md deleted file mode 100644 index 091a082..0000000 --- a/docs/plans/2026-02-20-authentik-traefik-plan.md +++ /dev/null @@ -1,1163 +0,0 @@ -# Authentik + Traefik Integration Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Deploy Authentik as the identity provider, wire it into Traefik's forward auth, and add OIDC/header middleware to go-api so protected services get authenticated user context. - -**Architecture:** Authentik runs alongside existing services on de2 (production). Traefik's file provider loads a `forwardAuth` middleware definition pointing at Authentik's outpost. Services opt-in via Docker label `middlewares: authentik@file`. go-api gains a `WithAuthentik()` option that extracts user identity from Authentik headers (forward auth mode) or validates JWTs directly (API client mode). - -**Tech Stack:** Authentik 2025.2, Traefik v3.6, Go 1.25, coreos/go-oidc/v3, golang.org/x/oauth2 - -**Design doc:** `docs/plans/2026-02-20-go-api-design.md` (Authentik section) - -**Key references:** -- Traefik role: `/Users/snider/Code/DevOps/roles/traefik/` -- Authentik role: `/Users/snider/Code/DevOps/roles/authentik/` -- Forward auth template: `/Users/snider/Code/DevOps/roles/traefik/templates/dynamic-authentik.yml.j2` -- go-api repo: `/Users/snider/Code/go-api/` - ---- - -## Current State - -The Ansible infrastructure is **already built but not activated**: - -| Component | Status | Location | -|-----------|--------|----------| -| Traefik v3.6 role | Deployed on de2 | `roles/traefik/` | -| Authentik 2025.2 role | Written, **never deployed** | `roles/authentik/` | -| Forward auth middleware template | Written, conditional on `traefik_authentik_enabled` | `dynamic-authentik.yml.j2` | -| Outpost routing in Authentik compose | Pre-configured | `roles/authentik/templates/docker-compose.yml.j2` | -| 5 services with `authentik@file` | Labels present, middleware not yet available | `prod_rebuild.yml` | -| go-api Authentik middleware | **Not started** | — | - -**Headers Authentik will pass to go-api (via Traefik):** -``` -X-authentik-username, X-authentik-groups, X-authentik-entitlements, -X-authentik-email, X-authentik-name, X-authentik-uid, X-authentik-jwt, -X-authentik-meta-jwks, X-authentik-meta-outpost, X-authentik-meta-provider, -X-authentik-meta-app, X-authentik-meta-version -``` - ---- - -### Task 1: Enable Authentik in Production Inventory - -This task sets the Ansible variables to enable Authentik deployment on the production host. - -**Files:** -- Modify: `/Users/snider/Code/DevOps/inventory/host_vars/de2.yml` (or equivalent group_vars) - -**Step 1: Find the correct inventory file for de2** - -Run: -```bash -find /Users/snider/Code/DevOps/inventory -name "*.yml" -o -name "*.yaml" | head -20 -ls /Users/snider/Code/DevOps/inventory/ -``` - -Identify where de2's host vars live. - -**Step 2: Add Authentik variables** - -Add these variables for the de2 host: - -```yaml -# Authentik -traefik_authentik_enabled: true -traefik_authentik_url: "https://auth.host.uk.com" - -authentik_host: "auth.host.uk.com" -authentik_bootstrap_password: "" -authentik_bootstrap_token: "" -authentik_bootstrap_email: "admin@host.uk.com" -``` - -Note: `authentik_secret_key` auto-generates and persists on first run. `authentik_pg_password` auto-generates via lookup. The Authentik role handles both. - -**Step 3: Verify prerequisites exist on de2** - -Authentik requires PostgreSQL + Dragonfly (Redis). Check they're in the prod playbook: -```bash -grep -n "postgres\|dragonfly" /Users/snider/Code/DevOps/playbooks/prod_rebuild.yml | head -10 -``` - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/DevOps -git add inventory/ -git commit -m "feat(authentik): enable Authentik and Traefik forward auth on de2 - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: Add Authentik to Production Playbook - -The Authentik Ansible role exists but is not included in the prod rebuild playbook. This task adds it. - -**Files:** -- Modify: `/Users/snider/Code/DevOps/playbooks/prod_rebuild.yml` - -**Step 1: Read the playbook to find the right insertion point** - -Authentik must deploy AFTER PostgreSQL + Dragonfly (it needs them) and AFTER Traefik (it needs the proxy network), but BEFORE services that use `authentik@file`. - -```bash -grep -n "Phase\|traefik\|postgres\|dragonfly\|portainer\|glance" /Users/snider/Code/DevOps/playbooks/prod_rebuild.yml | head -20 -``` - -**Step 2: Add Authentik role include** - -Insert after the Traefik phase, before services: - -```yaml - # ── Phase N: Identity (Authentik) ── - - name: Deploy Authentik - ansible.builtin.include_role: - name: authentik - tags: [authentik] -``` - -**Step 3: Verify the playbook parses** - -```bash -cd /Users/snider/Code/DevOps -ansible-playbook playbooks/prod_rebuild.yml --syntax-check -``` - -Expected: No errors. - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/DevOps -git add playbooks/prod_rebuild.yml -git commit -m "feat(authentik): add Authentik phase to prod rebuild playbook - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: Deploy Authentik (Run Playbook) - -This is a manual step — run the Ansible playbook to deploy Authentik on de2. - -**Step 1: Dry-run the Authentik tag only** - -```bash -cd /Users/snider/Code/DevOps -ansible-playbook playbooks/prod_rebuild.yml --tags authentik --check --diff -``` - -Review the output. Expect: directories created, docker-compose deployed, containers started. - -Note: `--check` will skip shell/command tasks (like the PostgreSQL user creation). This is expected — the actual run will handle those. - -**Step 2: Deploy Authentik** - -```bash -ansible-playbook playbooks/prod_rebuild.yml --tags authentik -``` - -**Step 3: Re-deploy Traefik to pick up the forward auth middleware** - -The Traefik role conditionally deploys `dynamic-authentik.yml` based on `traefik_authentik_enabled`. Re-running the role with the new variable will create the middleware file: - -```bash -ansible-playbook playbooks/prod_rebuild.yml --tags traefik -``` - -**Step 4: Verify Authentik is accessible** - -```bash -curl -sI https://auth.host.uk.com | head -5 -``` - -Expected: HTTP 200 or 302 redirect to login page. - -**Step 5: Complete initial setup** - -Open `https://auth.host.uk.com/if/flow/initial-setup/` in a browser. Set the admin password (the bootstrap password from Task 1 is used for the API token, but the UI setup flow creates the actual admin account). - ---- - -### Task 4: Create Authentik OIDC Application for go-api - -This configures Authentik to issue tokens for go-api. Done via the Authentik admin UI or API. - -**Step 1: Create an OAuth2/OIDC Provider** - -In Authentik Admin → Providers → Create: - -| Field | Value | -|-------|-------| -| Name | `Core API` | -| Protocol | OAuth2/OIDC | -| Client type | Confidential | -| Client ID | `core-api` | -| Redirect URIs | `https://api.lthn.ai/auth/callback` (for auth code flow) | -| Signing key | Select auto-generated signing key | -| Scopes | `openid`, `email`, `profile` | -| Subject mode | Based on user's hashed ID | - -Record the **Client Secret** — needed for go-api config. - -**Step 2: Create an Application** - -In Authentik Admin → Applications → Create: - -| Field | Value | -|-------|-------| -| Name | `Core API` | -| Slug | `core-api` | -| Provider | Core API (from step 1) | -| Launch URL | `https://api.lthn.ai/` | - -**Step 3: Create a Forward Auth (Proxy) Provider for Traefik** - -In Authentik Admin → Providers → Create: - -| Field | Value | -|-------|-------| -| Name | `Traefik Forward Auth — Core API` | -| Protocol | Proxy | -| Mode | Forward auth (single application) | -| External host | `https://api.lthn.ai` | - -**Step 4: Create an Outpost (if not exists)** - -In Authentik Admin → Outposts: -- If no outpost exists: Create → Type: Proxy, Integration: Local Docker -- Add both providers to the outpost - -**Step 5: Test forward auth is working** - -```bash -# This should redirect to Authentik login -curl -sI https://api.lthn.ai/ -``` - -Once authenticated, Traefik passes the X-authentik-* headers through. - ---- - -### Task 5: go-api Authentik User Type (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/authentik.go` -- Create: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Create `authentik_test.go`: -```go -package api_test - -import ( - "testing" - - api "forge.lthn.ai/core/go-api" -) - -func TestAuthentikUser_Good(t *testing.T) { - user := &api.AuthentikUser{ - Username: "alice", - Email: "alice@example.com", - Name: "Alice Smith", - UID: "abc-123", - Groups: []string{"admins", "developers"}, - } - - if user.Username != "alice" { - t.Fatalf("expected username alice, got %s", user.Username) - } - if len(user.Groups) != 2 { - t.Fatalf("expected 2 groups, got %d", len(user.Groups)) - } -} - -func TestAuthentikUserHasGroup_Good(t *testing.T) { - user := &api.AuthentikUser{ - Groups: []string{"admins", "developers"}, - } - - if !user.HasGroup("admins") { - t.Fatal("expected user to have admins group") - } - if user.HasGroup("viewers") { - t.Fatal("expected user to not have viewers group") - } -} - -func TestAuthentikUserHasGroup_Bad_Empty(t *testing.T) { - user := &api.AuthentikUser{} - - if user.HasGroup("admins") { - t.Fatal("expected empty user to have no groups") - } -} - -func TestAuthentikConfig_Good(t *testing.T) { - cfg := api.AuthentikConfig{ - Issuer: "https://auth.host.uk.com/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, - } - - if cfg.Issuer == "" { - t.Fatal("expected non-empty issuer") - } - if !cfg.TrustedProxy { - t.Fatal("expected TrustedProxy to be true") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestAuthentik -``` - -Expected: Compilation errors — `api.AuthentikUser`, `api.AuthentikConfig` not defined. - -**Step 3: Implement authentik.go** - -Create `authentik.go`: -```go -package api - -// AuthentikConfig configures Authentik OIDC integration. -type AuthentikConfig struct { - // Issuer is the OIDC issuer URL (e.g. "https://auth.host.uk.com/application/o/core-api/"). - // Used for JWT validation via OIDC discovery. - Issuer string - - // ClientID is the OAuth2 client identifier registered in Authentik. - ClientID string - - // TrustedProxy enables reading X-authentik-* headers set by Traefik forward auth. - // Only enable this when go-api sits behind a trusted reverse proxy. - TrustedProxy bool - - // PublicPaths lists path prefixes that skip authentication entirely. - // /health and /swagger are always public regardless of this setting. - PublicPaths []string -} - -// AuthentikUser represents an authenticated user extracted from Authentik headers or JWT claims. -type AuthentikUser struct { - Username string `json:"username"` - Email string `json:"email"` - Name string `json:"name"` - UID string `json:"uid"` - Groups []string `json:"groups"` - Entitlements []string `json:"entitlements,omitempty"` - JWT string `json:"-"` -} - -// HasGroup returns true if the user belongs to the named group. -func (u *AuthentikUser) HasGroup(group string) bool { - for _, g := range u.Groups { - if g == group { - return true - } - } - return false -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestAuthentik -``` - -Expected: All 4 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go -git commit -m "feat: add AuthentikUser and AuthentikConfig types - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: go-api Header Extraction Middleware (TDD) - -This implements the forward auth path — extracting user identity from X-authentik-* headers set by Traefik. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -import ( - "encoding/json" - "net/http" - "net/http/httptest" - - "github.com/gin-gonic/gin" -) - -// authentikTestGroup returns JSON with the user from context. -type authentikTestGroup struct{} - -func (g *authentikTestGroup) Name() string { return "authtest" } -func (g *authentikTestGroup) BasePath() string { return "/v1/authtest" } -func (g *authentikTestGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/whoami", func(c *gin.Context) { - user := api.GetUser(c) - if user == nil { - c.JSON(200, api.OK[any](nil)) - return - } - c.JSON(200, api.OK(user)) - }) -} - -func TestForwardAuthHeaders_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-email", "alice@example.com") - req.Header.Set("X-authentik-name", "Alice Smith") - req.Header.Set("X-authentik-uid", "abc-123") - req.Header.Set("X-authentik-groups", "admins|developers") - req.Header.Set("X-authentik-entitlements", "core:read|core:write") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data == nil { - t.Fatal("expected non-nil user data") - } - if resp.Data.Username != "alice" { - t.Fatalf("expected username alice, got %s", resp.Data.Username) - } - if resp.Data.Email != "alice@example.com" { - t.Fatalf("expected email alice@example.com, got %s", resp.Data.Email) - } - if len(resp.Data.Groups) != 2 { - t.Fatalf("expected 2 groups, got %d", len(resp.Data.Groups)) - } - if resp.Data.Groups[0] != "admins" { - t.Fatalf("expected first group admins, got %s", resp.Data.Groups[0]) - } -} - -func TestForwardAuthHeaders_Good_NoHeaders(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - // Request without Authentik headers — should pass through (middleware is permissive) - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user when no headers present") - } -} - -func TestForwardAuthHeaders_Bad_NotTrusted(t *testing.T) { - gin.SetMode(gin.TestMode) - // TrustedProxy: false — should NOT read X-authentik-* headers - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: false, - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("X-authentik-username", "alice") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user when TrustedProxy is false") - } -} - -func TestHealthBypassesAuthentik_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - TrustedProxy: true, - })) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for /health, got %d", w.Code) - } -} - -func TestGetUser_Good_NilContext(t *testing.T) { - gin.SetMode(gin.TestMode) - // Test GetUser with no user in context (no Authentik middleware) - engine, _ := api.New() - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestForwardAuth\|TestHealthBypassesAuthentik\|TestGetUser -``` - -Expected: Compilation errors — `api.WithAuthentik`, `api.GetUser` not defined. - -**Step 3: Add GetUser helper and middleware to authentik.go** - -Append to `authentik.go`: -```go -import ( - "strings" - - "github.com/gin-gonic/gin" -) - -const authentikUserKey = "authentik_user" - -// GetUser returns the authenticated Authentik user from the Gin context, or nil -// if no user is authenticated. -func GetUser(c *gin.Context) *AuthentikUser { - val, exists := c.Get(authentikUserKey) - if !exists { - return nil - } - user, ok := val.(*AuthentikUser) - if !ok { - return nil - } - return user -} - -// authentikMiddleware extracts user identity from X-authentik-* headers -// (when TrustedProxy is true) and stores it in the Gin context. -// This middleware is PERMISSIVE — it does not reject unauthenticated requests. -// Handlers must check GetUser() and decide whether to require auth. -func authentikMiddleware(cfg AuthentikConfig) gin.HandlerFunc { - publicPaths := append([]string{"/health", "/swagger"}, cfg.PublicPaths...) - - return func(c *gin.Context) { - // Skip public paths entirely. - for _, path := range publicPaths { - if strings.HasPrefix(c.Request.URL.Path, path) { - c.Next() - return - } - } - - // Forward auth mode: read trusted headers from Traefik. - if cfg.TrustedProxy { - username := c.GetHeader("X-authentik-username") - if username != "" { - user := &AuthentikUser{ - Username: username, - Email: c.GetHeader("X-authentik-email"), - Name: c.GetHeader("X-authentik-name"), - UID: c.GetHeader("X-authentik-uid"), - JWT: c.GetHeader("X-authentik-jwt"), - } - - if groups := c.GetHeader("X-authentik-groups"); groups != "" { - user.Groups = strings.Split(groups, "|") - } - if ent := c.GetHeader("X-authentik-entitlements"); ent != "" { - user.Entitlements = strings.Split(ent, "|") - } - - c.Set(authentikUserKey, user) - } - } - - c.Next() - } -} -``` - -**Step 4: Add WithAuthentik option to options.go** - -Append to `options.go`: -```go -// WithAuthentik adds Authentik identity middleware. -// When TrustedProxy is true, reads X-authentik-* headers from Traefik forward auth. -// When Issuer is set, also validates JWT Bearer tokens via OIDC discovery. -func WithAuthentik(cfg AuthentikConfig) Option { - return func(e *Engine) { - e.middlewares = append(e.middlewares, authentikMiddleware(cfg)) - } -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS (existing 36 + new 5). - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go options.go -git commit -m "feat: add Authentik header extraction middleware and GetUser helper - -Forward auth mode reads X-authentik-* headers from Traefik. -Middleware is permissive — handlers decide whether auth is required. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 7: go-api JWT Validation Middleware (TDD) - -This implements the direct OIDC path — validating JWT Bearer tokens for API clients. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` -- Modify: `/Users/snider/Code/go-api/go.mod` (new dependency) - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -func TestJWTValidation_Bad_InvalidToken(t *testing.T) { - gin.SetMode(gin.TestMode) - // Use a fake issuer — OIDC discovery will fail, but we test the flow - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.example.com/application/o/test/", - ClientID: "test-client", - })) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("Authorization", "Bearer invalid-jwt-token") - handler.ServeHTTP(w, req) - - // Without a reachable OIDC endpoint, JWT validation can't succeed. - // The middleware should pass through (permissive) with no user. - if w.Code != 200 { - t.Fatalf("expected 200 (permissive), got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data != nil { - t.Fatal("expected nil user for invalid JWT") - } -} - -func TestBearerAndAuthentikCoexist_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - // Both WithBearerAuth and WithAuthentik should work together. - // Bearer auth gates access, Authentik extracts user identity. - engine, _ := api.New( - api.WithBearerAuth("secret-token"), - api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true}), - ) - engine.Register(&authentikTestGroup{}) - handler := engine.Handler() - - // With bearer token + Authentik headers → 200 with user - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/authtest/whoami", nil) - req.Header.Set("Authorization", "Bearer secret-token") - req.Header.Set("X-authentik-username", "bob") - req.Header.Set("X-authentik-email", "bob@example.com") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp api.Response[*api.AuthentikUser] - json.Unmarshal(w.Body.Bytes(), &resp) - if resp.Data == nil { - t.Fatal("expected user data") - } - if resp.Data.Username != "bob" { - t.Fatalf("expected username bob, got %s", resp.Data.Username) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestJWTValidation\|TestBearerAndAuthentikCoexist -``` - -**Step 3: Add OIDC validation to authentik middleware** - -Update `authentikMiddleware` in `authentik.go` to handle JWT Bearer tokens when `Issuer` is configured. Add the go-oidc dependency: - -```bash -cd /Users/snider/Code/go-api -go get github.com/coreos/go-oidc/v3/oidc -go get golang.org/x/oauth2 -``` - -Add JWT validation logic to the middleware — after the header extraction block, before `c.Next()`: - -```go -// Direct OIDC mode: validate JWT from Authorization header. -if cfg.Issuer != "" && cfg.ClientID != "" { - // Only attempt JWT validation if no user was extracted from headers - // (headers take priority — they're pre-validated by Authentik). - if GetUser(c) == nil { - authHeader := c.GetHeader("Authorization") - if strings.HasPrefix(authHeader, "Bearer ") { - token := strings.TrimPrefix(authHeader, "Bearer ") - user, err := validateJWT(c.Request.Context(), cfg, token) - if err == nil && user != nil { - c.Set(authentikUserKey, user) - } - // Permissive: if validation fails, continue without user. - } - } -} -``` - -Add the validation function: -```go -import ( - "context" - "sync" - - oidc "github.com/coreos/go-oidc/v3/oidc" -) - -var ( - oidcProviderMu sync.Mutex - oidcProviders = make(map[string]*oidc.Provider) -) - -// getOIDCProvider returns a cached OIDC provider for the given issuer. -func getOIDCProvider(ctx context.Context, issuer string) (*oidc.Provider, error) { - oidcProviderMu.Lock() - defer oidcProviderMu.Unlock() - - if p, ok := oidcProviders[issuer]; ok { - return p, nil - } - - p, err := oidc.NewProvider(ctx, issuer) - if err != nil { - return nil, err - } - oidcProviders[issuer] = p - return p, nil -} - -// validateJWT verifies a JWT token against the OIDC provider and extracts the user. -func validateJWT(ctx context.Context, cfg AuthentikConfig, rawToken string) (*AuthentikUser, error) { - provider, err := getOIDCProvider(ctx, cfg.Issuer) - if err != nil { - return nil, err - } - - verifier := provider.Verifier(&oidc.Config{ClientID: cfg.ClientID}) - idToken, err := verifier.Verify(ctx, rawToken) - if err != nil { - return nil, err - } - - var claims struct { - PreferredUsername string `json:"preferred_username"` - Email string `json:"email"` - Name string `json:"name"` - Sub string `json:"sub"` - Groups []string `json:"groups"` - } - if err := idToken.Claims(&claims); err != nil { - return nil, err - } - - return &AuthentikUser{ - Username: claims.PreferredUsername, - Email: claims.Email, - Name: claims.Name, - UID: claims.Sub, - Groups: claims.Groups, - JWT: rawToken, - }, nil -} -``` - -**Step 4: Run go mod tidy** - -```bash -cd /Users/snider/Code/go-api -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go go.mod go.sum -git commit -m "feat: add OIDC JWT validation for direct API client auth - -Uses coreos/go-oidc for OIDC discovery and JWT verification. -Cached provider instances. Permissive — fails open if OIDC unreachable. -Forward auth headers take priority over JWT when both present. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: go-api RequireAuth Middleware Helper (TDD) - -The Authentik middleware is permissive. This task adds a helper for routes that REQUIRE authentication. - -**Files:** -- Modify: `/Users/snider/Code/go-api/authentik.go` -- Modify: `/Users/snider/Code/go-api/authentik_test.go` - -**Step 1: Write the failing tests** - -Append to `authentik_test.go`: -```go -// protectedGroup uses RequireAuth on its routes. -type protectedGroup struct{} - -func (g *protectedGroup) Name() string { return "protected" } -func (g *protectedGroup) BasePath() string { return "/v1/protected" } -func (g *protectedGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/data", api.RequireAuth(), func(c *gin.Context) { - user := api.GetUser(c) - c.JSON(200, api.OK(user.Username)) - }) -} - -func TestRequireAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-email", "alice@example.com") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 with auth, got %d", w.Code) - } -} - -func TestRequireAuth_Bad_NoUser(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 without auth, got %d", w.Code) - } -} - -func TestRequireAuth_Bad_NoAuthentikMiddleware(t *testing.T) { - gin.SetMode(gin.TestMode) - // No WithAuthentik — RequireAuth should still return 401 - engine, _ := api.New() - engine.Register(&protectedGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/protected/data", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401, got %d", w.Code) - } -} - -// groupRequireGroup uses RequireGroup. -type groupRequireGroup struct{} - -func (g *groupRequireGroup) Name() string { return "adminonly" } -func (g *groupRequireGroup) BasePath() string { return "/v1/admin" } -func (g *groupRequireGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/panel", api.RequireGroup("admins"), func(c *gin.Context) { - c.JSON(200, api.OK("admin panel")) - }) -} - -func TestRequireGroup_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&groupRequireGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/admin/panel", nil) - req.Header.Set("X-authentik-username", "alice") - req.Header.Set("X-authentik-groups", "admins|developers") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for admin user, got %d", w.Code) - } -} - -func TestRequireGroup_Bad_WrongGroup(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithAuthentik(api.AuthentikConfig{TrustedProxy: true})) - engine.Register(&groupRequireGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/admin/panel", nil) - req.Header.Set("X-authentik-username", "bob") - req.Header.Set("X-authentik-groups", "developers") - handler.ServeHTTP(w, req) - - if w.Code != 403 { - t.Fatalf("expected 403 for non-admin user, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -run TestRequireAuth\|TestRequireGroup -``` - -Expected: Compilation errors — `api.RequireAuth`, `api.RequireGroup` not defined. - -**Step 3: Implement RequireAuth and RequireGroup** - -Append to `authentik.go`: -```go -import "net/http" - -// RequireAuth is a Gin middleware that returns 401 if no authenticated user -// is present in the context. Use after WithAuthentik() middleware. -func RequireAuth() gin.HandlerFunc { - return func(c *gin.Context) { - if GetUser(c) == nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - Fail("unauthorised", "Authentication required")) - return - } - c.Next() - } -} - -// RequireGroup is a Gin middleware that returns 403 if the authenticated user -// does not belong to the specified group. Implies RequireAuth. -func RequireGroup(group string) gin.HandlerFunc { - return func(c *gin.Context) { - user := GetUser(c) - if user == nil { - c.AbortWithStatusJSON(http.StatusUnauthorized, - Fail("unauthorised", "Authentication required")) - return - } - if !user.HasGroup(group) { - c.AbortWithStatusJSON(http.StatusForbidden, - Fail("forbidden", "Insufficient permissions")) - return - } - c.Next() - } -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add authentik.go authentik_test.go -git commit -m "feat: add RequireAuth and RequireGroup middleware helpers - -RequireAuth returns 401 when no user in context. -RequireGroup returns 403 when user lacks the specified group. -Both use British English 'unauthorised' in error responses. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Update go-api Documentation - -**Files:** -- Modify: `/Users/snider/Code/go-api/CLAUDE.md` -- Modify: `/Users/snider/Code/go-api/README.md` - -**Step 1: Update CLAUDE.md** - -Add to the Project Overview section: -```markdown -## Authentik Integration - -go-api supports Authentik as the identity provider: - -- **Forward auth mode**: Reads `X-authentik-*` headers from Traefik (requires `TrustedProxy: true`) -- **OIDC mode**: Validates JWT Bearer tokens via OIDC discovery -- **Permissive middleware**: `WithAuthentik()` extracts user but doesn't block. Use `RequireAuth()` / `RequireGroup()` on routes that need auth. -- **Coexists with `WithBearerAuth()`** for service-to-service tokens - -```go -engine, _ := api.New( - api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.host.uk.com/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, - }), -) -``` -``` - -**Step 2: Update README.md** - -Add Authentik section with quick-start example showing `WithAuthentik()`, `GetUser()`, `RequireAuth()`, and `RequireGroup()`. - -**Step 3: Commit** - -```bash -cd /Users/snider/Code/go-api -git add CLAUDE.md README.md -git commit -m "docs: add Authentik integration guide to CLAUDE.md and README - -Co-Authored-By: Virgil " -``` - ---- - -### Task 10: Push go-api and DevOps Changes - -**Step 1: Push go-api** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 # Final verification -git push forge main -``` - -**Step 2: Push DevOps** - -```bash -cd /Users/snider/Code/DevOps -git push forge main -``` - -**Step 3: Update go-ecosystem memory** - -Update the go-api entry in the ecosystem inventory to note Authentik middleware. - ---- - -## Dependency Summary - -``` -Task 1 (enable vars) → Task 2 (playbook) → Task 3 (deploy) → Task 4 (OIDC app) - ↓ -Task 5 (user type) → Task 6 (header middleware) → Task 7 (JWT) → Task 8 (RequireAuth) - ↓ - Task 9 (docs) → Task 10 (push) -``` - -Tasks 1-4 (DevOps) and Tasks 5-8 (Go) are independent tracks that can run in parallel. Task 9-10 depend on both tracks. - -## Estimated Sizes - -| Task | LOC | Tests | -|------|-----|-------| -| Task 5: User type | ~50 | 4 | -| Task 6: Header middleware | ~60 | 5 | -| Task 7: JWT validation | ~80 | 2 | -| Task 8: RequireAuth/Group | ~30 | 5 | -| **go-api total** | **~220** | **16** | diff --git a/docs/plans/2026-02-21-core-help-design.md b/docs/plans/2026-02-21-core-help-design.md deleted file mode 100644 index 2943178..0000000 --- a/docs/plans/2026-02-21-core-help-design.md +++ /dev/null @@ -1,155 +0,0 @@ -# core.help Documentation Website — Design - -**Date:** 2026-02-21 -**Author:** Virgil -**Status:** Design approved -**Domain:** https://core.help - -## Problem - -Documentation is scattered across 39 repos (18 Go packages, 20 PHP packages, 1 CLI). There is no unified docs site. Developers need a single entry point to find CLI commands, Go package APIs, MCP tool references, and PHP module guides. - -## Solution - -A Hugo + Docsy static site at core.help, built from existing markdown docs aggregated by `core docs sync`. No new content — just collect and present what already exists across the ecosystem. - -## Architecture - -### Stack - -- **Hugo** — Go-native static site generator, sub-second builds -- **Docsy theme** — Purpose-built for technical docs (used by Kubernetes, gRPC, Knative) -- **BunnyCDN** — Static hosting with pull zone -- **`core docs sync --target hugo`** — Collects markdown from all repos into Hugo content tree - -### Why Hugo + Docsy (not VitePress or mdBook) - -- Go-native, no Node.js dependency -- Handles multi-section navigation (CLI, Go packages, PHP modules, MCP tools) -- Sub-second builds for ~250 markdown files -- Docsy has built-in search, versioned nav, API reference sections - -## Content Structure - -``` -docs-site/ -├── hugo.toml -├── content/ -│ ├── _index.md # Landing page -│ ├── getting-started/ # CLI top-level guides -│ │ ├── _index.md -│ │ ├── installation.md -│ │ ├── configuration.md -│ │ ├── user-guide.md -│ │ ├── troubleshooting.md -│ │ └── faq.md -│ ├── cli/ # CLI command reference (43 commands) -│ │ ├── _index.md -│ │ ├── dev/ # core dev commit, push, pull, etc. -│ │ ├── ai/ # core ai commands -│ │ ├── go/ # core go test, lint, etc. -│ │ └── ... -│ ├── go/ # Go ecosystem packages (18) -│ │ ├── _index.md # Ecosystem overview -│ │ ├── go-api/ # README + architecture/development/history -│ │ ├── go-ai/ -│ │ ├── go-mlx/ -│ │ ├── go-i18n/ -│ │ └── ... -│ ├── mcp/ # MCP tool reference (49 tools) -│ │ ├── _index.md -│ │ ├── file-operations.md -│ │ ├── process-management.md -│ │ ├── rag.md -│ │ └── ... -│ ├── php/ # PHP packages (from core-php/docs/packages/) -│ │ ├── _index.md -│ │ ├── admin/ -│ │ ├── tenant/ -│ │ ├── commerce/ -│ │ └── ... -│ └── kb/ # Knowledge base (wiki pages from go-mlx, go-i18n) -│ ├── _index.md -│ ├── mlx/ -│ └── i18n/ -├── static/ # Logos, favicons -├── layouts/ # Custom template overrides (minimal) -└── go.mod # Hugo modules (Docsy as module dep) -``` - -## Sync Pipeline - -`core docs sync --target hugo --output site/content/` performs: - -### Source Mapping - -``` -cli/docs/index.md → content/getting-started/_index.md -cli/docs/getting-started.md → content/getting-started/installation.md -cli/docs/user-guide.md → content/getting-started/user-guide.md -cli/docs/configuration.md → content/getting-started/configuration.md -cli/docs/troubleshooting.md → content/getting-started/troubleshooting.md -cli/docs/faq.md → content/getting-started/faq.md - -core/docs/cmd/**/*.md → content/cli/**/*.md - -go-*/README.md → content/go/{name}/_index.md -go-*/docs/*.md → content/go/{name}/*.md -go-*/KB/*.md → content/kb/{name-suffix}/*.md - -core-*/docs/**/*.md → content/php/{name-suffix}/**/*.md -``` - -### Front Matter Injection - -If a markdown file doesn't start with `---`, prepend: - -```yaml ---- -title: "{derived from filename}" -linkTitle: "{short name}" -weight: {auto-incremented} ---- -``` - -No other content transformations. Markdown stays as-is. - -### Build & Deploy - -```bash -core docs sync --target hugo --output docs-site/content/ -cd docs-site && hugo build -hugo deploy --target bunnycdn -``` - -Hugo deploy config in `hugo.toml`: - -```toml -[deployment] -[[deployment.targets]] -name = "bunnycdn" -URL = "s3://core-help?endpoint=storage.bunnycdn.com®ion=auto" -``` - -Credentials via env vars. - -## Registry - -All 39 repos registered in `.core/repos.yaml` with `docs: true`. Go repos use explicit `path:` fields since they live outside the PHP `base_path`. `FindRegistry()` checks `.core/repos.yaml` alongside `repos.yaml`. - -## Prerequisites Completed - -- [x] `.core/repos.yaml` created with all 39 repos -- [x] `FindRegistry()` updated to find `.core/repos.yaml` -- [x] `Repo.Path` supports explicit YAML override -- [x] go-api docs gap filled (architecture.md, development.md, history.md) -- [x] All 18 Go repos have standard docs trio - -## What Remains (Implementation Plan) - -1. Create docs-site repo with Hugo + Docsy scaffold -2. Extend `core docs sync` with `--target hugo` mode -3. Write section _index.md files (landing page, section intros) -4. Hugo config (navigation, search, theme colours) -5. BunnyCDN deployment config -6. CI pipeline on Forge (optional — can deploy manually initially) diff --git a/docs/plans/2026-02-21-core-help-plan.md b/docs/plans/2026-02-21-core-help-plan.md deleted file mode 100644 index e3bf5e1..0000000 --- a/docs/plans/2026-02-21-core-help-plan.md +++ /dev/null @@ -1,642 +0,0 @@ -# core.help Hugo Documentation Site — Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build a Hugo + Docsy documentation site at core.help that aggregates markdown from 39 repos via `core docs sync --target hugo`. - -**Architecture:** Hugo static site with Docsy theme, populated by extending `core docs sync` with a `--target hugo` flag that maps repo docs into Hugo's `content/` tree with auto-injected front matter. Deploy to BunnyCDN. - -**Tech Stack:** Hugo (Go SSG), Docsy theme (Hugo module), BunnyCDN, `core docs sync` CLI - ---- - -## Context - -The docs sync command lives in `/Users/snider/Code/host-uk/cli/cmd/docs/`. The site will be scaffolded at `/Users/snider/Code/host-uk/docs-site/`. The registry at `/Users/snider/Code/host-uk/.core/repos.yaml` already contains all 39 repos (20 PHP + 18 Go + 1 CLI) with explicit paths for Go repos. - -Key files: -- `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_sync.go` — sync command (modify) -- `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_scan.go` — repo scanner (modify) -- `/Users/snider/Code/host-uk/docs-site/` — Hugo site (create) - -## Task 1: Scaffold Hugo + Docsy site - -**Files:** -- Create: `/Users/snider/Code/host-uk/docs-site/hugo.toml` -- Create: `/Users/snider/Code/host-uk/docs-site/go.mod` -- Create: `/Users/snider/Code/host-uk/docs-site/content/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/getting-started/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/cli/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/go/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/mcp/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/php/_index.md` -- Create: `/Users/snider/Code/host-uk/docs-site/content/kb/_index.md` - -This is the one-time Hugo scaffolding. No tests — just files. - -**`hugo.toml`:** -```toml -baseURL = "https://core.help/" -title = "Core Documentation" -languageCode = "en" -defaultContentLanguage = "en" - -enableRobotsTXT = true -enableGitInfo = false - -[outputs] -home = ["HTML", "JSON"] -section = ["HTML"] - -[params] -description = "Documentation for the Core CLI, Go packages, PHP modules, and MCP tools" -copyright = "Host UK — EUPL-1.2" - -[params.ui] -sidebar_menu_compact = true -breadcrumb_disable = false -sidebar_search_disable = false -navbar_logo = false - -[params.ui.readingtime] -enable = false - -[module] -proxy = "direct" - -[module.hugoVersion] -extended = true -min = "0.120.0" - -[[module.imports]] -path = "github.com/google/docsy" -disable = false - -[markup.goldmark.renderer] -unsafe = true - -[menu] -[[menu.main]] -name = "Getting Started" -weight = 10 -url = "/getting-started/" -[[menu.main]] -name = "CLI Reference" -weight = 20 -url = "/cli/" -[[menu.main]] -name = "Go Packages" -weight = 30 -url = "/go/" -[[menu.main]] -name = "MCP Tools" -weight = 40 -url = "/mcp/" -[[menu.main]] -name = "PHP Packages" -weight = 50 -url = "/php/" -[[menu.main]] -name = "Knowledge Base" -weight = 60 -url = "/kb/" -``` - -**`go.mod`:** -``` -module github.com/host-uk/docs-site - -go 1.22 - -require github.com/google/docsy v0.11.0 -``` - -Note: Run `hugo mod get` after creating these files to populate `go.sum` and download Docsy. - -**Section `_index.md` files** — each needs Hugo front matter: - -`content/_index.md`: -```markdown ---- -title: "Core Documentation" -description: "Documentation for the Core CLI, Go packages, PHP modules, and MCP tools" ---- - -Welcome to the Core ecosystem documentation. - -## Sections - -- [Getting Started](/getting-started/) — Installation, configuration, and first steps -- [CLI Reference](/cli/) — Command reference for `core` CLI -- [Go Packages](/go/) — Go ecosystem package documentation -- [MCP Tools](/mcp/) — Model Context Protocol tool reference -- [PHP Packages](/php/) — PHP module documentation -- [Knowledge Base](/kb/) — Wiki articles and deep dives -``` - -`content/getting-started/_index.md`: -```markdown ---- -title: "Getting Started" -linkTitle: "Getting Started" -weight: 10 -description: "Installation, configuration, and first steps with the Core CLI" ---- -``` - -`content/cli/_index.md`: -```markdown ---- -title: "CLI Reference" -linkTitle: "CLI Reference" -weight: 20 -description: "Command reference for the core CLI tool" ---- -``` - -`content/go/_index.md`: -```markdown ---- -title: "Go Packages" -linkTitle: "Go Packages" -weight: 30 -description: "Documentation for the Go ecosystem packages" ---- -``` - -`content/mcp/_index.md`: -```markdown ---- -title: "MCP Tools" -linkTitle: "MCP Tools" -weight: 40 -description: "Model Context Protocol tool reference — file operations, RAG, ML inference, process management" ---- -``` - -`content/php/_index.md`: -```markdown ---- -title: "PHP Packages" -linkTitle: "PHP Packages" -weight: 50 -description: "Documentation for the PHP module ecosystem" ---- -``` - -`content/kb/_index.md`: -```markdown ---- -title: "Knowledge Base" -linkTitle: "Knowledge Base" -weight: 60 -description: "Wiki articles, deep dives, and reference material" ---- -``` - -**Verify:** After creating files, run from `/Users/snider/Code/host-uk/docs-site/`: -```bash -hugo mod get -hugo server -``` -The site should start and show the landing page with Docsy theme at `localhost:1313`. - -**Commit:** -```bash -cd /Users/snider/Code/host-uk/docs-site -git init -git add . -git commit -m "feat: scaffold Hugo + Docsy documentation site" -``` - ---- - -## Task 2: Extend scanRepoDocs to collect KB/ and README - -**Files:** -- Modify: `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_scan.go` - -Currently `scanRepoDocs` only collects files from `docs/`. For the Hugo target we also need: -- `KB/**/*.md` files (wiki pages from go-mlx, go-i18n) -- `README.md` content (becomes the package _index.md) - -Add a `KBFiles []string` field to `RepoDocInfo` and scan `KB/` alongside `docs/`: - -```go -type RepoDocInfo struct { - Name string - Path string - HasDocs bool - Readme string - ClaudeMd string - Changelog string - DocsFiles []string // All files in docs/ directory (recursive) - KBFiles []string // All files in KB/ directory (recursive) -} -``` - -In `scanRepoDocs`, after the `docs/` walk, add a second walk for `KB/`: - -```go -// Recursively scan KB/ directory for .md files -kbDir := filepath.Join(repo.Path, "KB") -if _, err := io.Local.List(kbDir); err == nil { - _ = filepath.WalkDir(kbDir, func(path string, d fs.DirEntry, err error) error { - if err != nil { - return nil - } - if d.IsDir() || !strings.HasSuffix(d.Name(), ".md") { - return nil - } - relPath, _ := filepath.Rel(kbDir, path) - info.KBFiles = append(info.KBFiles, relPath) - info.HasDocs = true - return nil - }) -} -``` - -**Tests:** The existing tests should still pass. No new test file needed — this is a data-collection change. - -**Verify:** `cd /Users/snider/Code/host-uk/cli && GOWORK=off go build ./cmd/docs/...` - -**Commit:** -```bash -git add cmd/docs/cmd_scan.go -git commit -m "feat(docs): scan KB/ directory alongside docs/" -``` - ---- - -## Task 3: Add `--target hugo` flag and Hugo sync logic - -**Files:** -- Modify: `/Users/snider/Code/host-uk/cli/cmd/docs/cmd_sync.go` - -This is the main task. Add a `--target` flag (default `"php"`) and a new `runHugoSync` function that maps repos to Hugo's content tree. - -**Add flag variable and registration:** - -```go -var ( - docsSyncRegistryPath string - docsSyncDryRun bool - docsSyncOutputDir string - docsSyncTarget string -) - -func init() { - docsSyncCmd.Flags().StringVar(&docsSyncRegistryPath, "registry", "", i18n.T("common.flag.registry")) - docsSyncCmd.Flags().BoolVar(&docsSyncDryRun, "dry-run", false, i18n.T("cmd.docs.sync.flag.dry_run")) - docsSyncCmd.Flags().StringVar(&docsSyncOutputDir, "output", "", i18n.T("cmd.docs.sync.flag.output")) - docsSyncCmd.Flags().StringVar(&docsSyncTarget, "target", "php", "Target format: php (default) or hugo") -} -``` - -**Update RunE to pass target:** -```go -RunE: func(cmd *cli.Command, args []string) error { - return runDocsSync(docsSyncRegistryPath, docsSyncOutputDir, docsSyncDryRun, docsSyncTarget) -}, -``` - -**Update `runDocsSync` signature and add target dispatch:** -```go -func runDocsSync(registryPath string, outputDir string, dryRun bool, target string) error { - reg, basePath, err := loadRegistry(registryPath) - if err != nil { - return err - } - - switch target { - case "hugo": - return runHugoSync(reg, basePath, outputDir, dryRun) - default: - return runPHPSync(reg, basePath, outputDir, dryRun) - } -} -``` - -**Rename current sync body to `runPHPSync`** — extract lines 67-159 of current `runDocsSync` into `runPHPSync(reg, basePath, outputDir string, dryRun bool) error`. This is a pure extract, no logic changes. - -**Add `hugoOutputName` mapping function:** -```go -// hugoOutputName maps repo name to Hugo content section and folder. -// Returns (section, folder) where section is the top-level content dir. -func hugoOutputName(repoName string) (string, string) { - // CLI guides - if repoName == "cli" { - return "getting-started", "" - } - // Core CLI command docs - if repoName == "core" { - return "cli", "" - } - // Go packages - if strings.HasPrefix(repoName, "go-") { - return "go", repoName - } - // PHP packages - if strings.HasPrefix(repoName, "core-") { - return "php", strings.TrimPrefix(repoName, "core-") - } - return "go", repoName -} -``` - -**Add front matter injection helper:** -```go -// injectFrontMatter prepends Hugo front matter to markdown content if missing. -func injectFrontMatter(content []byte, title string, weight int) []byte { - // Already has front matter - if bytes.HasPrefix(bytes.TrimSpace(content), []byte("---")) { - return content - } - fm := fmt.Sprintf("---\ntitle: %q\nweight: %d\n---\n\n", title, weight) - return append([]byte(fm), content...) -} - -// titleFromFilename derives a human-readable title from a filename. -func titleFromFilename(filename string) string { - name := strings.TrimSuffix(filepath.Base(filename), ".md") - name = strings.ReplaceAll(name, "-", " ") - name = strings.ReplaceAll(name, "_", " ") - // Title case - words := strings.Fields(name) - for i, w := range words { - if len(w) > 0 { - words[i] = strings.ToUpper(w[:1]) + w[1:] - } - } - return strings.Join(words, " ") -} -``` - -**Add `runHugoSync` function:** -```go -func runHugoSync(reg *repos.Registry, basePath string, outputDir string, dryRun bool) error { - if outputDir == "" { - outputDir = filepath.Join(basePath, "docs-site", "content") - } - - // Scan all repos - var docsInfo []RepoDocInfo - for _, repo := range reg.List() { - if repo.Name == "core-template" || repo.Name == "core-claude" { - continue - } - info := scanRepoDocs(repo) - if info.HasDocs { - docsInfo = append(docsInfo, info) - } - } - - if len(docsInfo) == 0 { - cli.Text("No documentation found") - return nil - } - - cli.Print("\n Hugo sync: %d repos with docs → %s\n\n", len(docsInfo), outputDir) - - // Show plan - for _, info := range docsInfo { - section, folder := hugoOutputName(info.Name) - target := section - if folder != "" { - target = section + "/" + folder - } - fileCount := len(info.DocsFiles) + len(info.KBFiles) - if info.Readme != "" { - fileCount++ - } - cli.Print(" %s → %s/ (%d files)\n", repoNameStyle.Render(info.Name), target, fileCount) - } - - if dryRun { - cli.Print("\n Dry run — no files written\n") - return nil - } - - cli.Blank() - if !confirm("Sync to Hugo content directory?") { - cli.Text("Aborted") - return nil - } - - cli.Blank() - var synced int - for _, info := range docsInfo { - section, folder := hugoOutputName(info.Name) - - // Build destination path - destDir := filepath.Join(outputDir, section) - if folder != "" { - destDir = filepath.Join(destDir, folder) - } - - // Copy docs/ files - weight := 10 - docsDir := filepath.Join(info.Path, "docs") - for _, f := range info.DocsFiles { - src := filepath.Join(docsDir, f) - dst := filepath.Join(destDir, f) - if err := copyWithFrontMatter(src, dst, weight); err != nil { - cli.Print(" %s %s: %s\n", errorStyle.Render("✗"), f, err) - continue - } - weight += 10 - } - - // Copy README.md as _index.md (if not CLI/core which use their own index) - if info.Readme != "" && folder != "" { - dst := filepath.Join(destDir, "_index.md") - if err := copyWithFrontMatter(info.Readme, dst, 1); err != nil { - cli.Print(" %s README: %s\n", errorStyle.Render("✗"), err) - } - } - - // Copy KB/ files to kb/{suffix}/ - if len(info.KBFiles) > 0 { - // Extract suffix: go-mlx → mlx, go-i18n → i18n - suffix := strings.TrimPrefix(info.Name, "go-") - kbDestDir := filepath.Join(outputDir, "kb", suffix) - kbDir := filepath.Join(info.Path, "KB") - kbWeight := 10 - for _, f := range info.KBFiles { - src := filepath.Join(kbDir, f) - dst := filepath.Join(kbDestDir, f) - if err := copyWithFrontMatter(src, dst, kbWeight); err != nil { - cli.Print(" %s KB/%s: %s\n", errorStyle.Render("✗"), f, err) - continue - } - kbWeight += 10 - } - } - - cli.Print(" %s %s\n", successStyle.Render("✓"), info.Name) - synced++ - } - - cli.Print("\n Synced %d repos to Hugo content\n", synced) - return nil -} - -// copyWithFrontMatter copies a markdown file, injecting front matter if missing. -func copyWithFrontMatter(src, dst string, weight int) error { - if err := io.Local.EnsureDir(filepath.Dir(dst)); err != nil { - return err - } - content, err := io.Local.Read(src) - if err != nil { - return err - } - title := titleFromFilename(src) - result := injectFrontMatter([]byte(content), title, weight) - return io.Local.Write(dst, string(result)) -} -``` - -**Add imports** at top of file: -```go -import ( - "bytes" - "fmt" - "path/filepath" - "strings" - - "forge.lthn.ai/core/go/pkg/cli" - "forge.lthn.ai/core/go/pkg/i18n" - "forge.lthn.ai/core/go/pkg/io" - "forge.lthn.ai/core/go/pkg/repos" -) -``` - -**Verify:** `cd /Users/snider/Code/host-uk/cli && GOWORK=off go build ./cmd/docs/...` - -**Commit:** -```bash -git add cmd/docs/cmd_sync.go -git commit -m "feat(docs): add --target hugo sync mode for core.help" -``` - ---- - -## Task 4: Test the full pipeline - -**No code changes.** Run the pipeline end-to-end. - -**Step 1:** Sync docs to Hugo: -```bash -cd /Users/snider/Code/host-uk -core docs sync --target hugo --dry-run -``` -Verify all 39 repos appear with correct section mappings. - -**Step 2:** Run actual sync: -```bash -core docs sync --target hugo -``` - -**Step 3:** Build and preview: -```bash -cd /Users/snider/Code/host-uk/docs-site -hugo server -``` -Open `localhost:1313` and verify: -- Landing page renders with section links -- Getting Started section has CLI guides -- CLI Reference section has command docs -- Go Packages section has 18 packages with architecture/development/history -- PHP Packages section has PHP module docs -- Knowledge Base has MLX and i18n wiki pages -- Navigation works, search works - -**Step 4:** Fix any issues found during preview. - -**Commit docs-site content:** -```bash -cd /Users/snider/Code/host-uk/docs-site -git add content/ -git commit -m "feat: sync initial content from 39 repos" -``` - ---- - -## Task 5: BunnyCDN deployment config - -**Files:** -- Modify: `/Users/snider/Code/host-uk/docs-site/hugo.toml` - -Add deployment target: - -```toml -[deployment] -[[deployment.targets]] -name = "production" -URL = "s3://core-help?endpoint=storage.bunnycdn.com®ion=auto" -``` - -Add a `Taskfile.yml` for convenience: - -**Create:** `/Users/snider/Code/host-uk/docs-site/Taskfile.yml` -```yaml -version: '3' - -tasks: - dev: - desc: Start Hugo dev server - cmds: - - hugo server --buildDrafts - - build: - desc: Build static site - cmds: - - hugo --minify - - sync: - desc: Sync docs from all repos - dir: .. - cmds: - - core docs sync --target hugo - - deploy: - desc: Build and deploy to BunnyCDN - cmds: - - task: sync - - task: build - - hugo deploy --target production - - clean: - desc: Remove generated content (keeps _index.md files) - cmds: - - find content -name "*.md" ! -name "_index.md" -delete -``` - -**Verify:** `task dev` starts the site. - -**Commit:** -```bash -git add hugo.toml Taskfile.yml -git commit -m "feat: add BunnyCDN deployment config and Taskfile" -``` - ---- - -## Dependency Sequencing - -``` -Task 1 (Hugo scaffold) — independent, do first -Task 2 (scan KB/) — independent, can parallel with Task 1 -Task 3 (--target hugo) — depends on Task 2 -Task 4 (test pipeline) — depends on Tasks 1 + 3 -Task 5 (deploy config) — depends on Task 1 -``` - -## Verification - -After all tasks: -1. `core docs sync --target hugo` populates `docs-site/content/` from all repos -2. `cd docs-site && hugo server` renders the full site -3. Navigation has 6 sections: Getting Started, CLI, Go, MCP, PHP, KB -4. All existing markdown renders correctly with auto-injected front matter -5. `hugo build` produces `public/` with no errors diff --git a/docs/plans/completed/2026-02-05-core-ide-job-runner-design-original.md b/docs/plans/completed/2026-02-05-core-ide-job-runner-design-original.md deleted file mode 100644 index bec933a..0000000 --- a/docs/plans/completed/2026-02-05-core-ide-job-runner-design-original.md +++ /dev/null @@ -1,271 +0,0 @@ -# Core-IDE Job Runner Design - -**Date:** 2026-02-05 -**Status:** Approved -**Author:** @Snider + Claude - ---- - -## Goal - -Turn core-ide into an autonomous job runner that polls for actionable pipeline work, executes it via typed MCP tool handlers, captures JSONL training data, and self-updates. Supports 12 nodes running headless on servers and desktop on developer machines. - ---- - -## Architecture Overview - -``` -+-------------------------------------------------+ -| core-ide | -| | -| +----------+ +-----------+ +----------+ | -| | Poller |-->| Dispatcher|-->| Handler | | -| | (Source) | | (MCP route)| | Registry | | -| +----------+ +-----------+ +----------+ | -| | | | | -| | +----v----+ +---v-------+ | -| | | Journal | | JobSource | | -| | | (JSONL) | | (adapter) | | -| | +---------+ +-----------+ | -| +----v-----+ | -| | Updater | (existing internal/cmd/updater) | -| +----------+ | -+-------------------------------------------------+ -``` - -**Three components:** -- **Poller** -- Periodic scan via pluggable JobSource adapters. Builds PipelineSignal structs from API responses. Never reads comment bodies (injection vector). -- **Dispatcher** -- Matches signals against handler registry in priority order. One action per signal per cycle (prevents cascades). -- **Journal** -- Appends JSONL after each completed action per issue-epic step 10 spec. Structural signals only -- IDs, SHAs, timestamps, cycle counts, instructions sent, automations performed. - ---- - -## Job Source Abstraction - -GitHub is the first adapter. The platform's own Agentic API replaces it later. Handler logic is source-agnostic. - -```go -type JobSource interface { - Name() string - Poll(ctx context.Context) ([]*PipelineSignal, error) - Report(ctx context.Context, result *ActionResult) error -} -``` - -| Adapter | When | Transport | -|-------------------|-------|----------------------------------------| -| `GitHubSource` | Now | REST API + conditional requests (ETag) | -| `HostUKSource` | Next | Agentic API (WebSocket or poll) | -| `HyperswarmSource`| Later | P2P encrypted channels via Holepunch | - -**Multi-source:** Poller runs multiple sources concurrently. Own repos get priority. When idle (zero signals for N consecutive cycles), external project sources activate (WailsApp first). - -**API budget:** 50% credit allocation for harvest mode is a config value on the source, not hardcoded. - ---- - -## Pipeline Signal - -The structural snapshot passed to handlers. Never contains comment bodies or free text. - -```go -type PipelineSignal struct { - EpicNumber int - ChildNumber int - PRNumber int - RepoOwner string - RepoName string - PRState string // OPEN, MERGED, CLOSED - IsDraft bool - Mergeable string // MERGEABLE, CONFLICTING, UNKNOWN - CheckStatus string // SUCCESS, FAILURE, PENDING - ThreadsTotal int - ThreadsResolved int - LastCommitSHA string - LastCommitAt time.Time - LastReviewAt time.Time -} -``` - ---- - -## Handler Registry - -Each action from the issue-epic flow is a registered handler. All Go functions with typed inputs/outputs. - -```go -type JobHandler interface { - Name() string - Match(signal *PipelineSignal) bool - Execute(ctx context.Context, signal *PipelineSignal) (*ActionResult, error) -} -``` - -| Handler | Epic Stage | Input Signals | Action | -|--------------------|-----------|---------------------------------------------------|---------------------------------------------| -| `publish_draft` | 3 | PR draft=true, checks=SUCCESS | Mark PR as ready for review | -| `send_fix_command` | 4/6 | PR CONFLICTING or threads without fix commit | Comment "fix merge conflict" / "fix the code reviews" | -| `resolve_threads` | 5 | Unresolved threads, fix commit exists after review | Resolve all pre-commit threads | -| `enable_auto_merge`| 7 | PR MERGEABLE, checks passing, threads resolved | Enable auto-merge via API | -| `tick_parent` | 8 | Child PR merged | Update epic issue checklist | -| `close_child` | 9 | Child PR merged + parent ticked | Close child issue | -| `capture_journal` | 10 | Any completed action | Append JSONL entry | - -**ActionResult** carries what was done -- action name, target IDs, success/failure, timestamps. Feeds directly into JSONL journal. - -Handlers register at init time, same pattern as CLI commands in the existing codebase. - ---- - -## Headless vs Desktop Mode - -Same binary, same handlers, different UI surface. - -**Detection:** - -```go -func hasDisplay() bool { - if runtime.GOOS == "windows" { return true } - return os.Getenv("DISPLAY") != "" || os.Getenv("WAYLAND_DISPLAY") != "" -} -``` - -**Headless mode** (Linux server, no display): -- Skip Wails window creation -- Start poller immediately -- Start MCP bridge (port 9877) for external tool access -- Log to stdout/file (structured JSON) -- Updater: check on startup, auto-apply + restart via watcher -- Managed by systemd: `Restart=always` - -**Desktop mode** (display available): -- Full Wails system tray + webview panel -- Tray icon shows status: idle, polling, executing, error -- Tray menu: Start/Stop poller, Force update, Open journal, Configure sources -- Poller off by default (developer toggle) -- Same MCP bridge, same handlers, same journal - -**CLI override:** `core-ide --headless` forces headless. `core-ide --desktop` forces GUI. - -**Shared startup:** - -```go -func main() { - // 1. Load config (repos, interval, channel, sources) - // 2. Build handler registry - // 3. Init journal - // 4. Init updater (check on startup) - // 5. Branch: - if hasDisplay() { - startDesktop() // Wails + tray + optional poller - } else { - startHeadless() // Poller + MCP bridge + signal handling - } -} -``` - ---- - -## Poller Configuration - -```go -type PollerConfig struct { - Sources []JobSource - Handlers []JobHandler - Journal *Journal - PollInterval time.Duration // default: 60s - DryRun bool // log without executing -} -``` - -**Rate limiting:** GitHub API allows 5000 req/hr with token. Full scan of 4 repos with ~30 PRs uses ~150 requests. Poller uses conditional requests (If-None-Match/ETag) to avoid counting unchanged responses. Backs off to 5min interval when idle. - -**CLI flags:** -- `--poll-interval` (default: 60s) -- `--repos` (comma-separated: `host-uk/core,host-uk/core-php`) -- `--dry-run` (log actions without executing) -- `--headless` / `--desktop` (mode override) - ---- - -## Self-Update - -Uses existing `internal/cmd/updater` package. Binary-safe replacement with platform-specific watcher process, SemVer channel selection (stable/beta/alpha/dev), automatic rollback on failure. - -**Integration:** -- Headless: `CheckAndUpdateOnStartup` -- auto-apply + restart -- Desktop: `CheckOnStartup` -- notify via tray, user confirms - ---- - -## Training Data (Journal) - -JSONL format per issue-epic step 10. One record per completed action. - -```json -{ - "ts": "2026-02-05T12:00:00Z", - "epic": 299, - "child": 212, - "pr": 316, - "repo": "host-uk/core", - "action": "publish_draft", - "signals": { - "pr_state": "OPEN", - "is_draft": true, - "check_status": "SUCCESS", - "mergeable": "UNKNOWN", - "threads_total": 0, - "threads_resolved": 0 - }, - "result": { - "success": true, - "duration_ms": 340 - }, - "cycle": 1 -} -``` - -**Rules:** -- NO content (no comments, no messages, no bodies) -- Structural signals only -- safe for training -- Append-only JSONL file per node -- File path: `~/.core/journal//.jsonl` - ---- - -## Files Summary - -| File | Action | -|------|--------| -| `pkg/jobrunner/types.go` | CREATE -- JobSource, JobHandler, PipelineSignal, ActionResult interfaces | -| `pkg/jobrunner/poller.go` | CREATE -- Poller, Dispatcher, multi-source orchestration | -| `pkg/jobrunner/journal.go` | CREATE -- JSONL writer, append-only, structured records | -| `pkg/jobrunner/github/source.go` | CREATE -- GitHubSource adapter, conditional requests | -| `pkg/jobrunner/github/signals.go` | CREATE -- PR/issue state extraction, signal building | -| `internal/core-ide/handlers/publish_draft.go` | CREATE -- Publish draft PR handler | -| `internal/core-ide/handlers/resolve_threads.go` | CREATE -- Resolve review threads handler | -| `internal/core-ide/handlers/send_fix_command.go` | CREATE -- Send fix command handler | -| `internal/core-ide/handlers/enable_auto_merge.go` | CREATE -- Enable auto-merge handler | -| `internal/core-ide/handlers/tick_parent.go` | CREATE -- Tick epic checklist handler | -| `internal/core-ide/handlers/close_child.go` | CREATE -- Close child issue handler | -| `internal/core-ide/main.go` | MODIFY -- Headless/desktop branching, poller integration | -| `internal/core-ide/mcp_bridge.go` | MODIFY -- Register job handlers as MCP tools | - ---- - -## What Doesn't Ship Yet - -- HostUK Agentic API adapter (future -- replaces GitHub) -- Hyperswarm P2P adapter (future) -- External project scanning / harvest mode (future -- WailsApp first) -- LoRA training pipeline (separate concern -- reads JSONL journal) - ---- - -## Testing Strategy - -- **Handlers:** Unit-testable. Mock PipelineSignal in, assert API calls out. -- **Poller:** httptest server returning fixture responses. -- **Journal:** Read back JSONL, verify schema. -- **Integration:** Dry-run mode against real repos, verify signals match expected state. diff --git a/docs/plans/completed/2026-02-05-mcp-integration-original.md b/docs/plans/completed/2026-02-05-mcp-integration-original.md deleted file mode 100644 index 9b3a109..0000000 --- a/docs/plans/completed/2026-02-05-mcp-integration-original.md +++ /dev/null @@ -1,851 +0,0 @@ -# MCP Integration Implementation Plan - -> **Status:** Completed. MCP command now lives in `go-ai/cmd/mcpcmd/`. Code examples below use the old `init()` + `RegisterCommands()` pattern — the current approach uses `cli.WithCommands()` (see cli-meta-package-design.md). - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add `core mcp serve` command with RAG and metrics tools, then configure the agentic-flows plugin to use it. - -**Architecture:** Create a new `mcp` command package that starts the pkg/mcp server with extended tools. RAG tools call the existing exported functions in internal/cmd/rag. Metrics tools call pkg/ai directly. The agentic-flows plugin gets a `.mcp.json` that spawns `core mcp serve`. - -**Tech Stack:** Go 1.25, github.com/modelcontextprotocol/go-sdk/mcp, pkg/rag, pkg/ai - ---- - -## Task 1: Add RAG tools to pkg/mcp - -**Files:** -- Create: `pkg/mcp/tools_rag.go` -- Modify: `pkg/mcp/mcp.go:99-101` (registerTools) -- Test: `pkg/mcp/tools_rag_test.go` - -**Step 1: Write the failing test** - -Create `pkg/mcp/tools_rag_test.go`: - -```go -package mcp - -import ( - "context" - "testing" - - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -func TestRAGQueryTool_Good(t *testing.T) { - // This test verifies the tool is registered and callable. - // It doesn't require Qdrant/Ollama running - just checks structure. - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - // Check that rag_query tool is registered - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "rag_query" { - found = true - break - } - } - if !found { - t.Error("rag_query tool not registered") - } -} - -func TestRAGQueryInput_Good(t *testing.T) { - input := RAGQueryInput{ - Question: "how do I deploy?", - Collection: "hostuk-docs", - TopK: 5, - } - if input.Question == "" { - t.Error("Question should not be empty") - } -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -run TestRAGQueryTool ./pkg/mcp/... -v` -Expected: FAIL with "rag_query tool not registered" - -**Step 3: Create tools_rag.go with types and tool registration** - -Create `pkg/mcp/tools_rag.go`: - -```go -package mcp - -import ( - "context" - "fmt" - - ragcmd "forge.lthn.ai/core/cli/internal/cmd/rag" - "forge.lthn.ai/core/cli/pkg/rag" - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -// RAG tool input/output types - -// RAGQueryInput contains parameters for querying the vector database. -type RAGQueryInput struct { - Question string `json:"question"` - Collection string `json:"collection,omitempty"` - TopK int `json:"top_k,omitempty"` -} - -// RAGQueryOutput contains the query results. -type RAGQueryOutput struct { - Results []RAGResult `json:"results"` - Context string `json:"context"` -} - -// RAGResult represents a single search result. -type RAGResult struct { - Content string `json:"content"` - Score float32 `json:"score"` - Source string `json:"source"` - Metadata map[string]string `json:"metadata,omitempty"` -} - -// RAGIngestInput contains parameters for ingesting documents. -type RAGIngestInput struct { - Path string `json:"path"` - Collection string `json:"collection,omitempty"` - Recreate bool `json:"recreate,omitempty"` -} - -// RAGIngestOutput contains the ingestion results. -type RAGIngestOutput struct { - Success bool `json:"success"` - Path string `json:"path"` - Chunks int `json:"chunks"` - Message string `json:"message,omitempty"` -} - -// RAGCollectionsInput contains parameters for listing collections. -type RAGCollectionsInput struct { - ShowStats bool `json:"show_stats,omitempty"` -} - -// RAGCollectionsOutput contains the list of collections. -type RAGCollectionsOutput struct { - Collections []CollectionInfo `json:"collections"` -} - -// CollectionInfo describes a Qdrant collection. -type CollectionInfo struct { - Name string `json:"name"` - PointsCount uint64 `json:"points_count,omitempty"` - Status string `json:"status,omitempty"` -} - -// registerRAGTools adds RAG tools to the MCP server. -func (s *Service) registerRAGTools(server *mcp.Server) { - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_query", - Description: "Query the vector database for relevant documents using semantic search", - }, s.ragQuery) - - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_ingest", - Description: "Ingest a file or directory into the vector database", - }, s.ragIngest) - - mcp.AddTool(server, &mcp.Tool{ - Name: "rag_collections", - Description: "List available vector database collections", - }, s.ragCollections) -} - -func (s *Service) ragQuery(ctx context.Context, req *mcp.CallToolRequest, input RAGQueryInput) (*mcp.CallToolResult, RAGQueryOutput, error) { - s.logger.Info("MCP tool execution", "tool", "rag_query", "question", input.Question) - - collection := input.Collection - if collection == "" { - collection = "hostuk-docs" - } - topK := input.TopK - if topK <= 0 { - topK = 5 - } - - results, err := ragcmd.QueryDocs(ctx, input.Question, collection, topK) - if err != nil { - return nil, RAGQueryOutput{}, fmt.Errorf("query failed: %w", err) - } - - // Convert to output format - out := RAGQueryOutput{ - Results: make([]RAGResult, 0, len(results)), - Context: rag.FormatResultsContext(results), - } - for _, r := range results { - out.Results = append(out.Results, RAGResult{ - Content: r.Content, - Score: r.Score, - Source: r.Source, - Metadata: r.Metadata, - }) - } - - return nil, out, nil -} - -func (s *Service) ragIngest(ctx context.Context, req *mcp.CallToolRequest, input RAGIngestInput) (*mcp.CallToolResult, RAGIngestOutput, error) { - s.logger.Security("MCP tool execution", "tool", "rag_ingest", "path", input.Path) - - collection := input.Collection - if collection == "" { - collection = "hostuk-docs" - } - - // Check if path is a file or directory - info, err := s.medium.Stat(input.Path) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("path not found: %w", err) - } - - if info.IsDir() { - err = ragcmd.IngestDirectory(ctx, input.Path, collection, input.Recreate) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("ingest directory failed: %w", err) - } - return nil, RAGIngestOutput{ - Success: true, - Path: input.Path, - Message: fmt.Sprintf("Ingested directory into collection %s", collection), - }, nil - } - - chunks, err := ragcmd.IngestFile(ctx, input.Path, collection) - if err != nil { - return nil, RAGIngestOutput{}, fmt.Errorf("ingest file failed: %w", err) - } - - return nil, RAGIngestOutput{ - Success: true, - Path: input.Path, - Chunks: chunks, - Message: fmt.Sprintf("Ingested %d chunks into collection %s", chunks, collection), - }, nil -} - -func (s *Service) ragCollections(ctx context.Context, req *mcp.CallToolRequest, input RAGCollectionsInput) (*mcp.CallToolResult, RAGCollectionsOutput, error) { - s.logger.Info("MCP tool execution", "tool", "rag_collections") - - client, err := rag.NewQdrantClient(rag.DefaultQdrantConfig()) - if err != nil { - return nil, RAGCollectionsOutput{}, fmt.Errorf("connect to Qdrant: %w", err) - } - defer func() { _ = client.Close() }() - - names, err := client.ListCollections(ctx) - if err != nil { - return nil, RAGCollectionsOutput{}, fmt.Errorf("list collections: %w", err) - } - - out := RAGCollectionsOutput{ - Collections: make([]CollectionInfo, 0, len(names)), - } - - for _, name := range names { - info := CollectionInfo{Name: name} - if input.ShowStats { - cinfo, err := client.CollectionInfo(ctx, name) - if err == nil { - info.PointsCount = cinfo.PointsCount - info.Status = cinfo.Status.String() - } - } - out.Collections = append(out.Collections, info) - } - - return nil, out, nil -} -``` - -**Step 4: Update mcp.go to call registerRAGTools** - -In `pkg/mcp/mcp.go`, modify the `registerTools` function (around line 104) to add: - -```go -func (s *Service) registerTools(server *mcp.Server) { - // File operations (existing) - // ... existing code ... - - // RAG operations - s.registerRAGTools(server) -} -``` - -**Step 5: Run test to verify it passes** - -Run: `go test -run TestRAGQuery ./pkg/mcp/... -v` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/mcp/tools_rag.go pkg/mcp/tools_rag_test.go pkg/mcp/mcp.go -git commit -m "feat(mcp): add RAG tools (query, ingest, collections)" -``` - ---- - -## Task 2: Add metrics tools to pkg/mcp - -**Files:** -- Create: `pkg/mcp/tools_metrics.go` -- Modify: `pkg/mcp/mcp.go` (registerTools) -- Test: `pkg/mcp/tools_metrics_test.go` - -**Step 1: Write the failing test** - -Create `pkg/mcp/tools_metrics_test.go`: - -```go -package mcp - -import ( - "testing" -) - -func TestMetricsRecordTool_Good(t *testing.T) { - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "metrics_record" { - found = true - break - } - } - if !found { - t.Error("metrics_record tool not registered") - } -} - -func TestMetricsQueryTool_Good(t *testing.T) { - s, err := New(WithWorkspaceRoot("")) - if err != nil { - t.Fatalf("New() error: %v", err) - } - - tools := s.Server().ListTools() - found := false - for _, tool := range tools { - if tool.Name == "metrics_query" { - found = true - break - } - } - if !found { - t.Error("metrics_query tool not registered") - } -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -run TestMetrics ./pkg/mcp/... -v` -Expected: FAIL - -**Step 3: Create tools_metrics.go** - -Create `pkg/mcp/tools_metrics.go`: - -```go -package mcp - -import ( - "context" - "fmt" - "time" - - "forge.lthn.ai/core/cli/pkg/ai" - "github.com/modelcontextprotocol/go-sdk/mcp" -) - -// Metrics tool input/output types - -// MetricsRecordInput contains parameters for recording a metric event. -type MetricsRecordInput struct { - Type string `json:"type"` - AgentID string `json:"agent_id,omitempty"` - Repo string `json:"repo,omitempty"` - Data map[string]any `json:"data,omitempty"` -} - -// MetricsRecordOutput contains the result of recording. -type MetricsRecordOutput struct { - Success bool `json:"success"` - Timestamp time.Time `json:"timestamp"` -} - -// MetricsQueryInput contains parameters for querying metrics. -type MetricsQueryInput struct { - Since string `json:"since,omitempty"` // e.g., "7d", "24h" -} - -// MetricsQueryOutput contains the query results. -type MetricsQueryOutput struct { - Total int `json:"total"` - ByType []MetricCount `json:"by_type"` - ByRepo []MetricCount `json:"by_repo"` - ByAgent []MetricCount `json:"by_agent"` - Events []MetricEventBrief `json:"events,omitempty"` -} - -// MetricCount represents a count by key. -type MetricCount struct { - Key string `json:"key"` - Count int `json:"count"` -} - -// MetricEventBrief is a simplified event for output. -type MetricEventBrief struct { - Type string `json:"type"` - Timestamp time.Time `json:"timestamp"` - AgentID string `json:"agent_id,omitempty"` - Repo string `json:"repo,omitempty"` -} - -// registerMetricsTools adds metrics tools to the MCP server. -func (s *Service) registerMetricsTools(server *mcp.Server) { - mcp.AddTool(server, &mcp.Tool{ - Name: "metrics_record", - Description: "Record a metric event (AI task, security scan, job creation, etc.)", - }, s.metricsRecord) - - mcp.AddTool(server, &mcp.Tool{ - Name: "metrics_query", - Description: "Query recorded metrics with aggregation by type, repo, and agent", - }, s.metricsQuery) -} - -func (s *Service) metricsRecord(ctx context.Context, req *mcp.CallToolRequest, input MetricsRecordInput) (*mcp.CallToolResult, MetricsRecordOutput, error) { - s.logger.Info("MCP tool execution", "tool", "metrics_record", "type", input.Type) - - if input.Type == "" { - return nil, MetricsRecordOutput{}, fmt.Errorf("type is required") - } - - event := ai.Event{ - Type: input.Type, - Timestamp: time.Now(), - AgentID: input.AgentID, - Repo: input.Repo, - Data: input.Data, - } - - if err := ai.Record(event); err != nil { - return nil, MetricsRecordOutput{}, fmt.Errorf("record event: %w", err) - } - - return nil, MetricsRecordOutput{ - Success: true, - Timestamp: event.Timestamp, - }, nil -} - -func (s *Service) metricsQuery(ctx context.Context, req *mcp.CallToolRequest, input MetricsQueryInput) (*mcp.CallToolResult, MetricsQueryOutput, error) { - s.logger.Info("MCP tool execution", "tool", "metrics_query", "since", input.Since) - - since := input.Since - if since == "" { - since = "7d" - } - - duration, err := parseDuration(since) - if err != nil { - return nil, MetricsQueryOutput{}, fmt.Errorf("invalid since value: %w", err) - } - - sinceTime := time.Now().Add(-duration) - events, err := ai.ReadEvents(sinceTime) - if err != nil { - return nil, MetricsQueryOutput{}, fmt.Errorf("read events: %w", err) - } - - summary := ai.Summary(events) - - out := MetricsQueryOutput{ - Total: summary["total"].(int), - } - - // Convert by_type - if byType, ok := summary["by_type"].([]map[string]any); ok { - for _, entry := range byType { - out.ByType = append(out.ByType, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Convert by_repo - if byRepo, ok := summary["by_repo"].([]map[string]any); ok { - for _, entry := range byRepo { - out.ByRepo = append(out.ByRepo, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Convert by_agent - if byAgent, ok := summary["by_agent"].([]map[string]any); ok { - for _, entry := range byAgent { - out.ByAgent = append(out.ByAgent, MetricCount{ - Key: entry["key"].(string), - Count: entry["count"].(int), - }) - } - } - - // Include last 10 events for context - limit := 10 - if len(events) < limit { - limit = len(events) - } - for i := len(events) - limit; i < len(events); i++ { - ev := events[i] - out.Events = append(out.Events, MetricEventBrief{ - Type: ev.Type, - Timestamp: ev.Timestamp, - AgentID: ev.AgentID, - Repo: ev.Repo, - }) - } - - return nil, out, nil -} - -// parseDuration parses a human-friendly duration like "7d", "24h", "30d". -func parseDuration(s string) (time.Duration, error) { - if len(s) < 2 { - return 0, fmt.Errorf("invalid duration: %s", s) - } - - unit := s[len(s)-1] - value := s[:len(s)-1] - - var n int - if _, err := fmt.Sscanf(value, "%d", &n); err != nil { - return 0, fmt.Errorf("invalid duration: %s", s) - } - - if n <= 0 { - return 0, fmt.Errorf("duration must be positive: %s", s) - } - - switch unit { - case 'd': - return time.Duration(n) * 24 * time.Hour, nil - case 'h': - return time.Duration(n) * time.Hour, nil - case 'm': - return time.Duration(n) * time.Minute, nil - default: - return 0, fmt.Errorf("unknown unit %c in duration: %s", unit, s) - } -} -``` - -**Step 4: Update mcp.go to call registerMetricsTools** - -In `pkg/mcp/mcp.go`, add to `registerTools`: - -```go -func (s *Service) registerTools(server *mcp.Server) { - // ... existing file operations ... - - // RAG operations - s.registerRAGTools(server) - - // Metrics operations - s.registerMetricsTools(server) -} -``` - -**Step 5: Run test to verify it passes** - -Run: `go test -run TestMetrics ./pkg/mcp/... -v` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/mcp/tools_metrics.go pkg/mcp/tools_metrics_test.go pkg/mcp/mcp.go -git commit -m "feat(mcp): add metrics tools (record, query)" -``` - ---- - -## Task 3: Create `core mcp serve` command - -**Files:** -- Create: `internal/cmd/mcpcmd/cmd_mcp.go` -- Modify: `internal/variants/full.go` (add import) -- Test: Manual test via `core mcp serve` - -**Step 1: Create the mcp command package** - -Create `internal/cmd/mcpcmd/cmd_mcp.go`: - -```go -package mcpcmd - -import ( - "context" - "os" - "os/signal" - "syscall" - - "forge.lthn.ai/core/cli/pkg/cli" - "forge.lthn.ai/core/cli/pkg/i18n" - "forge.lthn.ai/core/cli/pkg/mcp" -) - -func init() { - cli.RegisterCommands(AddMCPCommands) -} - -var ( - mcpWorkspace string -) - -var mcpCmd = &cli.Command{ - Use: "mcp", - Short: i18n.T("cmd.mcp.short"), - Long: i18n.T("cmd.mcp.long"), -} - -var serveCmd = &cli.Command{ - Use: "serve", - Short: i18n.T("cmd.mcp.serve.short"), - Long: i18n.T("cmd.mcp.serve.long"), - RunE: func(cmd *cli.Command, args []string) error { - return runServe() - }, -} - -func AddMCPCommands(root *cli.Command) { - initMCPFlags() - mcpCmd.AddCommand(serveCmd) - root.AddCommand(mcpCmd) -} - -func initMCPFlags() { - serveCmd.Flags().StringVar(&mcpWorkspace, "workspace", "", i18n.T("cmd.mcp.serve.flag.workspace")) -} - -func runServe() error { - opts := []mcp.Option{} - - if mcpWorkspace != "" { - opts = append(opts, mcp.WithWorkspaceRoot(mcpWorkspace)) - } else { - // Default to unrestricted for MCP server - opts = append(opts, mcp.WithWorkspaceRoot("")) - } - - svc, err := mcp.New(opts...) - if err != nil { - return cli.Wrap(err, "create MCP service") - } - - ctx, cancel := context.WithCancel(context.Background()) - defer cancel() - - // Handle shutdown signals - sigCh := make(chan os.Signal, 1) - signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM) - go func() { - <-sigCh - cancel() - }() - - return svc.Run(ctx) -} -``` - -**Step 2: Add i18n strings** - -Create or update `pkg/i18n/en.yaml` (if it exists) or add to the existing i18n mechanism: - -```yaml -cmd.mcp.short: "MCP (Model Context Protocol) server" -cmd.mcp.long: "Start an MCP server for Claude Code integration with file, RAG, and metrics tools." -cmd.mcp.serve.short: "Start the MCP server" -cmd.mcp.serve.long: "Start the MCP server in stdio mode. Use MCP_ADDR env var for TCP mode." -cmd.mcp.serve.flag.workspace: "Restrict file operations to this directory (empty = unrestricted)" -``` - -**Step 3: Add import to full.go** - -Modify `internal/variants/full.go` to add: - -```go -import ( - // ... existing imports ... - _ "forge.lthn.ai/core/cli/internal/cmd/mcpcmd" -) -``` - -**Step 4: Build and test** - -Run: `go build && ./core mcp serve --help` -Expected: Help output showing the serve command - -**Step 5: Test MCP server manually** - -Run: `echo '{"jsonrpc":"2.0","method":"tools/list","id":1}' | ./core mcp serve` -Expected: JSON response listing all tools including rag_query, metrics_record, etc. - -**Step 6: Commit** - -```bash -git add internal/cmd/mcpcmd/cmd_mcp.go internal/variants/full.go -git commit -m "feat: add 'core mcp serve' command" -``` - ---- - -## Task 4: Configure agentic-flows plugin with .mcp.json - -**Files:** -- Create: `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.mcp.json` -- Modify: `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.claude-plugin/plugin.json` (optional, add mcpServers) - -**Step 1: Create .mcp.json** - -Create `/home/shared/hostuk/claude-plugins/plugins/agentic-flows/.mcp.json`: - -```json -{ - "core-cli": { - "command": "core", - "args": ["mcp", "serve"], - "env": { - "MCP_WORKSPACE": "" - } - } -} -``` - -**Step 2: Verify plugin loads** - -Restart Claude Code and run `/mcp` to verify the core-cli server appears. - -**Step 3: Test MCP tools** - -Test that tools are available: -- `mcp__plugin_agentic-flows_core-cli__rag_query` -- `mcp__plugin_agentic-flows_core-cli__rag_ingest` -- `mcp__plugin_agentic-flows_core-cli__rag_collections` -- `mcp__plugin_agentic-flows_core-cli__metrics_record` -- `mcp__plugin_agentic-flows_core-cli__metrics_query` -- `mcp__plugin_agentic-flows_core-cli__file_read` -- etc. - -**Step 4: Commit plugin changes** - -```bash -cd /home/shared/hostuk/claude-plugins -git add plugins/agentic-flows/.mcp.json -git commit -m "feat(agentic-flows): add MCP server configuration for core-cli" -``` - ---- - -## Task 5: Update documentation - -**Files:** -- Modify: `/home/claude/.claude/projects/-home-claude/memory/MEMORY.md` -- Modify: `/home/claude/.claude/projects/-home-claude/memory/plugin-dev-notes.md` - -**Step 1: Update MEMORY.md** - -Add under "Core CLI MCP Server" section: - -```markdown -### Core CLI MCP Server -- **Command:** `core mcp serve` (stdio mode) or `MCP_ADDR=:9000 core mcp serve` (TCP) -- **Tools available:** - - File ops: file_read, file_write, file_edit, file_delete, file_rename, file_exists, dir_list, dir_create - - RAG: rag_query, rag_ingest, rag_collections - - Metrics: metrics_record, metrics_query - - Language: lang_detect, lang_list -- **Plugin config:** `plugins/agentic-flows/.mcp.json` -``` - -**Step 2: Update plugin-dev-notes.md** - -Add section: - -```markdown -## MCP Server (core mcp serve) - -### Available Tools -| Tool | Description | -|------|-------------| -| file_read | Read file contents | -| file_write | Write file contents | -| file_edit | Edit file (replace string) | -| file_delete | Delete file | -| file_rename | Rename/move file | -| file_exists | Check if file exists | -| dir_list | List directory contents | -| dir_create | Create directory | -| rag_query | Query vector DB | -| rag_ingest | Ingest file/directory | -| rag_collections | List collections | -| metrics_record | Record event | -| metrics_query | Query events | -| lang_detect | Detect file language | -| lang_list | List supported languages | - -### Example .mcp.json -```json -{ - "core-cli": { - "command": "core", - "args": ["mcp", "serve"] - } -} -``` -``` - -**Step 3: Commit documentation** - -```bash -git add ~/.claude/projects/-home-claude/memory/*.md -git commit -m "docs: update memory with MCP server tools" -``` - ---- - -## Summary - -| Task | Files | Purpose | -|------|-------|---------| -| 1 | `pkg/mcp/tools_rag.go` | RAG tools (query, ingest, collections) | -| 2 | `pkg/mcp/tools_metrics.go` | Metrics tools (record, query) | -| 3 | `internal/cmd/mcpcmd/cmd_mcp.go` | `core mcp serve` command | -| 4 | `plugins/agentic-flows/.mcp.json` | Plugin MCP configuration | -| 5 | Memory docs | Documentation updates | - -## Services Required - -- **Qdrant:** localhost:6333 (verified running) -- **Ollama:** localhost:11434 with nomic-embed-text (verified running) -- **InfluxDB:** localhost:8086 (optional, for future time-series metrics) diff --git a/docs/plans/completed/2026-02-13-bugseti-hub-service-design-original.md b/docs/plans/completed/2026-02-13-bugseti-hub-service-design-original.md deleted file mode 100644 index 2f132e4..0000000 --- a/docs/plans/completed/2026-02-13-bugseti-hub-service-design-original.md +++ /dev/null @@ -1,150 +0,0 @@ -# BugSETI HubService Design - -## Overview - -A thin HTTP client service in the BugSETI desktop app that coordinates with the agentic portal's `/api/bugseti/*` endpoints. Prevents duplicate work across the 11 community testers, aggregates stats for leaderboard, and registers client instances. - -## Decisions - -| Decision | Choice | Rationale | -|----------|--------|-----------| -| Target | Direct to portal API | Endpoints built for this purpose | -| Auth | Auto-register via forge token | No manual key management for users | -| Sync strategy | Lazy/manual | User-triggered claims, manual stats sync | -| Offline mode | Offline-first | Queue failed writes, retry on reconnect | -| Approach | Thin HTTP client (net/http) | Matches existing patterns, no deps | - -## Architecture - -**File:** `internal/bugseti/hub.go` + `hub_test.go` - -``` -HubService -├── HTTP client (net/http, 10s timeout) -├── Auth: auto-register via forge token → cached ak_ token -├── Config: HubURL, HubToken, ClientID in ConfigService -├── Offline-first: queue failed writes, drain on next success -└── Lazy sync: user-triggered, no background goroutines -``` - -**Dependencies:** ConfigService only. - -**Integration:** -- QueueService calls `hub.ClaimIssue()` when user picks an issue -- SubmitService calls `hub.UpdateStatus("completed")` after PR -- TrayService calls `hub.GetLeaderboard()` from UI -- main.go calls `hub.Register()` on startup - -## Data Types - -```go -type HubClient struct { - ClientID string // UUID, generated once, persisted in config - Name string // e.g. "Snider's MacBook" - Version string // bugseti.GetVersion() - OS string // runtime.GOOS - Arch string // runtime.GOARCH -} - -type HubClaim struct { - IssueID string // "owner/repo#123" - Repo string - IssueNumber int - Title string - URL string - Status string // claimed|in_progress|completed|skipped - ClaimedAt time.Time - PRUrl string - PRNumber int -} - -type LeaderboardEntry struct { - Rank int - ClientName string - IssuesCompleted int - PRsSubmitted int - PRsMerged int - CurrentStreak int -} - -type GlobalStats struct { - TotalParticipants int - ActiveParticipants int - TotalIssuesCompleted int - TotalPRsMerged int - ActiveClaims int -} -``` - -## API Mapping - -| Method | HTTP | Endpoint | Trigger | -|--------|------|----------|---------| -| `Register()` | POST /register | App startup | -| `Heartbeat()` | POST /heartbeat | Manual / periodic if enabled | -| `ClaimIssue(issue)` | POST /issues/claim | User picks issue | -| `UpdateStatus(id, status)` | PATCH /issues/{id}/status | PR submitted, skip | -| `ReleaseClaim(id)` | DELETE /issues/{id}/claim | User abandons | -| `IsIssueClaimed(id)` | GET /issues/{id} | Before showing issue | -| `ListClaims(filters)` | GET /issues/claimed | UI active claims view | -| `SyncStats(stats)` | POST /stats/sync | Manual from UI | -| `GetLeaderboard(limit)` | GET /leaderboard | UI leaderboard view | -| `GetGlobalStats()` | GET /stats | UI stats dashboard | - -## Auto-Register Flow - -New endpoint on portal: - -``` -POST /api/bugseti/auth/forge -Body: { "forge_url": "https://forge.lthn.io", "forge_token": "..." } -``` - -Portal validates token against Forgejo API (`/api/v1/user`), creates an AgentApiKey with `bugseti.read` + `bugseti.write` scopes, returns `{ "api_key": "ak_..." }`. - -HubService caches the `ak_` token in config.json. On 401, clears cached token and re-registers. - -## Error Handling - -| Error | Behaviour | -|-------|-----------| -| Network unreachable | Log, queue write ops, return cached reads | -| 401 Unauthorised | Clear token, re-register via forge | -| 409 Conflict (claim) | Return "already claimed" — not an error | -| 404 (claim not found) | Return nil | -| 429 Rate limited | Back off, queue the op | -| 5xx Server error | Log, queue write ops | - -**Pending operations queue:** -- Failed writes stored in `[]PendingOp`, persisted to `$DataDir/hub_pending.json` -- Drained on next successful user-triggered call (no background goroutine) -- Each op has: method, path, body, created_at - -## Config Changes - -New fields in `Config` struct: - -```go -HubURL string `json:"hubUrl,omitempty"` // portal API base URL -HubToken string `json:"hubToken,omitempty"` // cached ak_ token -ClientID string `json:"clientId,omitempty"` // UUID, generated once -ClientName string `json:"clientName,omitempty"` // display name -``` - -## Files Changed - -| File | Action | -|------|--------| -| `internal/bugseti/hub.go` | New — HubService | -| `internal/bugseti/hub_test.go` | New — httptest-based tests | -| `internal/bugseti/config.go` | Edit — add Hub* + ClientID fields | -| `cmd/bugseti/main.go` | Edit — create + register HubService | -| `cmd/bugseti/tray.go` | Edit — leaderboard/stats menu items | -| Laravel: auth controller | New — `/api/bugseti/auth/forge` | - -## Testing - -- `httptest.NewServer` mocks for all endpoints -- Test success, network error, 409 conflict, 401 re-auth flows -- Test pending ops queue: add when offline, drain on reconnect -- `_Good`, `_Bad`, `_Ugly` naming convention diff --git a/docs/plans/completed/2026-02-13-bugseti-hub-service-plan-original.md b/docs/plans/completed/2026-02-13-bugseti-hub-service-plan-original.md deleted file mode 100644 index 2b9e3bb..0000000 --- a/docs/plans/completed/2026-02-13-bugseti-hub-service-plan-original.md +++ /dev/null @@ -1,1620 +0,0 @@ -# BugSETI HubService Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add a HubService to BugSETI that coordinates issue claiming, stats sync, and leaderboard with the agentic portal API. - -**Architecture:** Thin HTTP client (`net/http`) in `internal/bugseti/hub.go` talking directly to the portal's `/api/bugseti/*` endpoints. Auto-registers via forge token to get an `ak_` bearer token. Offline-first with pending-ops queue. - -**Tech Stack:** Go stdlib (`net/http`, `encoding/json`), Laravel 12 (portal endpoint), httptest (Go tests) - ---- - -### Task 1: Config Fields - -Add hub-related fields to the Config struct so HubService can persist its state. - -**Files:** -- Modify: `internal/bugseti/config.go` -- Test: `internal/bugseti/fetcher_test.go` (uses `testConfigService`) - -**Step 1: Add config fields** - -In `internal/bugseti/config.go`, add these fields to the `Config` struct after the `ForgeToken` field: - -```go -// Hub coordination (agentic portal) -HubURL string `json:"hubUrl,omitempty"` // Portal API base URL (e.g. https://leth.in) -HubToken string `json:"hubToken,omitempty"` // Cached ak_ bearer token -ClientID string `json:"clientId,omitempty"` // UUID, generated once on first run -ClientName string `json:"clientName,omitempty"` // Display name for leaderboard -``` - -**Step 2: Add getters/setters** - -After the `GetForgeToken()` method, add: - -```go -// GetHubURL returns the hub portal URL. -func (c *ConfigService) GetHubURL() string { - c.mu.RLock() - defer c.mu.RUnlock() - return c.config.HubURL -} - -// SetHubURL sets the hub portal URL. -func (c *ConfigService) SetHubURL(url string) error { - c.mu.Lock() - defer c.mu.Unlock() - c.config.HubURL = url - return c.saveUnsafe() -} - -// GetHubToken returns the cached hub API token. -func (c *ConfigService) GetHubToken() string { - c.mu.RLock() - defer c.mu.RUnlock() - return c.config.HubToken -} - -// SetHubToken caches the hub API token. -func (c *ConfigService) SetHubToken(token string) error { - c.mu.Lock() - defer c.mu.Unlock() - c.config.HubToken = token - return c.saveUnsafe() -} - -// GetClientID returns the persistent client UUID. -func (c *ConfigService) GetClientID() string { - c.mu.RLock() - defer c.mu.RUnlock() - return c.config.ClientID -} - -// SetClientID sets the persistent client UUID. -func (c *ConfigService) SetClientID(id string) error { - c.mu.Lock() - defer c.mu.Unlock() - c.config.ClientID = id - return c.saveUnsafe() -} - -// GetClientName returns the display name. -func (c *ConfigService) GetClientName() string { - c.mu.RLock() - defer c.mu.RUnlock() - return c.config.ClientName -} - -// SetClientName sets the display name. -func (c *ConfigService) SetClientName(name string) error { - c.mu.Lock() - defer c.mu.Unlock() - c.config.ClientName = name - return c.saveUnsafe() -} -``` - -**Step 3: Run tests** - -Run: `go test ./internal/bugseti/... -count=1` -Expected: All existing tests pass (config fields are additive, no breakage). - -**Step 4: Commit** - -```bash -git add internal/bugseti/config.go -git commit -m "feat(bugseti): add hub config fields (HubURL, HubToken, ClientID, ClientName)" -``` - ---- - -### Task 2: HubService Core — Types and Constructor - -Create the HubService with data types, constructor, and ServiceName. - -**Files:** -- Create: `internal/bugseti/hub.go` -- Create: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Create `internal/bugseti/hub_test.go`: - -```go -package bugseti - -import ( - "testing" -) - -func testHubService(t *testing.T, serverURL string) *HubService { - t.Helper() - cfg := testConfigService(t, nil, nil) - if serverURL != "" { - cfg.config.HubURL = serverURL - } - return NewHubService(cfg) -} - -// --- Constructor / ServiceName --- - -func TestNewHubService_Good(t *testing.T) { - h := testHubService(t, "") - if h == nil { - t.Fatal("expected non-nil HubService") - } - if h.config == nil { - t.Fatal("expected config to be set") - } -} - -func TestHubServiceName_Good(t *testing.T) { - h := testHubService(t, "") - if got := h.ServiceName(); got != "HubService" { - t.Fatalf("expected HubService, got %s", got) - } -} - -func TestNewHubService_Good_GeneratesClientID(t *testing.T) { - h := testHubService(t, "") - id := h.GetClientID() - if id == "" { - t.Fatal("expected client ID to be generated") - } - if len(id) < 32 { - t.Fatalf("expected UUID-length client ID, got %d chars", len(id)) - } -} - -func TestNewHubService_Good_ReusesClientID(t *testing.T) { - cfg := testConfigService(t, nil, nil) - cfg.config.ClientID = "existing-id-12345" - h := NewHubService(cfg) - if h.GetClientID() != "existing-id-12345" { - t.Fatal("expected existing client ID to be preserved") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `go test ./internal/bugseti/... -run TestNewHubService -count=1` -Expected: FAIL — `NewHubService` not defined. - -**Step 3: Write HubService core** - -Create `internal/bugseti/hub.go`: - -```go -// Package bugseti provides services for the BugSETI distributed bug fixing application. -package bugseti - -import ( - "bytes" - "crypto/rand" - "encoding/hex" - "encoding/json" - "fmt" - "io" - "log" - "net/http" - "net/url" - "runtime" - "sync" - "time" -) - -// HubService coordinates with the agentic portal for issue claiming, -// stats sync, and leaderboard. -type HubService struct { - config *ConfigService - httpClient *http.Client - mu sync.Mutex - connected bool - pendingOps []PendingOp -} - -// PendingOp represents a failed write operation queued for retry. -type PendingOp struct { - Method string `json:"method"` - Path string `json:"path"` - Body []byte `json:"body"` - CreatedAt time.Time `json:"createdAt"` -} - -// HubClaim represents an issue claim from the portal. -type HubClaim struct { - IssueID string `json:"issue_id"` - Repo string `json:"repo"` - IssueNumber int `json:"issue_number"` - Title string `json:"issue_title"` - URL string `json:"issue_url"` - Status string `json:"status"` - ClaimedAt time.Time `json:"claimed_at"` - PRUrl string `json:"pr_url,omitempty"` - PRNumber int `json:"pr_number,omitempty"` -} - -// LeaderboardEntry represents a single entry on the leaderboard. -type LeaderboardEntry struct { - Rank int `json:"rank"` - ClientName string `json:"client_name"` - ClientVersion string `json:"client_version,omitempty"` - IssuesCompleted int `json:"issues_completed"` - PRsSubmitted int `json:"prs_submitted"` - PRsMerged int `json:"prs_merged"` - CurrentStreak int `json:"current_streak"` - LongestStreak int `json:"longest_streak"` -} - -// GlobalStats represents aggregate stats from the portal. -type GlobalStats struct { - TotalParticipants int `json:"total_participants"` - ActiveParticipants int `json:"active_participants"` - TotalIssuesAttempted int `json:"total_issues_attempted"` - TotalIssuesCompleted int `json:"total_issues_completed"` - TotalPRsSubmitted int `json:"total_prs_submitted"` - TotalPRsMerged int `json:"total_prs_merged"` - ActiveClaims int `json:"active_claims"` - CompletedClaims int `json:"completed_claims"` -} - -// NewHubService creates a new HubService. -func NewHubService(config *ConfigService) *HubService { - h := &HubService{ - config: config, - httpClient: &http.Client{ - Timeout: 10 * time.Second, - }, - } - - // Ensure a persistent client ID exists - if config.GetClientID() == "" { - id := generateClientID() - if err := config.SetClientID(id); err != nil { - log.Printf("Warning: could not persist client ID: %v", err) - } - } - - // Load pending ops from disk - h.loadPendingOps() - - return h -} - -// ServiceName returns the service name for Wails. -func (h *HubService) ServiceName() string { - return "HubService" -} - -// GetClientID returns the persistent client identifier. -func (h *HubService) GetClientID() string { - return h.config.GetClientID() -} - -// IsConnected returns whether the last hub request succeeded. -func (h *HubService) IsConnected() bool { - h.mu.Lock() - defer h.mu.Unlock() - return h.connected -} - -// generateClientID creates a random hex client identifier. -func generateClientID() string { - b := make([]byte, 16) - if _, err := rand.Read(b); err != nil { - // Fallback to timestamp-based ID - return fmt.Sprintf("bugseti-%d", time.Now().UnixNano()) - } - return hex.EncodeToString(b) -} -``` - -**Step 4: Run tests** - -Run: `go test ./internal/bugseti/... -run TestNewHubService -count=1 && go test ./internal/bugseti/... -run TestHubServiceName -count=1` -Expected: PASS - -**Step 5: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): add HubService types and constructor" -``` - ---- - -### Task 3: HTTP Request Helpers - -Add the internal `doRequest` and `doJSON` methods that all API calls use. - -**Files:** -- Modify: `internal/bugseti/hub.go` -- Modify: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Add to `hub_test.go`: - -```go -import ( - "encoding/json" - "net/http" - "net/http/httptest" - "testing" -) - -func TestDoRequest_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Header.Get("Authorization") != "Bearer test-token" { - t.Fatal("expected bearer token") - } - if r.Header.Get("Content-Type") != "application/json" { - t.Fatal("expected JSON content type") - } - w.WriteHeader(200) - w.Write([]byte(`{"ok":true}`)) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "test-token" - - resp, err := h.doRequest("GET", "/test", nil) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - defer resp.Body.Close() - if resp.StatusCode != 200 { - t.Fatalf("expected 200, got %d", resp.StatusCode) - } -} - -func TestDoRequest_Bad_NoHubURL(t *testing.T) { - h := testHubService(t, "") - _, err := h.doRequest("GET", "/test", nil) - if err == nil { - t.Fatal("expected error when hub URL is empty") - } -} - -func TestDoRequest_Bad_NetworkError(t *testing.T) { - h := testHubService(t, "http://127.0.0.1:1") // Nothing listening - h.config.config.HubToken = "test-token" - - _, err := h.doRequest("GET", "/test", nil) - if err == nil { - t.Fatal("expected network error") - } -} -``` - -**Step 2: Run to verify failure** - -Run: `go test ./internal/bugseti/... -run TestDoRequest -count=1` -Expected: FAIL — `doRequest` not defined. - -**Step 3: Implement helpers** - -Add to `hub.go`: - -```go -// doRequest performs an HTTP request to the hub API. -// Returns the response (caller must close body) or an error. -func (h *HubService) doRequest(method, path string, body interface{}) (*http.Response, error) { - hubURL := h.config.GetHubURL() - if hubURL == "" { - return nil, fmt.Errorf("hub URL not configured") - } - - fullURL := hubURL + "/api/bugseti" + path - - var bodyReader io.Reader - if body != nil { - data, err := json.Marshal(body) - if err != nil { - return nil, fmt.Errorf("failed to marshal request body: %w", err) - } - bodyReader = bytes.NewReader(data) - } - - req, err := http.NewRequest(method, fullURL, bodyReader) - if err != nil { - return nil, fmt.Errorf("failed to create request: %w", err) - } - - req.Header.Set("Content-Type", "application/json") - req.Header.Set("Accept", "application/json") - - if token := h.config.GetHubToken(); token != "" { - req.Header.Set("Authorization", "Bearer "+token) - } - - resp, err := h.httpClient.Do(req) - if err != nil { - h.mu.Lock() - h.connected = false - h.mu.Unlock() - return nil, fmt.Errorf("hub request failed: %w", err) - } - - h.mu.Lock() - h.connected = true - h.mu.Unlock() - - return resp, nil -} - -// doJSON performs a request and decodes the JSON response into dest. -func (h *HubService) doJSON(method, path string, body interface{}, dest interface{}) error { - resp, err := h.doRequest(method, path, body) - if err != nil { - return err - } - defer resp.Body.Close() - - if resp.StatusCode == 401 { - return fmt.Errorf("unauthorised") - } - if resp.StatusCode == 409 { - return &ConflictError{StatusCode: resp.StatusCode} - } - if resp.StatusCode == 404 { - return &NotFoundError{StatusCode: resp.StatusCode} - } - if resp.StatusCode >= 400 { - bodyBytes, _ := io.ReadAll(resp.Body) - return fmt.Errorf("hub error %d: %s", resp.StatusCode, string(bodyBytes)) - } - - if dest != nil { - if err := json.NewDecoder(resp.Body).Decode(dest); err != nil { - return fmt.Errorf("failed to decode response: %w", err) - } - } - - return nil -} - -// ConflictError indicates a 409 response (e.g. issue already claimed). -type ConflictError struct { - StatusCode int -} - -func (e *ConflictError) Error() string { - return fmt.Sprintf("conflict (HTTP %d)", e.StatusCode) -} - -// NotFoundError indicates a 404 response. -type NotFoundError struct { - StatusCode int -} - -func (e *NotFoundError) Error() string { - return fmt.Sprintf("not found (HTTP %d)", e.StatusCode) -} -``` - -**Step 4: Run tests** - -Run: `go test ./internal/bugseti/... -run TestDoRequest -count=1` -Expected: PASS - -**Step 5: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): add hub HTTP request helpers with error types" -``` - ---- - -### Task 4: Auto-Register via Forge Token - -Implement the auth flow: send forge token to portal, receive `ak_` token. - -**Files:** -- Modify: `internal/bugseti/hub.go` -- Modify: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Add to `hub_test.go`: - -```go -func TestAutoRegister_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/bugseti/auth/forge" { - t.Fatalf("unexpected path: %s", r.URL.Path) - } - if r.Method != "POST" { - t.Fatalf("expected POST, got %s", r.Method) - } - - var body map[string]string - json.NewDecoder(r.Body).Decode(&body) - - if body["forge_url"] == "" || body["forge_token"] == "" { - w.WriteHeader(400) - return - } - - w.WriteHeader(201) - json.NewEncoder(w).Encode(map[string]string{ - "api_key": "ak_test123456789012345678901234", - }) - })) - defer server.Close() - - cfg := testConfigService(t, nil, nil) - cfg.config.HubURL = server.URL - cfg.config.ForgeURL = "https://forge.lthn.io" - cfg.config.ForgeToken = "forge-test-token" - h := NewHubService(cfg) - - err := h.AutoRegister() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if cfg.GetHubToken() != "ak_test123456789012345678901234" { - t.Fatalf("expected token to be cached, got %q", cfg.GetHubToken()) - } -} - -func TestAutoRegister_Bad_NoForgeToken(t *testing.T) { - cfg := testConfigService(t, nil, nil) - cfg.config.HubURL = "http://localhost" - h := NewHubService(cfg) - - err := h.AutoRegister() - if err == nil { - t.Fatal("expected error when forge token is missing") - } -} - -func TestAutoRegister_Good_SkipsIfAlreadyRegistered(t *testing.T) { - cfg := testConfigService(t, nil, nil) - cfg.config.HubToken = "ak_existing_token" - h := NewHubService(cfg) - - err := h.AutoRegister() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - // Token should remain unchanged - if cfg.GetHubToken() != "ak_existing_token" { - t.Fatal("existing token should not be overwritten") - } -} -``` - -**Step 2: Run to verify failure** - -Run: `go test ./internal/bugseti/... -run TestAutoRegister -count=1` -Expected: FAIL — `AutoRegister` not defined. - -**Step 3: Implement AutoRegister** - -Add to `hub.go`: - -```go -// AutoRegister exchanges forge credentials for a hub API key. -// Skips if a token is already cached. On 401, clears cached token. -func (h *HubService) AutoRegister() error { - // Skip if already registered - if h.config.GetHubToken() != "" { - return nil - } - - hubURL := h.config.GetHubURL() - if hubURL == "" { - return fmt.Errorf("hub URL not configured") - } - - forgeURL := h.config.GetForgeURL() - forgeToken := h.config.GetForgeToken() - - // Fall back to pkg/forge config resolution - if forgeURL == "" || forgeToken == "" { - resolvedURL, resolvedToken, err := resolveForgeConfig(forgeURL, forgeToken) - if err != nil { - return fmt.Errorf("failed to resolve forge config: %w", err) - } - forgeURL = resolvedURL - forgeToken = resolvedToken - } - - if forgeToken == "" { - return fmt.Errorf("forge token not configured — cannot auto-register with hub") - } - - body := map[string]string{ - "forge_url": forgeURL, - "forge_token": forgeToken, - "client_id": h.GetClientID(), - } - - var result struct { - APIKey string `json:"api_key"` - } - - data, err := json.Marshal(body) - if err != nil { - return fmt.Errorf("failed to marshal register body: %w", err) - } - - resp, err := h.httpClient.Post( - hubURL+"/api/bugseti/auth/forge", - "application/json", - bytes.NewReader(data), - ) - if err != nil { - return fmt.Errorf("hub auto-register failed: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode != 201 && resp.StatusCode != 200 { - bodyBytes, _ := io.ReadAll(resp.Body) - return fmt.Errorf("hub auto-register failed (HTTP %d): %s", resp.StatusCode, string(bodyBytes)) - } - - if err := json.NewDecoder(resp.Body).Decode(&result); err != nil { - return fmt.Errorf("failed to decode register response: %w", err) - } - - if result.APIKey == "" { - return fmt.Errorf("hub returned empty API key") - } - - // Cache the token - if err := h.config.SetHubToken(result.APIKey); err != nil { - return fmt.Errorf("failed to cache hub token: %w", err) - } - - log.Printf("Hub: registered with portal, token cached") - return nil -} - -// resolveForgeConfig gets forge URL/token from pkg/forge config chain. -func resolveForgeConfig(flagURL, flagToken string) (string, string, error) { - // Import forge package for config resolution - // This uses the same resolution chain: config.yaml → env vars → flags - forgeURL, forgeToken, err := forgeResolveConfig(flagURL, flagToken) - if err != nil { - return "", "", err - } - return forgeURL, forgeToken, nil -} -``` - -Note: `resolveForgeConfig` wraps `forge.ResolveConfig` — we'll use the import directly in the real code. For the plan, this shows the intent. - -**Step 4: Run tests** - -Run: `go test ./internal/bugseti/... -run TestAutoRegister -count=1` -Expected: PASS - -**Step 5: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): hub auto-register via forge token" -``` - ---- - -### Task 5: Write Operations — Register, Heartbeat, Claim, Update, Release, SyncStats - -Implement all write API methods. - -**Files:** -- Modify: `internal/bugseti/hub.go` -- Modify: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Add to `hub_test.go`: - -```go -func TestRegister_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/bugseti/register" { - t.Fatalf("unexpected path: %s", r.URL.Path) - } - var body map[string]string - json.NewDecoder(r.Body).Decode(&body) - if body["client_id"] == "" || body["name"] == "" { - w.WriteHeader(400) - return - } - w.WriteHeader(201) - json.NewEncoder(w).Encode(map[string]interface{}{"client": body}) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - h.config.config.ClientName = "Test Mac" - - err := h.Register() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } -} - -func TestHeartbeat_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{"ok": true}) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - err := h.Heartbeat() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } -} - -func TestClaimIssue_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(201) - json.NewEncoder(w).Encode(map[string]interface{}{ - "claim": map[string]interface{}{ - "issue_id": "owner/repo#42", - "status": "claimed", - }, - }) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - claim, err := h.ClaimIssue(&Issue{ - ID: "owner/repo#42", Repo: "owner/repo", Number: 42, - Title: "Fix bug", URL: "https://forge.lthn.io/owner/repo/issues/42", - }) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if claim == nil || claim.IssueID != "owner/repo#42" { - t.Fatal("expected claim with correct issue ID") - } -} - -func TestClaimIssue_Bad_Conflict(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(409) - json.NewEncoder(w).Encode(map[string]interface{}{ - "error": "Issue already claimed", - }) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - _, err := h.ClaimIssue(&Issue{ID: "owner/repo#42", Repo: "owner/repo", Number: 42}) - if err == nil { - t.Fatal("expected conflict error") - } - if _, ok := err.(*ConflictError); !ok { - t.Fatalf("expected ConflictError, got %T", err) - } -} - -func TestUpdateStatus_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != "PATCH" { - t.Fatalf("expected PATCH, got %s", r.Method) - } - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{"claim": map[string]string{"status": "completed"}}) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - err := h.UpdateStatus("owner/repo#42", "completed", "https://forge.lthn.io/pr/1", 1) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } -} - -func TestSyncStats_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{"synced": true}) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - err := h.SyncStats(&Stats{ - IssuesCompleted: 5, - PRsSubmitted: 3, - PRsMerged: 2, - }) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } -} -``` - -**Step 2: Run to verify failure** - -Run: `go test ./internal/bugseti/... -run "TestRegister_Good|TestHeartbeat|TestClaimIssue|TestUpdateStatus|TestSyncStats" -count=1` -Expected: FAIL - -**Step 3: Implement write methods** - -Add to `hub.go`: - -```go -// Register sends client registration to the hub portal. -func (h *HubService) Register() error { - h.drainPendingOps() - - name := h.config.GetClientName() - if name == "" { - name = fmt.Sprintf("BugSETI-%s", h.GetClientID()[:8]) - } - - body := map[string]string{ - "client_id": h.GetClientID(), - "name": name, - "version": GetVersion(), - "os": runtime.GOOS, - "arch": runtime.GOARCH, - } - - return h.doJSON("POST", "/register", body, nil) -} - -// Heartbeat sends a heartbeat to the hub portal. -func (h *HubService) Heartbeat() error { - body := map[string]string{ - "client_id": h.GetClientID(), - } - return h.doJSON("POST", "/heartbeat", body, nil) -} - -// ClaimIssue claims an issue on the hub portal. -// Returns the claim on success, ConflictError if already claimed. -func (h *HubService) ClaimIssue(issue *Issue) (*HubClaim, error) { - if issue == nil { - return nil, fmt.Errorf("issue is nil") - } - - h.drainPendingOps() - - body := map[string]interface{}{ - "client_id": h.GetClientID(), - "issue_id": issue.ID, - "repo": issue.Repo, - "issue_number": issue.Number, - "title": issue.Title, - "url": issue.URL, - } - - var result struct { - Claim *HubClaim `json:"claim"` - } - - if err := h.doJSON("POST", "/issues/claim", body, &result); err != nil { - return nil, err - } - - return result.Claim, nil -} - -// UpdateStatus updates the status of a claimed issue. -func (h *HubService) UpdateStatus(issueID, status, prURL string, prNumber int) error { - body := map[string]interface{}{ - "client_id": h.GetClientID(), - "status": status, - } - if prURL != "" { - body["pr_url"] = prURL - body["pr_number"] = prNumber - } - - encodedID := url.PathEscape(issueID) - return h.doJSON("PATCH", "/issues/"+encodedID+"/status", body, nil) -} - -// ReleaseClaim releases a claim on an issue. -func (h *HubService) ReleaseClaim(issueID string) error { - body := map[string]string{ - "client_id": h.GetClientID(), - } - - encodedID := url.PathEscape(issueID) - return h.doJSON("DELETE", "/issues/"+encodedID+"/claim", body, nil) -} - -// SyncStats uploads local stats to the hub portal. -func (h *HubService) SyncStats(stats *Stats) error { - if stats == nil { - return fmt.Errorf("stats is nil") - } - - repoNames := make([]string, 0, len(stats.ReposContributed)) - for name := range stats.ReposContributed { - repoNames = append(repoNames, name) - } - - body := map[string]interface{}{ - "client_id": h.GetClientID(), - "stats": map[string]interface{}{ - "issues_attempted": stats.IssuesAttempted, - "issues_completed": stats.IssuesCompleted, - "issues_skipped": stats.IssuesSkipped, - "prs_submitted": stats.PRsSubmitted, - "prs_merged": stats.PRsMerged, - "prs_rejected": stats.PRsRejected, - "current_streak": stats.CurrentStreak, - "longest_streak": stats.LongestStreak, - "total_time_minutes": int(stats.TotalTimeSpent.Minutes()), - "repos_contributed": repoNames, - }, - } - - return h.doJSON("POST", "/stats/sync", body, nil) -} -``` - -**Step 4: Run tests** - -Run: `go test ./internal/bugseti/... -run "TestRegister_Good|TestHeartbeat|TestClaimIssue|TestUpdateStatus|TestSyncStats" -count=1` -Expected: PASS - -**Step 5: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): hub write operations (register, heartbeat, claim, update, sync)" -``` - ---- - -### Task 6: Read Operations — IsIssueClaimed, ListClaims, GetLeaderboard, GetGlobalStats - -**Files:** -- Modify: `internal/bugseti/hub.go` -- Modify: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Add to `hub_test.go`: - -```go -func TestIsIssueClaimed_Good_Claimed(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{ - "claim": map[string]interface{}{"issue_id": "o/r#1", "status": "claimed"}, - }) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - claim, err := h.IsIssueClaimed("o/r#1") - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if claim == nil { - t.Fatal("expected claim") - } -} - -func TestIsIssueClaimed_Good_NotClaimed(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(404) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - claim, err := h.IsIssueClaimed("o/r#1") - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if claim != nil { - t.Fatal("expected nil claim for unclaimed issue") - } -} - -func TestGetLeaderboard_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Query().Get("limit") != "10" { - t.Fatalf("expected limit=10, got %s", r.URL.Query().Get("limit")) - } - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{ - "leaderboard": []map[string]interface{}{{"rank": 1, "client_name": "Alice"}}, - "total_participants": 5, - }) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - entries, total, err := h.GetLeaderboard(10) - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if len(entries) != 1 || total != 5 { - t.Fatalf("expected 1 entry, 5 total; got %d, %d", len(entries), total) - } -} - -func TestGetGlobalStats_Good(t *testing.T) { - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{ - "global": map[string]interface{}{ - "total_participants": 11, - "active_claims": 3, - }, - }) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - stats, err := h.GetGlobalStats() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - if stats.TotalParticipants != 11 { - t.Fatalf("expected 11 participants, got %d", stats.TotalParticipants) - } -} -``` - -**Step 2: Run to verify failure, then implement** - -Add to `hub.go`: - -```go -// IsIssueClaimed checks if an issue is claimed on the hub. -// Returns the claim if found, nil if not claimed. -func (h *HubService) IsIssueClaimed(issueID string) (*HubClaim, error) { - var result struct { - Claim *HubClaim `json:"claim"` - } - - encodedID := url.PathEscape(issueID) - err := h.doJSON("GET", "/issues/"+encodedID, nil, &result) - if err != nil { - if _, ok := err.(*NotFoundError); ok { - return nil, nil // Not claimed - } - return nil, err - } - - return result.Claim, nil -} - -// ListClaims returns active claims from the hub, with optional filters. -func (h *HubService) ListClaims(status, repo string) ([]*HubClaim, error) { - path := "/issues/claimed" - params := url.Values{} - if status != "" { - params.Set("status", status) - } - if repo != "" { - params.Set("repo", repo) - } - if len(params) > 0 { - path += "?" + params.Encode() - } - - var result struct { - Claims []*HubClaim `json:"claims"` - } - - if err := h.doJSON("GET", path, nil, &result); err != nil { - return nil, err - } - - return result.Claims, nil -} - -// GetLeaderboard returns the leaderboard from the hub portal. -func (h *HubService) GetLeaderboard(limit int) ([]LeaderboardEntry, int, error) { - if limit <= 0 { - limit = 20 - } - - path := fmt.Sprintf("/leaderboard?limit=%d", limit) - - var result struct { - Leaderboard []LeaderboardEntry `json:"leaderboard"` - TotalParticipants int `json:"total_participants"` - } - - if err := h.doJSON("GET", path, nil, &result); err != nil { - return nil, 0, err - } - - return result.Leaderboard, result.TotalParticipants, nil -} - -// GetGlobalStats returns aggregate stats from the hub portal. -func (h *HubService) GetGlobalStats() (*GlobalStats, error) { - var result struct { - Global *GlobalStats `json:"global"` - } - - if err := h.doJSON("GET", "/stats", nil, &result); err != nil { - return nil, err - } - - return result.Global, nil -} -``` - -**Step 3: Run tests** - -Run: `go test ./internal/bugseti/... -run "TestIsIssueClaimed|TestGetLeaderboard|TestGetGlobalStats" -count=1` -Expected: PASS - -**Step 4: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): hub read operations (claims, leaderboard, global stats)" -``` - ---- - -### Task 7: Pending Operations Queue - -Implement offline-first: queue failed writes, persist to disk, drain on reconnect. - -**Files:** -- Modify: `internal/bugseti/hub.go` -- Modify: `internal/bugseti/hub_test.go` - -**Step 1: Write failing tests** - -Add to `hub_test.go`: - -```go -func TestPendingOps_Good_QueueAndDrain(t *testing.T) { - callCount := 0 - server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - callCount++ - if r.URL.Path == "/api/bugseti/register" { - // First register drains pending ops — the heartbeat will come first - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{"client": nil}) - return - } - w.WriteHeader(200) - json.NewEncoder(w).Encode(map[string]interface{}{"ok": true}) - })) - defer server.Close() - - h := testHubService(t, server.URL) - h.config.config.HubToken = "ak_test" - - // Manually add a pending op - h.mu.Lock() - h.pendingOps = append(h.pendingOps, PendingOp{ - Method: "POST", - Path: "/heartbeat", - Body: []byte(`{"client_id":"test"}`), - CreatedAt: time.Now(), - }) - h.mu.Unlock() - - // Register should drain the pending heartbeat first - err := h.Register() - if err != nil { - t.Fatalf("unexpected error: %v", err) - } - - if callCount < 2 { - t.Fatalf("expected at least 2 calls (drain + register), got %d", callCount) - } -} - -func TestPendingOps_Good_PersistAndLoad(t *testing.T) { - cfg := testConfigService(t, nil, nil) - h1 := NewHubService(cfg) - - // Add pending op - h1.mu.Lock() - h1.pendingOps = append(h1.pendingOps, PendingOp{ - Method: "POST", - Path: "/heartbeat", - Body: []byte(`{"test":true}`), - CreatedAt: time.Now(), - }) - h1.mu.Unlock() - h1.savePendingOps() - - // Create new service — should load persisted ops - h2 := NewHubService(cfg) - h2.mu.Lock() - count := len(h2.pendingOps) - h2.mu.Unlock() - - if count != 1 { - t.Fatalf("expected 1 pending op after reload, got %d", count) - } -} -``` - -**Step 2: Implement pending ops** - -Add to `hub.go`: - -```go -// queueOp adds a failed write to the pending queue. -func (h *HubService) queueOp(method, path string, body interface{}) { - data, _ := json.Marshal(body) - h.mu.Lock() - h.pendingOps = append(h.pendingOps, PendingOp{ - Method: method, - Path: path, - Body: data, - CreatedAt: time.Now(), - }) - h.mu.Unlock() - h.savePendingOps() -} - -// drainPendingOps replays queued operations. Called before write methods. -func (h *HubService) drainPendingOps() { - h.mu.Lock() - ops := h.pendingOps - h.pendingOps = nil - h.mu.Unlock() - - if len(ops) == 0 { - return - } - - log.Printf("Hub: draining %d pending operations", len(ops)) - var failed []PendingOp - - for _, op := range ops { - resp, err := h.doRequest(op.Method, op.Path, json.RawMessage(op.Body)) - if err != nil { - failed = append(failed, op) - continue - } - resp.Body.Close() - if resp.StatusCode >= 500 { - failed = append(failed, op) - } - // 4xx errors are dropped (stale data) - } - - if len(failed) > 0 { - h.mu.Lock() - h.pendingOps = append(failed, h.pendingOps...) - h.mu.Unlock() - } - - h.savePendingOps() -} - -// savePendingOps persists the pending queue to disk. -func (h *HubService) savePendingOps() { - dataDir := h.config.GetDataDir() - if dataDir == "" { - return - } - - h.mu.Lock() - ops := h.pendingOps - h.mu.Unlock() - - data, err := json.Marshal(ops) - if err != nil { - return - } - - path := filepath.Join(dataDir, "hub_pending.json") - os.WriteFile(path, data, 0600) -} - -// loadPendingOps loads persisted pending operations from disk. -func (h *HubService) loadPendingOps() { - dataDir := h.config.GetDataDir() - if dataDir == "" { - return - } - - path := filepath.Join(dataDir, "hub_pending.json") - data, err := os.ReadFile(path) - if err != nil { - return - } - - var ops []PendingOp - if err := json.Unmarshal(data, &ops); err != nil { - return - } - - h.mu.Lock() - h.pendingOps = ops - h.mu.Unlock() -} - -// PendingCount returns the number of queued operations. -func (h *HubService) PendingCount() int { - h.mu.Lock() - defer h.mu.Unlock() - return len(h.pendingOps) -} -``` - -Also add `"os"` and `"path/filepath"` to the imports in `hub.go`. - -**Step 3: Run tests** - -Run: `go test ./internal/bugseti/... -run TestPendingOps -count=1` -Expected: PASS - -**Step 4: Commit** - -```bash -git add internal/bugseti/hub.go internal/bugseti/hub_test.go -git commit -m "feat(bugseti): hub pending operations queue with disk persistence" -``` - ---- - -### Task 8: Integration — main.go and Wails Registration - -Wire HubService into the app lifecycle. - -**Files:** -- Modify: `cmd/bugseti/main.go` - -**Step 1: Create HubService in main.go** - -After the `submitService` creation, add: - -```go -hubService := bugseti.NewHubService(configService) -``` - -Add to the services slice: - -```go -application.NewService(hubService), -``` - -After `log.Println("Starting BugSETI...")`, add: - -```go -// Attempt hub registration (non-blocking, logs warnings on failure) -if hubURL := configService.GetHubURL(); hubURL != "" { - if err := hubService.AutoRegister(); err != nil { - log.Printf("Hub: auto-register skipped: %v", err) - } else if err := hubService.Register(); err != nil { - log.Printf("Hub: registration failed: %v", err) - } -} -``` - -**Step 2: Build and verify** - -Run: `task bugseti:build` -Expected: Builds successfully. - -Run: `go test ./internal/bugseti/... -count=1` -Expected: All tests pass. - -**Step 3: Commit** - -```bash -git add cmd/bugseti/main.go -git commit -m "feat(bugseti): wire HubService into app lifecycle" -``` - ---- - -### Task 9: Laravel Auth/Forge Endpoint - -Create the portal-side endpoint that exchanges a forge token for an `ak_` API key. - -**Files:** -- Create: `agentic/app/Mod/BugSeti/Controllers/AuthController.php` -- Modify: `agentic/app/Mod/BugSeti/Routes/api.php` - -**Step 1: Create AuthController** - -Create `agentic/app/Mod/BugSeti/Controllers/AuthController.php`: - -```php -validate([ - 'forge_url' => 'required|url|max:500', - 'forge_token' => 'required|string|max:255', - 'client_id' => 'required|string|max:64', - ]); - - // Validate the forge token against the Forgejo API - $response = Http::withToken($validated['forge_token']) - ->timeout(10) - ->get(rtrim($validated['forge_url'], '/') . '/api/v1/user'); - - if (! $response->ok()) { - return response()->json([ - 'error' => 'Invalid Forgejo token — could not verify identity.', - ], 401); - } - - $forgeUser = $response->json(); - $forgeName = $forgeUser['full_name'] ?: $forgeUser['login'] ?? 'Unknown'; - - // Find or create workspace for BugSETI clients - $workspace = Workspace::firstOrCreate( - ['slug' => 'bugseti-community'], - ['name' => 'BugSETI Community', 'owner_id' => null] - ); - - // Check if this client already has a key - $existingKey = AgentApiKey::where('workspace_id', $workspace->id) - ->where('name', 'like', '%' . $validated['client_id'] . '%') - ->whereNull('revoked_at') - ->first(); - - if ($existingKey) { - // Revoke old key and issue new one - $existingKey->update(['revoked_at' => now()]); - } - - $apiKey = AgentApiKey::generate( - workspace: $workspace->id, - name: "BugSETI — {$forgeName} ({$validated['client_id']})", - permissions: ['bugseti.read', 'bugseti.write'], - rateLimit: 100, - expiresAt: null, - ); - - return response()->json([ - 'api_key' => $apiKey->plainTextKey, - 'forge_user' => $forgeName, - ], 201); - } -} -``` - -**Step 2: Add route** - -In `agentic/app/Mod/BugSeti/Routes/api.php`, add **outside** the authenticated groups: - -```php -// Unauthenticated bootstrap — exchanges forge token for API key -Route::post('/auth/forge', [AuthController::class, 'forge']); -``` - -Add the use statement at top of file: - -```php -use Mod\BugSeti\Controllers\AuthController; -``` - -**Step 3: Test manually** - -```bash -cd /Users/snider/Code/host-uk/agentic -php artisan migrate -curl -X POST http://leth.test/api/bugseti/auth/forge \ - -H "Content-Type: application/json" \ - -d '{"forge_url":"https://forge.lthn.io","forge_token":"500ecb79c79da940205f37580438575dbf7a82be","client_id":"test-client-1"}' -``` - -Expected: 201 with `{"api_key":"ak_...","forge_user":"..."}`. - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/host-uk/agentic -git add app/Mod/BugSeti/Controllers/AuthController.php app/Mod/BugSeti/Routes/api.php -git commit -m "feat(bugseti): add /auth/forge endpoint for token exchange" -``` - ---- - -### Task 10: Full Integration Test - -Build the binary, configure hub URL, and verify end-to-end. - -**Files:** None (verification only) - -**Step 1: Run all Go tests** - -```bash -cd /Users/snider/Code/host-uk/core -go test ./internal/bugseti/... -count=1 -v -``` - -Expected: All tests pass. - -**Step 2: Build binary** - -```bash -task bugseti:build -``` - -Expected: Binary builds at `bin/bugseti`. - -**Step 3: Configure hub URL and test launch** - -```bash -# Set hub URL to devnet -cat ~/.config/bugseti/config.json | python3 -c " -import json,sys -c = json.load(sys.stdin) -c['hubUrl'] = 'https://leth.in' -json.dump(c, sys.stdout, indent=2) -" > /tmp/bugseti-config.json && mv /tmp/bugseti-config.json ~/.config/bugseti/config.json -``` - -Launch `./bin/bugseti` — should start without errors, attempt hub registration. - -**Step 4: Final commit if needed** - -```bash -git add -A && git commit -m "feat(bugseti): HubService integration complete" -``` - ---- - -### Summary - -| Task | Description | Files | -|------|-------------|-------| -| 1 | Config fields | config.go | -| 2 | HubService types + constructor | hub.go, hub_test.go | -| 3 | HTTP request helpers | hub.go, hub_test.go | -| 4 | Auto-register via forge | hub.go, hub_test.go | -| 5 | Write operations | hub.go, hub_test.go | -| 6 | Read operations | hub.go, hub_test.go | -| 7 | Pending ops queue | hub.go, hub_test.go | -| 8 | main.go integration | main.go | -| 9 | Laravel auth/forge endpoint | AuthController.php, api.php | -| 10 | Full integration test | (verification) | diff --git a/docs/plans/completed/2026-02-17-lem-chat-design.md b/docs/plans/completed/2026-02-17-lem-chat-design.md deleted file mode 100644 index 3ff9f36..0000000 --- a/docs/plans/completed/2026-02-17-lem-chat-design.md +++ /dev/null @@ -1,82 +0,0 @@ -# LEM Chat — Web Components Design - -**Date**: 2026-02-17 -**Status**: Approved - -## Summary - -Standalone chat UI built with vanilla Web Components (Custom Elements + Shadow DOM). Connects to the MLX inference server's OpenAI-compatible SSE streaming endpoint. Zero framework dependencies. Single JS file output, embeddable anywhere. - -## Components - -| Element | Purpose | -|---------|---------| -| `` | Container. Conversation state, SSE connection, config via attributes | -| `` | Scrollable message list with auto-scroll anchoring | -| `` | Single message bubble. Streams tokens for assistant messages | -| `` | Text input, Enter to send, Shift+Enter for newline | - -## Data Flow - -``` -User types in - → dispatches 'lem-send' CustomEvent - → catches it - → adds user message to - → POST /v1/chat/completions {stream: true, messages: [...history]} - → reads SSE chunks via fetch + ReadableStream - → appends tokens to streaming - → on [DONE], finalises message -``` - -## Configuration - -```html - -``` - -Attributes: `endpoint`, `model`, `system-prompt`, `max-tokens`, `temperature` - -## Theming - -Shadow DOM with CSS custom properties: - -```css ---lem-bg: #1a1a1e; ---lem-msg-user: #2a2a3e; ---lem-msg-assistant: #1e1e2a; ---lem-accent: #5865f2; ---lem-text: #e0e0e0; ---lem-font: system-ui; -``` - -## Markdown - -Minimal inline parsing: fenced code blocks, inline code, bold, italic. No library. - -## File Structure - -``` -lem-chat/ -├── index.html # Demo page -├── src/ -│ ├── lem-chat.ts # Main container + SSE client -│ ├── lem-messages.ts # Message list with scroll anchoring -│ ├── lem-message.ts # Single message with streaming -│ ├── lem-input.ts # Text input -│ ├── markdown.ts # Minimal markdown → HTML -│ └── styles.ts # CSS template literals -├── package.json # typescript + esbuild -└── tsconfig.json -``` - -Build: `esbuild src/lem-chat.ts --bundle --outfile=dist/lem-chat.js` - -## Not in v1 - -- Model selection UI -- Conversation persistence -- File/image upload -- Syntax highlighting -- Typing indicators -- User avatars diff --git a/docs/plans/completed/2026-02-20-go-api-design-original.md b/docs/plans/completed/2026-02-20-go-api-design-original.md deleted file mode 100644 index c979f81..0000000 --- a/docs/plans/completed/2026-02-20-go-api-design-original.md +++ /dev/null @@ -1,657 +0,0 @@ -# go-api Design — HTTP Gateway + OpenAPI SDK Generation - -**Date:** 2026-02-20 -**Author:** Virgil -**Status:** Phase 1 + Phase 2 + Phase 3 Complete (176 tests in go-api) -**Module:** `forge.lthn.ai/core/go-api` - -## Problem - -The Core Go ecosystem exposes 42+ tools via MCP (JSON-RPC), which is ideal for AI agents but inaccessible to regular HTTP clients, frontend applications, and third-party integrators. There is no unified HTTP gateway, no OpenAPI specification, and no generated SDKs. - -Both external customers (Host UK products) and Lethean network peers need programmatic access to the same services. The gateway also serves web routes, static assets, and streaming endpoints — not just REST APIs. - -## Solution - -A `go-api` package that acts as the central HTTP gateway: - -1. **Gin-based HTTP gateway** with extensible middleware via gin-contrib plugins -2. **RouteGroup interface** that subsystems implement to register their own endpoints (API, web, or both) -3. **WebSocket + SSE integration** for real-time streaming -4. **OpenAPI 3.1 spec generation** via runtime SpecBuilder (not swaggo annotations) -5. **SDK generation pipeline** targeting 11 languages via openapi-generator-cli - -## Architecture - -### Four-Protocol Access - -Same backend services, four client protocols: - -``` - ┌─── REST (go-api) POST /v1/ml/generate → JSON - │ - ├─── GraphQL (gqlgen) mutation { mlGenerate(...) { response } } -Client ────────────┤ - ├─── WebSocket (go-ws) subscribe ml.generate → streaming - │ - └─── MCP (go-ai) ml_generate → JSON-RPC -``` - -### Dependency Graph - -``` -go-api (Gin engine + middleware + OpenAPI) - ↑ imported by (each registers its own routes) - ├── go-ai/api/ → /v1/file/*, /v1/process/*, /v1/metrics/* - ├── go-ml/api/ → /v1/ml/* - ├── go-rag/api/ → /v1/rag/* - ├── go-agentic/api/ → /v1/tasks/* - ├── go-help/api/ → /v1/help/* - └── go-ws/api/ → /ws (WebSocket upgrade) -``` - -go-api has zero internal ecosystem dependencies. Subsystems import go-api, not the other way round. - -### Subsystem Opt-In - -Not every MCP tool becomes a REST endpoint. Each subsystem decides what to expose via a separate `RegisterAPI()` method, independent of MCP's `RegisterTools()`. A subsystem with 15 MCP tools might expose 5 REST endpoints. - -## Package Structure - -``` -forge.lthn.ai/core/go-api -├── api.go # Engine struct, New(), Serve(), Shutdown() -├── middleware.go # Auth, CORS, rate limiting, request logging, recovery -├── options.go # WithAddr, WithAuth, WithCORS, WithRateLimit, etc. -├── group.go # RouteGroup interface + registration -├── response.go # Envelope type, error responses, pagination -├── docs/ # Generated swagger docs (swaggo output) -├── sdk/ # SDK generation tooling / Makefile targets -└── go.mod # forge.lthn.ai/core/go-api -``` - -## Core Interface - -```go -// RouteGroup registers API routes onto a Gin router group. -// Subsystems implement this to expose their endpoints. -type RouteGroup interface { - // Name returns the route group identifier (e.g. "ml", "rag", "tasks") - Name() string - // BasePath returns the URL prefix (e.g. "/v1/ml") - BasePath() string - // RegisterRoutes adds handlers to the provided router group - RegisterRoutes(rg *gin.RouterGroup) -} - -// StreamGroup optionally declares WebSocket channels a subsystem publishes to. -type StreamGroup interface { - Channels() []string -} -``` - -### Subsystem Example (go-ml) - -```go -// In go-ml/api/routes.go -package api - -type Routes struct { - service *ml.Service -} - -func NewRoutes(svc *ml.Service) *Routes { - return &Routes{service: svc} -} - -func (r *Routes) Name() string { return "ml" } -func (r *Routes) BasePath() string { return "/v1/ml" } - -func (r *Routes) RegisterRoutes(rg *gin.RouterGroup) { - rg.POST("/generate", r.Generate) - rg.POST("/score", r.Score) - rg.GET("/backends", r.Backends) - rg.GET("/status", r.Status) -} - -func (r *Routes) Channels() []string { - return []string{"ml.generate", "ml.status"} -} - -// @Summary Generate text via ML backend -// @Tags ml -// @Accept json -// @Produce json -// @Param input body MLGenerateInput true "Generation parameters" -// @Success 200 {object} Response[MLGenerateOutput] -// @Router /v1/ml/generate [post] -func (r *Routes) Generate(c *gin.Context) { - var input MLGenerateInput - if err := c.ShouldBindJSON(&input); err != nil { - c.JSON(400, api.Fail("invalid_input", err.Error())) - return - } - result, err := r.service.Generate(c.Request.Context(), input.Backend, input.Prompt, ml.GenOpts{ - Temperature: input.Temperature, - MaxTokens: input.MaxTokens, - Model: input.Model, - }) - if err != nil { - c.JSON(500, api.Fail("ml.generate_failed", err.Error())) - return - } - c.JSON(200, api.OK(MLGenerateOutput{ - Response: result, - Backend: input.Backend, - Model: input.Model, - })) -} -``` - -### Engine Wiring (in core CLI) - -```go -engine := api.New( - api.WithAddr(":8080"), - api.WithCORS("*"), - api.WithAuth(api.BearerToken(cfg.APIKey)), - api.WithRateLimit(100, time.Minute), - api.WithWSHub(wsHub), -) - -engine.Register(mlapi.NewRoutes(mlService)) -engine.Register(ragapi.NewRoutes(ragService)) -engine.Register(agenticapi.NewRoutes(agenticService)) - -engine.Serve(ctx) // Blocks until context cancelled -``` - -## Response Envelope - -All endpoints return a consistent envelope: - -```go -type Response[T any] struct { - Success bool `json:"success"` - Data T `json:"data,omitempty"` - Error *Error `json:"error,omitempty"` - Meta *Meta `json:"meta,omitempty"` -} - -type Error struct { - Code string `json:"code"` - Message string `json:"message"` - Details any `json:"details,omitempty"` -} - -type Meta struct { - RequestID string `json:"request_id"` - Duration string `json:"duration"` - Page int `json:"page,omitempty"` - PerPage int `json:"per_page,omitempty"` - Total int `json:"total,omitempty"` -} -``` - -Helper functions: - -```go -func OK[T any](data T) Response[T] -func Fail(code, message string) Response[any] -func Paginated[T any](data T, page, perPage, total int) Response[T] -``` - -## Middleware Stack - -```go -api.New( - api.WithAddr(":8080"), - api.WithCORS(api.CORSConfig{...}), // gin-contrib/cors - api.WithAuth(api.BearerToken("...")), // Phase 1: simple bearer token - api.WithRateLimit(100, time.Minute), // Per-IP sliding window - api.WithRequestID(), // X-Request-ID header generation - api.WithRecovery(), // Panic recovery → 500 response - api.WithLogger(slog.Default()), // Structured request logging -) -``` - -Auth evolution path: bearer token → API keys → Authentik (OIDC/forward auth). Middleware slot stays the same. - -## WebSocket Integration - -go-api wraps the existing go-ws Hub as a first-class transport: - -```go -// Automatic registration: -// GET /ws → WebSocket upgrade (go-ws Hub) - -// Client subscribes: {"type":"subscribe","channel":"ml.generate"} -// Events arrive: {"type":"event","channel":"ml.generate","data":{...}} -// Client unsubscribes: {"type":"unsubscribe","channel":"ml.generate"} -``` - -Subsystems implementing `StreamGroup` declare which channels they publish to. This metadata feeds into the OpenAPI spec as documentation. - -## OpenAPI + SDK Generation - -### Runtime Spec Generation (SpecBuilder) - -swaggo annotations were rejected because routes are dynamic via RouteGroup, Response[T] generics break swaggo, and MCP tools already carry JSON Schema at runtime. Instead, a `SpecBuilder` constructs the full OpenAPI 3.1 spec from registered RouteGroups at runtime. - -```go -// Groups that implement DescribableGroup contribute endpoint metadata -type DescribableGroup interface { - RouteGroup - Describe() []RouteDescription -} - -// SpecBuilder assembles the spec from all groups -builder := &api.SpecBuilder{Title: "Core API", Description: "...", Version: "1.0.0"} -spec, _ := builder.Build(engine.Groups()) -``` - -### MCP-to-REST Bridge (ToolBridge) - -The `ToolBridge` converts MCP tool descriptors into REST POST endpoints and implements both `RouteGroup` and `DescribableGroup`. Each tool becomes `POST /{tool_name}`. Generic types are captured at MCP registration time via closures, enabling JSON unmarshalling to the correct input type at request time. - -```go -bridge := api.NewToolBridge("/v1/tools") -mcp.BridgeToAPI(mcpService, bridge) // Populates bridge from MCP tool registry -engine.Register(bridge) // Registers REST endpoints + OpenAPI metadata -``` - -### Swagger UI - -```go -// Built-in at GET /swagger/*any -// SpecBuilder output served via gin-swagger, cached via sync.Once -api.New(api.WithSwagger("Core API", "...", "1.0.0")) -``` - -### SDK Generation - -```bash -# Via openapi-generator-cli (11 languages supported) -core api sdk --lang go # Generate Go SDK -core api sdk --lang typescript-fetch,python # Multiple languages -core api sdk --lang rust --output ./sdk/ # Custom output dir -``` - -### CLI Commands - -```bash -core api spec # Emit OpenAPI JSON to stdout -core api spec --format yaml # YAML variant -core api spec --output spec.json # Write to file -core api sdk --lang python # Generate Python SDK -core api sdk --lang go,rust # Multiple SDKs -``` - -## Dependencies - -| Package | Purpose | -|---------|---------| -| `github.com/gin-gonic/gin` | HTTP framework | -| `github.com/swaggo/gin-swagger` | Swagger UI middleware | -| `github.com/gin-contrib/cors` | CORS middleware | -| `github.com/gin-contrib/secure` | Security headers | -| `github.com/gin-contrib/sessions` | Server-side sessions | -| `github.com/gin-contrib/authz` | Casbin authorisation | -| `github.com/gin-contrib/httpsign` | HTTP signature verification | -| `github.com/gin-contrib/slog` | Structured request logging | -| `github.com/gin-contrib/timeout` | Per-request timeouts | -| `github.com/gin-contrib/gzip` | Gzip compression | -| `github.com/gin-contrib/static` | Static file serving | -| `github.com/gin-contrib/pprof` | Runtime profiling | -| `github.com/gin-contrib/expvar` | Runtime metrics | -| `github.com/gin-contrib/location/v2` | Reverse proxy detection | -| `github.com/99designs/gqlgen` | GraphQL endpoint | -| `go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin` | Distributed tracing | -| `gopkg.in/yaml.v3` | YAML spec export | -| `forge.lthn.ai/core/go-ws` | WebSocket Hub (existing) | - -## Estimated Size - -| Component | LOC | -|-----------|-----| -| Engine + options | ~200 | -| Middleware | ~150 | -| Response envelope | ~80 | -| RouteGroup interface | ~30 | -| WebSocket integration | ~60 | -| Tests | ~300 | -| **Total go-api** | **~820** | - -Each subsystem's `api/` package adds ~100-200 LOC per route group. - -## Phase 1 — Implemented (20 Feb 2026) - -**Commit:** `17ae945` on Forge (`core/go-api`) - -| Component | Status | Tests | -|-----------|--------|-------| -| Response envelope (OK, Fail, Paginated) | Done | 9 | -| RouteGroup + StreamGroup interfaces | Done | 4 | -| Engine (New, Register, Handler, Serve) | Done | 9 | -| Bearer auth middleware | Done | 3 | -| Request ID middleware | Done | 2 | -| CORS middleware (gin-contrib/cors) | Done | 3 | -| WebSocket endpoint | Done | 3 | -| Swagger UI (gin-swagger) | Done | 2 | -| Health endpoint | Done | 1 | -| **Total** | **~840 LOC** | **36** | - -**Integration proof:** go-ml/api/ registers 3 endpoints with 12 tests (`0c23858`). - -## Phase 2 Wave 1 — Implemented (20 Feb 2026) - -**Commits:** `6bb7195..daae6f7` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | -|-----------|--------|------------|-------| -| Authentik (forward auth + OIDC) | `WithAuthentik()` | `go-oidc/v3`, `oauth2` | 14 | -| Security headers (HSTS, CSP, etc.) | `WithSecure()` | `gin-contrib/secure` | 8 | -| Structured request logging | `WithSlog()` | `gin-contrib/slog` | 6 | -| Per-request timeouts | `WithTimeout()` | `gin-contrib/timeout` | 5 | -| Gzip compression | `WithGzip()` | `gin-contrib/gzip` | 5 | -| Static file serving | `WithStatic()` | `gin-contrib/static` | 5 | -| **Wave 1 Total** | | | **43** | - -**Cumulative:** 76 tests (36 Phase 1 + 43 Wave 1 - 3 shared), all passing. - -## Phase 2 Wave 2 — Implemented (20 Feb 2026) - -**Commits:** `64a8b16..67dcc83` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| Brotli compression | `WithBrotli()` | `andybalholm/brotli` | 5 | Custom middleware; `gin-contrib/brotli` is empty stub | -| Response caching | `WithCache()` | none (in-memory) | 5 | Custom middleware; `gin-contrib/cache` is per-handler, not global | -| Server-side sessions | `WithSessions()` | `gin-contrib/sessions` | 5 | Cookie store, configurable name + secret | -| Casbin authorisation | `WithAuthz()` | `gin-contrib/authz`, `casbin/v2` | 5 | Subject via Basic Auth; RBAC policy model | -| **Wave 2 Total** | | | **20** | | - -**Cumulative:** 102 passing tests (2 integration skipped), all green. - -## Phase 2 Wave 3 — Implemented (20 Feb 2026) - -**Commits:** `7b3f99e..d517fa2` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| HTTP signature verification | `WithHTTPSign()` | `gin-contrib/httpsign` | 5 | HMAC-SHA256; extensible via httpsign.Option | -| Server-Sent Events | `WithSSE()` | none (custom SSEBroker) | 6 | Channel filtering, multi-client broadcast, GET /events | -| Reverse proxy detection | `WithLocation()` | `gin-contrib/location/v2` | 5 | X-Forwarded-Host/Proto parsing | -| Locale detection | `WithI18n()` | `golang.org/x/text/language` | 5 | Accept-Language parsing, message lookup, GetLocale/GetMessage | -| GraphQL endpoint | `WithGraphQL()` | `99designs/gqlgen` | 5 | /graphql + optional /graphql/playground | -| **Wave 3 Total** | | | **26** | | - -**Cumulative:** 128 passing tests (2 integration skipped), all green. - -## Phase 2 Wave 4 — Implemented (21 Feb 2026) - -**Commits:** `32b3680..8ba1716` on Forge (`core/go-api`) - -| Component | Option | Dependency | Tests | Notes | -|-----------|--------|------------|-------|-------| -| Runtime profiling | `WithPprof()` | `gin-contrib/pprof` | 5 | /debug/pprof/* endpoints, flag-based mount | -| Runtime metrics | `WithExpvar()` | `gin-contrib/expvar` | 5 | /debug/vars endpoint, flag-based mount | -| Distributed tracing | `WithTracing()` | `otelgin` + OpenTelemetry SDK | 5 | W3C traceparent propagation, span attributes | -| **Wave 4 Total** | | | **15** | | - -**Cumulative:** 143 passing tests (2 integration skipped), all green. - -**Phase 2 complete.** All 4 waves implemented. Every planned plugin has a `With*()` option and tests. - -## Phase 3 — OpenAPI Spec Generation + SDK Codegen (21 Feb 2026) - -**Architecture:** Runtime OpenAPI generation via SpecBuilder (NOT swaggo annotations). Routes are dynamic via RouteGroup, Response[T] generics break swaggo, and MCP tools carry JSON Schema at runtime. A `ToolBridge` converts tool descriptors into RouteGroup + OpenAPI metadata. A `SpecBuilder` constructs the full OpenAPI 3.1 spec. SDK codegen wraps `openapi-generator-cli`. - -### Wave 1: go-api (Tasks 1-5) - -**Commits:** `465bd60..1910aec` on Forge (`core/go-api`) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| DescribableGroup interface | `group.go` | 5 | Opt-in OpenAPI metadata for RouteGroups | -| ToolBridge | `bridge.go` | 6 | Tool descriptors → POST endpoints + DescribableGroup | -| SpecBuilder | `openapi.go` | 6 | OpenAPI 3.1 JSON with Response[T] envelope wrapping | -| Swagger refactor | `swagger.go` | 5 | Replaced hardcoded empty spec with SpecBuilder | -| Spec export | `export.go` | 5 | JSON + YAML export to file/writer | -| SDK codegen | `codegen.go` | 5 | 11-language wrapper for openapi-generator-cli | -| **Wave 1 Total** | | **32** | | - -### Wave 2: go-ai MCP bridge (Tasks 6-7) - -**Commits:** `2107eda..c37e1cf` on Forge (`core/go-ai`) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| Tool registry | `mcp/registry.go` | 5 | Generic `addToolRecorded[In,Out]` captures types in closures | -| BridgeToAPI | `mcp/bridge.go` | 5 | MCP tools → go-api ToolBridge, 10MB body limit, error classification | -| **Wave 2 Total** | | **10** | | - -### Wave 3: CLI commands (Tasks 8-9) - -**Commit:** `d6eec4d` on Forge (`core/cli` dev branch) - -| Component | File | Tests | Notes | -|-----------|------|-------|-------| -| `core api spec` | `cmd/api/cmd_spec.go` | 2 | JSON/YAML export, --output/--format flags | -| `core api sdk` | `cmd/api/cmd_sdk.go` | 2 | --lang (required), --output, --spec, --package flags | -| **Wave 3 Total** | | **4** | | - -**Cumulative go-api:** 176 passing tests. **Phase 3 complete.** - -### Known Limitations - -- **Subsystem tools excluded from bridge:** Subsystems call `mcp.AddTool` directly, bypassing `addToolRecorded`. Only the 10 built-in MCP tools appear in the REST bridge. Future: pass `*Service` to `RegisterTools` instead of `*mcp.Server`. -- **Flat schema only:** `structSchema` reflection handles flat structs but does not recurse into nested structs. Adequate for current tool inputs. -- **CLI spec produces empty bridge:** `core api spec` currently generates a spec with only `/health`. Full MCP integration requires wiring the MCP service into the CLI command. - -## Phase 2 — Gin Plugin Roadmap (Complete) - -All plugins drop in as `With*()` options on the Engine. No architecture changes needed. - -### Security & Auth - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~**Authentik**~~ | ~~`WithAuthentik()`~~ | ~~OIDC + forward auth integration.~~ | ~~**Done**~~ | -| ~~gin-contrib/secure~~ | ~~`WithSecure()`~~ | ~~Security headers: HSTS, X-Frame-Options, X-Content-Type-Options, CSP.~~ | ~~**Done**~~ | -| ~~gin-contrib/sessions~~ | ~~`WithSessions()`~~ | ~~Server-side sessions (cookie store). Web session management alongside Authentik tokens.~~ | ~~**Done**~~ | -| ~~gin-contrib/authz~~ | ~~`WithAuthz()`~~ | ~~Casbin-based authorisation. Policy-driven access control via RBAC.~~ | ~~**Done**~~ | -| ~~gin-contrib/httpsign~~ | ~~`WithHTTPSign()`~~ | ~~HTTP signature verification. HMAC-SHA256 with extensible options.~~ | ~~**Done**~~ | - -### Performance & Reliability - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/cache~~ | ~~`WithCache()`~~ | ~~Response caching (in-memory). GET response caching with TTL, lazy eviction.~~ | ~~**Done**~~ | -| ~~gin-contrib/timeout~~ | ~~`WithTimeout()`~~ | ~~Per-request timeouts.~~ | ~~**Done**~~ | -| ~~gin-contrib/gzip~~ | ~~`WithGzip()`~~ | ~~Gzip response compression.~~ | ~~**Done**~~ | -| ~~gin-contrib/brotli~~ | ~~`WithBrotli()`~~ | ~~Brotli compression via `andybalholm/brotli`. Custom middleware (gin-contrib stub empty).~~ | ~~**Done**~~ | - -### Observability - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/slog~~ | ~~`WithSlog()`~~ | ~~Structured request logging via slog.~~ | ~~**Done**~~ | -| ~~gin-contrib/pprof~~ | ~~`WithPprof()`~~ | ~~Runtime profiling endpoints at /debug/pprof/. Flag-based mount.~~ | ~~**Done**~~ | -| ~~gin-contrib/expvar~~ | ~~`WithExpvar()`~~ | ~~Go runtime metrics at /debug/vars. Flag-based mount.~~ | ~~**Done**~~ | -| ~~otelgin~~ | ~~`WithTracing()`~~ | ~~OpenTelemetry distributed tracing. W3C traceparent propagation.~~ | ~~**Done**~~ | - -### Content & Streaming - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/static~~ | ~~`WithStatic()`~~ | ~~Serve static files.~~ | ~~**Done**~~ | -| ~~gin-contrib/sse~~ | ~~`WithSSE()`~~ | ~~Server-Sent Events. Custom SSEBroker with channel filtering, GET /events.~~ | ~~**Done**~~ | -| ~~gin-contrib/location~~ | ~~`WithLocation()`~~ | ~~Auto-detect scheme/host from X-Forwarded-* headers.~~ | ~~**Done**~~ | - -### Query Layer - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~99designs/gqlgen~~ | ~~`WithGraphQL()`~~ | ~~GraphQL endpoint at `/graphql` + optional playground. Accepts gqlgen ExecutableSchema.~~ | ~~**Done**~~ | - -The GraphQL schema can be generated from the same Go Input/Output structs that define the REST endpoints. gqlgen produces an `http.Handler` that mounts directly on Gin. Subsystems opt-in via: - -```go -// Subsystems that want GraphQL implement this alongside RouteGroup -type ResolverGroup interface { - // RegisterResolvers adds query/mutation resolvers to the GraphQL schema - RegisterResolvers(schema *graphql.Schema) -} -``` - -This means a subsystem like go-ml exposes: -- **REST:** `POST /v1/ml/generate` (existing) -- **GraphQL:** `mutation { mlGenerate(prompt: "...", backend: "mlx") { response, model } }` (same handler) -- **MCP:** `ml_generate` tool (existing) - -Four protocols, one set of handlers. - -### Ecosystem Integration - -| Plugin | Option | Purpose | Priority | -|--------|--------|---------|----------| -| ~~gin-contrib/i18n~~ | ~~`WithI18n()`~~ | ~~Locale detection via Accept-Language. Custom middleware using `golang.org/x/text/language`.~~ | ~~**Done**~~ | -| [gin-contrib/graceful](https://github.com/gin-contrib/graceful) | — | Already implemented in Engine.Serve(). Could swap to this for more robust lifecycle management if needed. | — | -| [gin-contrib/requestid](https://github.com/gin-contrib/requestid) | — | Already implemented. Theirs uses UUID, ours uses hex. Could swap for standards compliance. | — | - -### Implementation Order - -**Wave 1 (gateway hardening):** ~~Authentik, secure, slog, timeout, gzip, static~~ **DONE** (20 Feb 2026) -**Wave 2 (performance + auth):** ~~cache, sessions, authz, brotli~~ **DONE** (20 Feb 2026) -**Wave 3 (network + streaming):** ~~httpsign, sse, location, i18n, gqlgen~~ **DONE** (20 Feb 2026) -**Wave 4 (observability):** ~~pprof, expvar, tracing~~ **DONE** (21 Feb 2026) - -Each wave adds `With*()` options + tests. No breaking changes — existing code continues to work without any new options enabled. - -## Authentik Integration - -[Authentik](https://goauthentik.io/) is the identity provider and edge auth proxy. It handles user registration, login, MFA, social auth, SAML, and OIDC — so go-api doesn't have to. - -### Two Integration Modes - -**1. Forward Auth (web traffic)** - -Traefik sits in front of go-api. For web routes, Traefik's `forwardAuth` middleware checks with Authentik before passing the request through. Authentik handles login flows, session cookies, and consent. go-api receives pre-authenticated requests with identity headers. - -``` -Browser → Traefik → Authentik (forward auth) → go-api - ↓ - Login page (if unauthenticated) -``` - -go-api reads trusted headers set by Authentik: -``` -X-Authentik-Username: alice -X-Authentik-Groups: admins,developers -X-Authentik-Email: alice@example.com -X-Authentik-Uid: -X-Authentik-Jwt: -``` - -**2. OIDC Token Validation (API traffic)** - -API clients (SDKs, CLI tools, network peers) authenticate directly with Authentik's OAuth2 token endpoint, then send the JWT to go-api. go-api validates the JWT using Authentik's OIDC discovery endpoint (`.well-known/openid-configuration`). - -``` -SDK client → Authentik (token endpoint) → receives JWT -SDK client → go-api (Authorization: Bearer ) → validates via OIDC -``` - -### Implementation in go-api - -```go -engine := api.New( - api.WithAuthentik(api.AuthentikConfig{ - Issuer: "https://auth.lthn.ai/application/o/core-api/", - ClientID: "core-api", - TrustedProxy: true, // Trust X-Authentik-* headers from Traefik - }), -) -``` - -`WithAuthentik()` adds middleware that: -1. Checks for `X-Authentik-Jwt` header (forward auth mode) — validates signature, extracts claims -2. Falls back to `Authorization: Bearer ` header (direct OIDC mode) — validates via JWKS -3. Populates `c.Set("user", AuthentikUser{...})` in the Gin context for handlers to use -4. Skips /health, /swagger, and any public paths - -```go -// In any handler: -func (r *Routes) ListItems(c *gin.Context) { - user := api.GetUser(c) // Returns *AuthentikUser or nil - if user == nil { - c.JSON(401, api.Fail("unauthorised", "Authentication required")) - return - } - // user.Username, user.Groups, user.Email, user.UID available -} -``` - -### Auth Layers - -``` -Authentik (identity) → WHO is this? (user, groups, email) - ↓ -go-api middleware → IS their token valid? (JWT verification) - ↓ -Casbin authz (optional) → CAN they do this? (role → endpoint policies) - ↓ -Handler → DOES this (business logic) -``` - -Phase 1 bearer auth continues to work alongside Authentik — useful for service-to-service tokens, CI/CD, and development. `WithBearerAuth` and `WithAuthentik` can coexist. - -### Authentik Deployment - -Authentik runs as a Docker service alongside go-api, fronted by Traefik: -- **auth.lthn.ai** — Authentik UI + OIDC endpoints (production) -- **auth.leth.in** — Authentik for devnet/testnet -- Traefik routes `/outpost.goauthentik.io/` to Authentik's embedded outpost for forward auth - -### Dependencies - -| Package | Purpose | -|---------|---------| -| `github.com/coreos/go-oidc/v3` | OIDC discovery + JWT validation | -| `golang.org/x/oauth2` | OAuth2 token exchange (for server-side flows) | - -Both are standard Go libraries with no heavy dependencies. - -## Non-Goals - -- gRPC gateway -- Built-in user registration/login (Authentik handles this) -- API versioning beyond /v1/ prefix - -## Success Criteria - -### Phase 1 (Done) - -1. ~~`core api serve` starts a Gin server with registered subsystem routes~~ -2. ~~WebSocket subscriptions work alongside REST~~ -3. ~~Swagger UI accessible at `/swagger/`~~ -4. ~~All endpoints return consistent Response envelope~~ -5. ~~Bearer token auth protects all routes~~ -6. ~~First subsystem integration (go-ml/api/) proves the pattern~~ - -### Phase 2 (Done) - -7. ~~Security headers, compression, and caching active in production~~ -8. ~~Session-based auth alongside bearer tokens~~ -9. ~~HTTP signature verification for Lethean network peers~~ -10. ~~Static file serving for docs site and SDK downloads~~ -11. ~~GraphQL endpoint at `/graphql` with playground~~ - -### Phase 3 (Done) - -12. ~~`core api spec` emits valid OpenAPI 3.1 JSON via runtime SpecBuilder~~ -13. ~~`core api sdk` generates SDKs for 11 languages via openapi-generator-cli~~ -14. ~~MCP tools bridged to REST endpoints via ToolBridge + BridgeToAPI~~ -15. ~~OpenAPI spec includes Response[T] envelope wrapping~~ -16. ~~Spec export to file in JSON and YAML formats~~ diff --git a/docs/plans/completed/2026-02-20-go-api-plan-original.md b/docs/plans/completed/2026-02-20-go-api-plan-original.md deleted file mode 100644 index 11d164d..0000000 --- a/docs/plans/completed/2026-02-20-go-api-plan-original.md +++ /dev/null @@ -1,1503 +0,0 @@ -# go-api Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build `forge.lthn.ai/core/go-api`, a Gin-based REST framework with OpenAPI generation that subsystems plug into via a RouteGroup interface. - -**Architecture:** go-api provides the HTTP engine, middleware stack, response envelope, and OpenAPI tooling. Each ecosystem package (go-ml, go-rag, etc.) imports go-api and registers its own route group. WebSocket support via go-ws Hub runs alongside REST. - -**Tech Stack:** Go 1.25, Gin, swaggo/swag, gin-swagger, gin-contrib/cors, go-ws - -**Design doc:** `docs/plans/2026-02-20-go-api-design.md` - -**Repo location:** `/Users/snider/Code/go-api` (module: `forge.lthn.ai/core/go-api`) - -**Licence:** EUPL-1.2 - -**Convention:** UK English in comments and user-facing strings. Test naming: `_Good`, `_Bad`, `_Ugly`. - ---- - -### Task 1: Scaffold Repository - -**Files:** -- Create: `/Users/snider/Code/go-api/go.mod` -- Create: `/Users/snider/Code/go-api/response.go` -- Create: `/Users/snider/Code/go-api/response_test.go` -- Create: `/Users/snider/Code/go-api/LICENCE` - -**Step 1: Create repo and go.mod** - -```bash -mkdir -p /Users/snider/Code/go-api -cd /Users/snider/Code/go-api -git init -``` - -Create `go.mod`: -``` -module forge.lthn.ai/core/go-api - -go 1.25.5 - -require github.com/gin-gonic/gin v1.10.0 -``` - -Then run: -```bash -go mod tidy -``` - -**Step 2: Create LICENCE file** - -Copy the EUPL-1.2 licence text. Use the same LICENCE file as other ecosystem repos: -```bash -cp /Users/snider/Code/go-ai/LICENCE /Users/snider/Code/go-api/LICENCE -``` - -**Step 3: Commit scaffold** - -```bash -git add go.mod go.sum LICENCE -git commit -m "chore: scaffold go-api module with Gin dependency" -``` - ---- - -### Task 2: Response Envelope (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/response.go` -- Create: `/Users/snider/Code/go-api/response_test.go` - -**Step 1: Write the failing tests** - -Create `response_test.go`: -```go -package api_test - -import ( - "encoding/json" - "testing" - - api "forge.lthn.ai/core/go-api" -) - -func TestOK_Good(t *testing.T) { - type Payload struct { - Name string `json:"name"` - } - resp := api.OK(Payload{Name: "test"}) - - if !resp.Success { - t.Fatal("expected Success to be true") - } - if resp.Data.Name != "test" { - t.Fatalf("expected Data.Name = test, got %s", resp.Data.Name) - } - if resp.Error != nil { - t.Fatal("expected Error to be nil") - } -} - -func TestFail_Good(t *testing.T) { - resp := api.Fail("not_found", "Resource not found") - - if resp.Success { - t.Fatal("expected Success to be false") - } - if resp.Error == nil { - t.Fatal("expected Error to be non-nil") - } - if resp.Error.Code != "not_found" { - t.Fatalf("expected Code = not_found, got %s", resp.Error.Code) - } - if resp.Error.Message != "Resource not found" { - t.Fatalf("expected Message = Resource not found, got %s", resp.Error.Message) - } -} - -func TestFailWithDetails_Good(t *testing.T) { - details := map[string]string{"field": "email"} - resp := api.FailWithDetails("validation_error", "Invalid input", details) - - if resp.Error.Details == nil { - t.Fatal("expected Details to be non-nil") - } -} - -func TestPaginated_Good(t *testing.T) { - items := []string{"a", "b", "c"} - resp := api.Paginated(items, 1, 10, 42) - - if !resp.Success { - t.Fatal("expected Success to be true") - } - if resp.Meta == nil { - t.Fatal("expected Meta to be non-nil") - } - if resp.Meta.Page != 1 { - t.Fatalf("expected Page = 1, got %d", resp.Meta.Page) - } - if resp.Meta.PerPage != 10 { - t.Fatalf("expected PerPage = 10, got %d", resp.Meta.PerPage) - } - if resp.Meta.Total != 42 { - t.Fatalf("expected Total = 42, got %d", resp.Meta.Total) - } -} - -func TestOK_JSON_Good(t *testing.T) { - resp := api.OK("hello") - data, err := json.Marshal(resp) - if err != nil { - t.Fatalf("marshal failed: %v", err) - } - - var raw map[string]any - if err := json.Unmarshal(data, &raw); err != nil { - t.Fatalf("unmarshal failed: %v", err) - } - - if raw["success"] != true { - t.Fatal("expected success = true in JSON") - } - if raw["data"] != "hello" { - t.Fatalf("expected data = hello, got %v", raw["data"]) - } - // error and meta should be omitted - if _, ok := raw["error"]; ok { - t.Fatal("expected error to be omitted from JSON") - } - if _, ok := raw["meta"]; ok { - t.Fatal("expected meta to be omitted from JSON") - } -} - -func TestFail_JSON_Good(t *testing.T) { - resp := api.Fail("err", "msg") - data, err := json.Marshal(resp) - if err != nil { - t.Fatalf("marshal failed: %v", err) - } - - var raw map[string]any - if err := json.Unmarshal(data, &raw); err != nil { - t.Fatalf("unmarshal failed: %v", err) - } - - if raw["success"] != false { - t.Fatal("expected success = false in JSON") - } - // data should be omitted - if _, ok := raw["data"]; ok { - t.Fatal("expected data to be omitted from JSON") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.OK`, `api.Fail`, etc. not defined. - -**Step 3: Implement response.go** - -Create `response.go`: -```go -// Package api provides a Gin-based REST framework with OpenAPI generation. -// Subsystems implement RouteGroup to register their own endpoints. -package api - -// Response is the standard envelope for all API responses. -type Response[T any] struct { - Success bool `json:"success"` - Data T `json:"data,omitempty"` - Error *Error `json:"error,omitempty"` - Meta *Meta `json:"meta,omitempty"` -} - -// Error describes a failed API request. -type Error struct { - Code string `json:"code"` - Message string `json:"message"` - Details any `json:"details,omitempty"` -} - -// Meta carries pagination and request metadata. -type Meta struct { - RequestID string `json:"request_id,omitempty"` - Duration string `json:"duration,omitempty"` - Page int `json:"page,omitempty"` - PerPage int `json:"per_page,omitempty"` - Total int `json:"total,omitempty"` -} - -// OK returns a successful response wrapping data. -func OK[T any](data T) Response[T] { - return Response[T]{Success: true, Data: data} -} - -// Fail returns an error response with code and message. -func Fail(code, message string) Response[any] { - return Response[any]{ - Success: false, - Error: &Error{Code: code, Message: message}, - } -} - -// FailWithDetails returns an error response with additional detail payload. -func FailWithDetails(code, message string, details any) Response[any] { - return Response[any]{ - Success: false, - Error: &Error{Code: code, Message: message, Details: details}, - } -} - -// Paginated returns a successful response with pagination metadata. -func Paginated[T any](data T, page, perPage, total int) Response[T] { - return Response[T]{ - Success: true, - Data: data, - Meta: &Meta{Page: page, PerPage: perPage, Total: total}, - } -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add response.go response_test.go -git commit -m "feat: add response envelope with OK, Fail, Paginated helpers" -``` - ---- - -### Task 3: RouteGroup Interface - -**Files:** -- Create: `/Users/snider/Code/go-api/group.go` -- Create: `/Users/snider/Code/go-api/group_test.go` - -**Step 1: Write the failing test** - -Create `group_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -// stubGroup is a minimal RouteGroup for testing. -type stubGroup struct{} - -func (s *stubGroup) Name() string { return "stub" } -func (s *stubGroup) BasePath() string { return "/v1/stub" } - -func (s *stubGroup) RegisterRoutes(rg *gin.RouterGroup) { - rg.GET("/ping", func(c *gin.Context) { - c.JSON(200, api.OK("pong")) - }) -} - -// stubStreamGroup implements both RouteGroup and StreamGroup. -type stubStreamGroup struct { - stubGroup -} - -func (s *stubStreamGroup) Channels() []string { - return []string{"stub.events", "stub.updates"} -} - -func TestRouteGroup_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - g := gin.New() - group := &stubGroup{} - - rg := g.Group(group.BasePath()) - group.RegisterRoutes(rg) - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - g.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} - -func TestStreamGroup_Good(t *testing.T) { - group := &stubStreamGroup{} - - // Verify it satisfies StreamGroup - var sg api.StreamGroup = group - channels := sg.Channels() - - if len(channels) != 2 { - t.Fatalf("expected 2 channels, got %d", len(channels)) - } - if channels[0] != "stub.events" { - t.Fatalf("expected stub.events, got %s", channels[0]) - } -} - -func TestRouteGroupName_Good(t *testing.T) { - group := &stubGroup{} - - var rg api.RouteGroup = group - if rg.Name() != "stub" { - t.Fatalf("expected name stub, got %s", rg.Name()) - } - if rg.BasePath() != "/v1/stub" { - t.Fatalf("expected basepath /v1/stub, got %s", rg.BasePath()) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.RouteGroup`, `api.StreamGroup` not defined. - -**Step 3: Implement group.go** - -Create `group.go`: -```go -package api - -import "github.com/gin-gonic/gin" - -// RouteGroup registers API routes onto a Gin router group. -// Subsystems implement this to expose their REST endpoints. -type RouteGroup interface { - // Name returns the route group identifier (e.g. "ml", "rag", "tasks"). - Name() string - // BasePath returns the URL prefix (e.g. "/v1/ml"). - BasePath() string - // RegisterRoutes adds handlers to the provided router group. - RegisterRoutes(rg *gin.RouterGroup) -} - -// StreamGroup optionally declares WebSocket channels a subsystem publishes to. -// Subsystems implementing both RouteGroup and StreamGroup expose both REST -// endpoints and real-time event channels. -type StreamGroup interface { - // Channels returns the WebSocket channel names this group publishes to. - Channels() []string -} -``` - -**Step 4: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: All tests PASS (previous 6 + new 3). - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/go-api -git add group.go group_test.go -git commit -m "feat: add RouteGroup and StreamGroup interfaces" -``` - ---- - -### Task 4: Engine + Options (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/api.go` -- Create: `/Users/snider/Code/go-api/options.go` -- Create: `/Users/snider/Code/go-api/api_test.go` - -**Step 1: Write the failing tests** - -Create `api_test.go`: -```go -package api_test - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" - "time" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestNew_Good(t *testing.T) { - engine, err := api.New() - if err != nil { - t.Fatalf("New() failed: %v", err) - } - if engine == nil { - t.Fatal("expected non-nil engine") - } -} - -func TestNewWithAddr_Good(t *testing.T) { - engine, err := api.New(api.WithAddr(":9090")) - if err != nil { - t.Fatalf("New() failed: %v", err) - } - if engine.Addr() != ":9090" { - t.Fatalf("expected addr :9090, got %s", engine.Addr()) - } -} - -func TestDefaultAddr_Good(t *testing.T) { - engine, _ := api.New() - if engine.Addr() != ":8080" { - t.Fatalf("expected default addr :8080, got %s", engine.Addr()) - } -} - -func TestRegister_Good(t *testing.T) { - engine, _ := api.New() - group := &stubGroup{} - - engine.Register(group) - - if len(engine.Groups()) != 1 { - t.Fatalf("expected 1 group, got %d", len(engine.Groups())) - } - if engine.Groups()[0].Name() != "stub" { - t.Fatalf("expected group name stub, got %s", engine.Groups()[0].Name()) - } -} - -func TestRegisterMultiple_Good(t *testing.T) { - engine, _ := api.New() - engine.Register(&stubGroup{}) - engine.Register(&stubStreamGroup{}) - - if len(engine.Groups()) != 2 { - t.Fatalf("expected 2 groups, got %d", len(engine.Groups())) - } -} - -func TestHandler_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - engine.Register(&stubGroup{}) - - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } - - var resp map[string]any - json.Unmarshal(w.Body.Bytes(), &resp) - if resp["success"] != true { - t.Fatal("expected success = true") - } -} - -func TestHealthEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200, got %d", w.Code) - } -} - -func TestServeAndShutdown_Good(t *testing.T) { - engine, _ := api.New(api.WithAddr(":0")) - engine.Register(&stubGroup{}) - - ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond) - defer cancel() - - errCh := make(chan error, 1) - go func() { - errCh <- engine.Serve(ctx) - }() - - // Wait for context cancellation to trigger shutdown - <-ctx.Done() - - select { - case err := <-errCh: - if err != nil && err != http.ErrServerClosed && err != context.DeadlineExceeded { - t.Fatalf("Serve() returned unexpected error: %v", err) - } - case <-time.After(2 * time.Second): - t.Fatal("Serve() did not return after context cancellation") - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `api.New`, `api.WithAddr`, `api.Engine` not defined. - -**Step 3: Implement options.go** - -Create `options.go`: -```go -package api - -// Option configures the Engine. -type Option func(*Engine) error - -// WithAddr sets the listen address (default ":8080"). -func WithAddr(addr string) Option { - return func(e *Engine) error { - e.addr = addr - return nil - } -} -``` - -**Step 4: Implement api.go** - -Create `api.go`: -```go -package api - -import ( - "context" - "fmt" - "log/slog" - "net/http" - - "github.com/gin-gonic/gin" -) - -// Engine is the central REST API server. -// Register RouteGroups to add endpoints, then call Serve to start. -type Engine struct { - gin *gin.Engine - addr string - groups []RouteGroup - logger *slog.Logger - built bool -} - -// New creates an Engine with the given options. -func New(opts ...Option) (*Engine, error) { - e := &Engine{ - addr: ":8080", - logger: slog.Default(), - } - - for _, opt := range opts { - if err := opt(e); err != nil { - return nil, fmt.Errorf("apply option: %w", err) - } - } - - return e, nil -} - -// Addr returns the configured listen address. -func (e *Engine) Addr() string { - return e.addr -} - -// Groups returns all registered route groups. -func (e *Engine) Groups() []RouteGroup { - return e.groups -} - -// Register adds a RouteGroup to the engine. -// Routes are mounted when Handler() or Serve() is called. -func (e *Engine) Register(group RouteGroup) { - e.groups = append(e.groups, group) - e.built = false -} - -// build constructs the Gin engine with all registered groups. -func (e *Engine) build() { - if e.built && e.gin != nil { - return - } - - e.gin = gin.New() - e.gin.Use(gin.Recovery()) - - // Health endpoint - e.gin.GET("/health", func(c *gin.Context) { - c.JSON(200, OK("healthy")) - }) - - // Mount each route group - for _, group := range e.groups { - rg := e.gin.Group(group.BasePath()) - group.RegisterRoutes(rg) - e.logger.Info("registered route group", "name", group.Name(), "path", group.BasePath()) - } - - e.built = true -} - -// Handler returns the http.Handler for testing or custom server usage. -func (e *Engine) Handler() http.Handler { - e.build() - return e.gin -} - -// Serve starts the HTTP server and blocks until the context is cancelled. -// Performs graceful shutdown on context cancellation. -func (e *Engine) Serve(ctx context.Context) error { - e.build() - - srv := &http.Server{ - Addr: e.addr, - Handler: e.gin, - } - - errCh := make(chan error, 1) - go func() { - if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed { - errCh <- err - } - close(errCh) - }() - - <-ctx.Done() - - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5_000_000_000) // 5s - defer cancel() - - if err := srv.Shutdown(shutdownCtx); err != nil { - return fmt.Errorf("shutdown: %w", err) - } - - if err, ok := <-errCh; ok { - return err - } - - return nil -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add api.go options.go api_test.go -git commit -m "feat: add Engine with Register, Handler, Serve, and graceful shutdown" -``` - ---- - -### Task 5: Middleware (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/middleware.go` -- Create: `/Users/snider/Code/go-api/middleware_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add middleware options - -**Step 1: Write the failing tests** - -Create `middleware_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestBearerAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Request without token → 401 - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 without token, got %d", w.Code) - } - - // Request with correct token → 200 - w = httptest.NewRecorder() - req, _ = http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("Authorization", "Bearer secret-token") - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 with correct token, got %d", w.Code) - } -} - -func TestBearerAuth_Bad(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Wrong token → 401 - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("Authorization", "Bearer wrong-token") - handler.ServeHTTP(w, req) - - if w.Code != 401 { - t.Fatalf("expected 401 with wrong token, got %d", w.Code) - } -} - -func TestHealthBypassesAuth_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithBearerAuth("secret-token")) - handler := engine.Handler() - - // Health endpoint should not require auth - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/health", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for /health without auth, got %d", w.Code) - } -} - -func TestRequestID_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithRequestID()) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - handler.ServeHTTP(w, req) - - rid := w.Header().Get("X-Request-ID") - if rid == "" { - t.Fatal("expected X-Request-ID header to be set") - } -} - -func TestRequestIDPreserved_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithRequestID()) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/v1/stub/ping", nil) - req.Header.Set("X-Request-ID", "my-custom-id") - handler.ServeHTTP(w, req) - - rid := w.Header().Get("X-Request-ID") - if rid != "my-custom-id" { - t.Fatalf("expected X-Request-ID = my-custom-id, got %s", rid) - } -} - -func TestCORS_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithCORS("https://example.com")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Preflight request - w := httptest.NewRecorder() - req, _ := http.NewRequest("OPTIONS", "/v1/stub/ping", nil) - req.Header.Set("Origin", "https://example.com") - req.Header.Set("Access-Control-Request-Method", "POST") - handler.ServeHTTP(w, req) - - origin := w.Header().Get("Access-Control-Allow-Origin") - if origin != "https://example.com" { - t.Fatalf("expected CORS origin https://example.com, got %s", origin) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors — `WithBearerAuth`, `WithRequestID`, `WithCORS` not defined. - -**Step 3: Implement middleware.go** - -Create `middleware.go`: -```go -package api - -import ( - "crypto/rand" - "encoding/hex" - "strings" - - "github.com/gin-gonic/gin" -) - -// bearerAuthMiddleware validates Bearer tokens. -// Skips paths listed in skip (e.g. /health, /swagger). -func bearerAuthMiddleware(token string, skip []string) gin.HandlerFunc { - return func(c *gin.Context) { - path := c.Request.URL.Path - for _, s := range skip { - if strings.HasPrefix(path, s) { - c.Next() - return - } - } - - header := c.GetHeader("Authorization") - if header == "" { - c.JSON(401, Fail("unauthorised", "Missing Authorization header")) - c.Abort() - return - } - - parts := strings.SplitN(header, " ", 2) - if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") || parts[1] != token { - c.JSON(401, Fail("unauthorised", "Invalid bearer token")) - c.Abort() - return - } - - c.Next() - } -} - -// requestIDMiddleware sets X-Request-ID on every response. -// If the client sends one, it is preserved; otherwise a random ID is generated. -func requestIDMiddleware() gin.HandlerFunc { - return func(c *gin.Context) { - rid := c.GetHeader("X-Request-ID") - if rid == "" { - b := make([]byte, 16) - rand.Read(b) - rid = hex.EncodeToString(b) - } - c.Header("X-Request-ID", rid) - c.Set("request_id", rid) - c.Next() - } -} -``` - -**Step 4: Add middleware options to options.go** - -Append to `options.go`: -```go -import "github.com/gin-contrib/cors" - -// WithBearerAuth adds bearer token authentication middleware. -// The /health and /swagger paths are excluded from authentication. -func WithBearerAuth(token string) Option { - return func(e *Engine) error { - e.middlewares = append(e.middlewares, bearerAuthMiddleware(token, []string{"/health", "/swagger"})) - return nil - } -} - -// WithRequestID adds a middleware that sets X-Request-ID on every response. -func WithRequestID() Option { - return func(e *Engine) error { - e.middlewares = append(e.middlewares, requestIDMiddleware()) - return nil - } -} - -// WithCORS configures Cross-Origin Resource Sharing. -// Pass "*" to allow all origins, or specific origins. -func WithCORS(allowOrigins ...string) Option { - return func(e *Engine) error { - config := cors.DefaultConfig() - if len(allowOrigins) == 1 && allowOrigins[0] == "*" { - config.AllowAllOrigins = true - } else { - config.AllowOrigins = allowOrigins - } - config.AllowMethods = []string{"GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"} - config.AllowHeaders = []string{"Authorization", "Content-Type", "X-Request-ID"} - e.middlewares = append(e.middlewares, cors.New(config)) - return nil - } -} -``` - -Update `Engine` struct in `api.go` to include `middlewares []gin.HandlerFunc` field, and apply them in `build()`: -```go -// Add to Engine struct: -middlewares []gin.HandlerFunc - -// In build(), after gin.New() and gin.Recovery(), before health endpoint: -for _, mw := range e.middlewares { - e.gin.Use(mw) -} -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add middleware.go middleware_test.go options.go api.go -git commit -m "feat: add bearer auth, request ID, and CORS middleware" -``` - ---- - -### Task 6: WebSocket Integration (TDD) - -**Files:** -- Create: `/Users/snider/Code/go-api/websocket.go` -- Create: `/Users/snider/Code/go-api/websocket_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add WithWSHub -- Modify: `/Users/snider/Code/go-api/api.go` — mount /ws route - -**Step 1: Write the failing test** - -Create `websocket_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "strings" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" - "github.com/gorilla/websocket" -) - -func TestWSEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithWSHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - upgrader := websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return true }} - conn, err := upgrader.Upgrade(w, r, nil) - if err != nil { - return - } - defer conn.Close() - conn.WriteMessage(websocket.TextMessage, []byte("hello")) - }))) - - srv := httptest.NewServer(engine.Handler()) - defer srv.Close() - - wsURL := "ws" + strings.TrimPrefix(srv.URL, "http") + "/ws" - conn, _, err := websocket.DefaultDialer.Dial(wsURL, nil) - if err != nil { - t.Fatalf("dial failed: %v", err) - } - defer conn.Close() - - _, msg, err := conn.ReadMessage() - if err != nil { - t.Fatalf("read failed: %v", err) - } - if string(msg) != "hello" { - t.Fatalf("expected hello, got %s", string(msg)) - } -} - -func TestNoWSHandler_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - // /ws should 404 when no handler configured - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/ws", nil) - handler.ServeHTTP(w, req) - - if w.Code != 404 { - t.Fatalf("expected 404 without WS handler, got %d", w.Code) - } -} - -func TestChannelListing_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - engine.Register(&stubStreamGroup{}) - - channels := engine.Channels() - if len(channels) != 2 { - t.Fatalf("expected 2 channels, got %d", len(channels)) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors. - -**Step 3: Implement websocket.go + option + engine changes** - -Create `websocket.go`: -```go -package api - -import ( - "net/http" - - "github.com/gin-gonic/gin" -) - -// wrapWSHandler adapts a standard http.Handler to a Gin handler for the /ws route. -func wrapWSHandler(h http.Handler) gin.HandlerFunc { - return func(c *gin.Context) { - h.ServeHTTP(c.Writer, c.Request) - } -} -``` - -Add to `options.go`: -```go -// WithWSHandler registers a WebSocket handler at GET /ws. -// Typically this wraps a go-ws Hub.Handler(). -func WithWSHandler(h http.Handler) Option { - return func(e *Engine) error { - e.wsHandler = h - return nil - } -} -``` - -Add to `Engine` struct in `api.go`: -```go -wsHandler http.Handler -``` - -Add to `build()` after mounting route groups: -```go -// WebSocket endpoint -if e.wsHandler != nil { - e.gin.GET("/ws", wrapWSHandler(e.wsHandler)) -} -``` - -Add `Channels()` method to `Engine`: -```go -// Channels returns all WebSocket channel names from registered StreamGroups. -func (e *Engine) Channels() []string { - var channels []string - for _, g := range e.groups { - if sg, ok := g.(StreamGroup); ok { - channels = append(channels, sg.Channels()...) - } - } - return channels -} -``` - -**Step 4: Run go mod tidy to pick up gorilla/websocket** - -```bash -cd /Users/snider/Code/go-api -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add websocket.go websocket_test.go options.go api.go go.mod go.sum -git commit -m "feat: add WebSocket endpoint and channel listing from StreamGroups" -``` - ---- - -### Task 7: Swagger/OpenAPI Integration - -**Files:** -- Create: `/Users/snider/Code/go-api/swagger.go` -- Create: `/Users/snider/Code/go-api/swagger_test.go` -- Modify: `/Users/snider/Code/go-api/options.go` — add WithSwagger -- Modify: `/Users/snider/Code/go-api/api.go` — mount swagger routes - -**Step 1: Write the failing test** - -Create `swagger_test.go`: -```go -package api_test - -import ( - "net/http" - "net/http/httptest" - "testing" - - api "forge.lthn.ai/core/go-api" - "github.com/gin-gonic/gin" -) - -func TestSwaggerEndpoint_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New(api.WithSwagger("Core API", "REST API for the Lethean ecosystem", "0.1.0")) - engine.Register(&stubGroup{}) - handler := engine.Handler() - - // Swagger JSON endpoint - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/swagger/doc.json", nil) - handler.ServeHTTP(w, req) - - if w.Code != 200 { - t.Fatalf("expected 200 for swagger doc.json, got %d", w.Code) - } - - body := w.Body.String() - if len(body) == 0 { - t.Fatal("expected non-empty swagger doc") - } -} - -func TestSwaggerDisabledByDefault_Good(t *testing.T) { - gin.SetMode(gin.TestMode) - engine, _ := api.New() - handler := engine.Handler() - - w := httptest.NewRecorder() - req, _ := http.NewRequest("GET", "/swagger/doc.json", nil) - handler.ServeHTTP(w, req) - - if w.Code != 404 { - t.Fatalf("expected 404 when swagger disabled, got %d", w.Code) - } -} -``` - -**Step 2: Run tests to verify they fail** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -``` - -Expected: Compilation errors. - -**Step 3: Implement swagger.go + option** - -Create `swagger.go`: -```go -package api - -import ( - "github.com/gin-gonic/gin" - swaggerFiles "github.com/swaggo/files" - ginSwagger "github.com/swaggo/gin-swagger" - "github.com/swaggo/swag" -) - -// swaggerSpec holds a minimal OpenAPI spec for runtime serving. -type swaggerSpec struct { - title string - description string - version string -} - -func (s *swaggerSpec) ReadDoc() string { - // Minimal OpenAPI 3.0 document — swaggo generates the full one at build time. - // This serves as the runtime fallback and base template. - return `{ - "swagger": "2.0", - "info": { - "title": "` + s.title + `", - "description": "` + s.description + `", - "version": "` + s.version + `" - }, - "basePath": "/", - "paths": {} -}` -} - -// registerSwagger mounts the swagger UI and doc.json endpoint. -func registerSwagger(g *gin.Engine, title, description, version string) { - spec := &swaggerSpec{title: title, description: description, version: version} - swag.Register(swag.Name, spec) - - g.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler)) -} -``` - -Add to `options.go`: -```go -// WithSwagger enables the Swagger UI at /swagger/. -func WithSwagger(title, description, version string) Option { - return func(e *Engine) error { - e.swaggerTitle = title - e.swaggerDesc = description - e.swaggerVersion = version - e.swaggerEnabled = true - return nil - } -} -``` - -Add fields to `Engine` struct: -```go -swaggerEnabled bool -swaggerTitle string -swaggerDesc string -swaggerVersion string -``` - -Add to `build()` after WebSocket: -```go -// Swagger UI -if e.swaggerEnabled { - registerSwagger(e.gin, e.swaggerTitle, e.swaggerDesc, e.swaggerVersion) -} -``` - -**Step 4: Run go mod tidy** - -```bash -cd /Users/snider/Code/go-api -go get github.com/swaggo/gin-swagger github.com/swaggo/files github.com/swaggo/swag -go mod tidy -``` - -**Step 5: Run tests to verify they pass** - -```bash -cd /Users/snider/Code/go-api -go test ./... -v -count=1 -``` - -Expected: All tests PASS. - -**Step 6: Commit** - -```bash -cd /Users/snider/Code/go-api -git add swagger.go swagger_test.go options.go api.go go.mod go.sum -git commit -m "feat: add Swagger UI endpoint with runtime spec serving" -``` - ---- - -### Task 8: CLAUDE.md + README.md - -**Files:** -- Create: `/Users/snider/Code/go-api/CLAUDE.md` -- Create: `/Users/snider/Code/go-api/README.md` - -**Step 1: Write CLAUDE.md** - -```markdown -# CLAUDE.md - -This file provides guidance to Claude Code when working with the go-api repository. - -## Project Overview - -**go-api** is the REST framework for the Lethean Go ecosystem. It provides a Gin-based HTTP engine with middleware, response envelopes, WebSocket integration, and OpenAPI generation. Subsystems implement the `RouteGroup` interface to register their own endpoints. - -- **Module path**: `forge.lthn.ai/core/go-api` -- **Language**: Go 1.25 -- **Licence**: EUPL-1.2 - -## Build & Test Commands - -```bash -go test ./... # Run all tests -go test -run TestName ./... # Run a single test -go test -v -race ./... # Verbose with race detector -go build ./... # Build (library — no main package) -go vet ./... # Vet -``` - -## Coding Standards - -- **UK English** in comments and user-facing strings (colour, organisation, unauthorised) -- **Conventional commits**: `type(scope): description` -- **Co-Author**: `Co-Authored-By: Virgil ` -- **Error handling**: Return wrapped errors with context, never panic -- **Test naming**: `_Good` (happy path), `_Bad` (expected errors), `_Ugly` (panics/edge cases) -- **Licence**: EUPL-1.2 -``` - -**Step 2: Write README.md** - -Brief README with quick start and links to design doc. - -**Step 3: Commit** - -```bash -cd /Users/snider/Code/go-api -git add CLAUDE.md README.md -git commit -m "docs: add CLAUDE.md and README.md" -``` - ---- - -### Task 9: Create Forge Repo + Push - -**Step 1: Create repo on Forge** - -```bash -curl -s -X POST "https://forge.lthn.ai/api/v1/orgs/core/repos" \ - -H "Authorization: token 375068d101922dd1cf269e8b8cb77a0f99d1b486" \ - -H "Content-Type: application/json" \ - -d '{"name":"go-api","description":"REST framework + OpenAPI SDK generation for the Lethean Go ecosystem","default_branch":"main","auto_init":false,"license":"EUPL-1.2"}' -``` - -**Step 2: Add remote and push** - -```bash -cd /Users/snider/Code/go-api -git remote add forge ssh://git@forge.lthn.ai:2223/core/go-api.git -git branch -M main -git push -u forge main -``` - -**Step 3: Verify on Forge** - -```bash -curl -s "https://forge.lthn.ai/api/v1/repos/core/go-api" \ - -H "Authorization: token 375068d101922dd1cf269e8b8cb77a0f99d1b486" | jq .name -``` - -Expected: `"go-api"` - ---- - -### Task 10: Integration Test — First Subsystem (go-ml/api) - -This task validates the framework by building the first real subsystem integration. It lives in go-ml, not go-api. - -**Files:** -- Create: `/Users/snider/Code/go-ml/api/routes.go` -- Create: `/Users/snider/Code/go-ml/api/routes_test.go` - -**Step 1: Write the failing test in go-ml** - -Create `api/routes_test.go` in go-ml that: -1. Creates a `Routes` with a mock `ml.Service` -2. Registers it on an `api.Engine` -3. Sends `POST /v1/ml/backends` and asserts a 200 response with the response envelope - -**Step 2: Implement api/routes.go** - -Implement `Routes` struct that wraps `*ml.Service` and exposes: -- `POST /v1/ml/generate` -- `POST /v1/ml/score` -- `GET /v1/ml/backends` -- `GET /v1/ml/status` - -Each handler uses `c.ShouldBindJSON()` for input and `api.OK()` / `api.Fail()` for responses. - -**Step 3: Run tests** - -```bash -cd /Users/snider/Code/go-ml -go test ./api/... -v -``` - -**Step 4: Commit in go-ml** - -```bash -cd /Users/snider/Code/go-ml -git add api/ -git commit -m "feat(api): add REST route group for ML endpoints via go-api" -``` - ---- - -## Dependency Summary - -``` -Task 1 (scaffold) → Task 2 (response) → Task 3 (group) → Task 4 (engine) - → Task 5 (middleware) → Task 6 (websocket) → Task 7 (swagger) - → Task 8 (docs) → Task 9 (forge) → Task 10 (integration) -``` - -All tasks are sequential — each builds on the previous. - -## Estimated Timeline - -- Tasks 1-7: Core go-api package (~820 LOC) -- Task 8: Documentation -- Task 9: Forge deployment -- Task 10: First subsystem integration proof diff --git a/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md b/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md deleted file mode 100644 index eaf886f..0000000 --- a/docs/plans/completed/2026-02-21-cli-meta-package-design-original.md +++ /dev/null @@ -1,128 +0,0 @@ -# CLI Meta-Package Restructure — Design - -**Goal:** Transform `core/cli` from a 35K LOC monolith into a thin assembly repo that ships variant binaries. Domain repos own their commands. `go/pkg/cli` is the only import any domain package needs for CLI concerns. - -**Architecture:** Commands register as framework services via `cli.WithCommands()`, passed to `cli.Main()`. Command code lives in the domain repos that own the business logic. The cli repo is a thin `main.go` that wires them together. - -**Tech Stack:** go/pkg/cli (wraps cobra + charmbracelet), Core framework lifecycle, Taskfile - ---- - -## 1. CLI SDK — The Single Import - -`forge.lthn.ai/core/go/pkg/cli` is the **only** import domain packages use for CLI concerns. It wraps cobra, charmbracelet, and stdlib behind a stable API. If the underlying libraries change, only `go/pkg/cli` is touched — every domain repo is insulated. - -### Already done - -- **Cobra:** `Command` type alias, `NewCommand()`, `NewGroup()`, `NewRun()`, flag helpers (`StringFlag`, `BoolFlag`, `IntFlag`, `StringSliceFlag`), arg validators -- **Output:** `Success()`, `Error()`, `Warn()`, `Info()`, `Table`, `Section()`, `Label()`, `Task()`, `Hint()` -- **Prompts:** `Confirm()`, `Question()`, `Choose()`, `ChooseMulti()` with grammar-based action variants -- **Styles:** 17 pre-built styles, `AnsiStyle` builder, Tailwind colour constants (47 hex values) -- **Glyphs:** `:check:`, `:cross:`, `:warn:` etc. with Unicode/Emoji/ASCII themes -- **Layout:** HLCRF composite renderer (Header/Left/Content/Right/Footer) -- **Errors:** `Wrap()`, `WrapVerb()`, `ExitError`, `Is()`, `As()` -- **Logging:** `LogDebug()`, `LogInfo()`, `LogWarn()`, `LogError()`, `LogSecurity()` -- **TUI primitives:** `Spinner`, `ProgressBar`, `InteractiveList`, `TextInput`, `Viewport`, `RunTUI` -- **Command registration:** `WithCommands(name, fn)` — registers commands as framework services - -### Stubbed for later (interface exists, returns simple fallback) - -- `Form(fields []FormField) (map[string]string, error)` — multi-field form (backed by huh later) -- `FilePicker(opts ...FilePickerOption) (string, error)` — file browser -- `Tabs(items []TabItem) error` — tabbed content panes - -### Rule - -Domain packages import `forge.lthn.ai/core/go/pkg/cli` and **nothing else** for CLI concerns. No `cobra`, no `lipgloss`, no `bubbletea`. - ---- - -## 2. Command Registration — Framework Lifecycle - -Commands register through the Core framework's service lifecycle, not through global state or `init()` functions. - -### The contract - -Each domain repo exports an `Add*Commands(root *cli.Command)` function. The CLI binary wires it in via `cli.WithCommands()`: - -```go -// go-ai/cmd/daemon/cmd.go -package daemon - -import "forge.lthn.ai/core/go/pkg/cli" - -// AddDaemonCommand adds the 'daemon' command group to the root. -func AddDaemonCommand(root *cli.Command) { - daemonCmd := cli.NewGroup("daemon", "Manage the core daemon", "") - root.AddCommand(daemonCmd) - // subcommands... -} -``` - -No `init()`. No blank imports. No `cli.RegisterCommands()`. - -### How it works - -`cli.WithCommands(name, fn)` wraps the registration function as a framework service implementing `Startable`. During `Core.ServiceStartup()`, the service's `OnStartup()` casts `Core.App` to `*cobra.Command` and calls the registration function. Core services (i18n, log, workspace) start first since they're registered before command services. - -```go -// cli/main.go -func main() { - cli.Main( - cli.WithCommands("config", config.AddConfigCommands), - cli.WithCommands("doctor", doctor.AddDoctorCommands), - // ... - ) -} -``` - -### Migration status (completed) - -| Source | Destination | Status | -|--------|-------------|--------| -| `cmd/dev, setup, qa, docs, gitcmd, monitor` | `go-devops/cmd/` | Done | -| `cmd/lab` | `go-ai/cmd/` | Done | -| `cmd/workspace` | `go-agentic/cmd/` | Done | -| `cmd/go` | `core/go/cmd/gocmd` | Done | -| `cmd/vanity-import, community` | `go-devops/cmd/` | Done | -| `cmd/updater` | `go-update` | Done (own repo) | -| `cmd/daemon, mcpcmd, security` | `go-ai/cmd/` | Done | -| `cmd/crypt` | `go-crypt/cmd/` | Done | -| `cmd/rag` | `go-rag/cmd/` | Done | -| `cmd/unifi` | `go-netops/cmd/` | Done | -| `cmd/api` | `go-api/cmd/` | Done | -| `cmd/collect, forge, gitea` | `go-scm/cmd/` | Done | -| `cmd/deploy, prod, vm` | `go-devops/cmd/` | Done | - -### Stays in cli/ (meta/framework commands) - -`config`, `doctor`, `help`, `module`, `pkgcmd`, `plugin`, `session` - ---- - -## 3. Variant Binaries (future) - -The cli/ repo can produce variant binaries by creating multiple `main.go` files that wire different sets of commands. - -``` -cli/ -├── main.go # Current — meta commands only -├── cmd/core-full/main.go # Full CLI — all ecosystem commands -├── cmd/core-ci/main.go # CI agent dispatch + SCM -├── cmd/core-mlx/main.go # ML inference subprocess -└── cmd/core-ops/main.go # DevOps + infra management -``` - -Each variant calls `cli.Main()` with its specific `cli.WithCommands()` set. No blank imports needed. - -### Why variants matter - -- `core-mlx` ships to the homelab as a ~10MB binary, not 50MB with devops/forge/netops -- `core-ci` deploys to agent machines without ML or CGO dependencies -- Adding a new variant = one new `main.go` with the right `WithCommands` calls - ---- - -## 4. Current State - -cli/ has 7 meta packages, one `main.go`, and zero business logic. Everything else lives in the domain repos that own it. Total cli/ LOC is ~2K. diff --git a/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md b/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md deleted file mode 100644 index c2efef1..0000000 --- a/docs/plans/completed/2026-02-21-cli-sdk-expansion-plan-original.md +++ /dev/null @@ -1,1724 +0,0 @@ -# CLI SDK Expansion (Phase 0) Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Extend `go/pkg/cli` with charmbracelet TUI primitives (Spinner, ProgressBar, List, TextInput, Viewport) so domain repos never import anything but `forge.lthn.ai/core/go/pkg/cli` for CLI concerns. - -**Architecture:** Each TUI primitive gets its own file in `pkg/cli/`. Charmbracelet libraries (bubbletea, bubbles, lipgloss) are imported only inside `pkg/cli/` — the public API uses our own types. Stubs for future features (Form, FilePicker, Tabs) define the interface but fall back to simple bufio implementations until charm backends are wired in later. - -**Tech Stack:** `charmbracelet/bubbletea` (app loop), `charmbracelet/bubbles` (spinner, progress, list, textinput, viewport), `charmbracelet/lipgloss` (styling — replaces our ANSI builder long-term) - ---- - -## Context - -`go/pkg/cli` currently provides: -- Cobra wrappers: `Command`, `NewCommand()`, `NewGroup()`, flag helpers -- Output: `Success()`, `Error()`, `Table`, `Section()`, `Label()` -- Prompts: `Confirm()`, `Question()`, `Choose()`, `ChooseMulti()` (all bufio-based) -- Styles: `AnsiStyle` builder with 17 pre-built styles, 47 Tailwind colour constants -- Glyphs: `:check:`, `:cross:` etc. with theme switching -- Layout: HLCRF composite renderer - -Zero charmbracelet dependencies exist today. All styling is pure ANSI escape codes. - -The 34 files in `cli/cmd/*` that import `github.com/spf13/cobra` directly need `cli.*` equivalents. This plan does NOT migrate those files — it builds the SDK surface they'll need. Migration happens in Phase 1+. - -## Critical Files - -All changes are in `/Users/snider/Code/host-uk/core/pkg/cli/`: - -- `spinner.go` + `spinner_test.go` — Async spinner -- `progress.go` + `progress_test.go` — Progress bar -- `list.go` + `list_test.go` — Interactive scrollable list -- `textinput.go` + `textinput_test.go` — Styled text input -- `viewport.go` + `viewport_test.go` — Scrollable content pane -- `tui.go` + `tui_test.go` — RunTUI escape hatch + Model interface -- `stubs.go` + `stubs_test.go` — Form, FilePicker, Tabs interfaces (simple fallback) - ---- - -### Task 1: Add charmbracelet dependencies - -**Files:** -- Modify: `/Users/snider/Code/host-uk/core/go.mod` - -**Step 1: Add bubbletea, bubbles, and lipgloss** - -Run: -```bash -cd /Users/snider/Code/host-uk/core && go get github.com/charmbracelet/bubbletea/v2@latest github.com/charmbracelet/bubbles/v2@latest github.com/charmbracelet/lipgloss/v2@latest -``` - -**Step 2: Verify module resolves** - -Run: `cd /Users/snider/Code/host-uk/core && go mod tidy` -Expected: Clean, no errors. - -**Step 3: Verify existing tests still pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./pkg/cli/...` -Expected: All existing tests pass (no behaviour changed). - -**Step 4: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add go.mod go.sum && git commit -m "chore(cli): add charmbracelet dependencies (bubbletea, bubbles, lipgloss) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: Spinner - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/spinner.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/spinner_test.go` - -A non-blocking spinner that runs in a goroutine. The caller gets a handle to update the message, mark it done, or mark it failed. Uses `bubbles/spinner` internally. - -**Step 1: Write the tests** - -```go -// spinner_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestSpinner_Good_CreateAndStop(t *testing.T) { - s := NewSpinner("Loading...") - require.NotNil(t, s) - assert.Equal(t, "Loading...", s.Message()) - s.Stop() -} - -func TestSpinner_Good_UpdateMessage(t *testing.T) { - s := NewSpinner("Step 1") - s.Update("Step 2") - assert.Equal(t, "Step 2", s.Message()) - s.Stop() -} - -func TestSpinner_Good_Done(t *testing.T) { - s := NewSpinner("Building") - s.Done("Build complete") - // After Done, spinner is stopped — calling Stop again is safe - s.Stop() -} - -func TestSpinner_Good_Fail(t *testing.T) { - s := NewSpinner("Checking") - s.Fail("Check failed") - s.Stop() -} - -func TestSpinner_Good_DoubleStop(t *testing.T) { - s := NewSpinner("Loading") - s.Stop() - s.Stop() // Should not panic -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestSpinner ./pkg/cli/...` -Expected: FAIL — `NewSpinner` undefined. - -**Step 3: Write the implementation** - -```go -// spinner.go -package cli - -import ( - "fmt" - "sync" - "time" -) - -// SpinnerHandle controls a running spinner. -type SpinnerHandle struct { - mu sync.Mutex - message string - done bool - ticker *time.Ticker - stopCh chan struct{} -} - -// NewSpinner starts an async spinner with the given message. -// Call Stop(), Done(), or Fail() to stop it. -func NewSpinner(message string) *SpinnerHandle { - s := &SpinnerHandle{ - message: message, - ticker: time.NewTicker(100 * time.Millisecond), - stopCh: make(chan struct{}), - } - - frames := []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"} - if !ColorEnabled() { - frames = []string{"|", "/", "-", "\\"} - } - - go func() { - i := 0 - for { - select { - case <-s.stopCh: - return - case <-s.ticker.C: - s.mu.Lock() - if !s.done { - fmt.Printf("\033[2K\r%s %s", DimStyle.Render(frames[i%len(frames)]), s.message) - } - s.mu.Unlock() - i++ - } - } - }() - - return s -} - -// Message returns the current spinner message. -func (s *SpinnerHandle) Message() string { - s.mu.Lock() - defer s.mu.Unlock() - return s.message -} - -// Update changes the spinner message. -func (s *SpinnerHandle) Update(message string) { - s.mu.Lock() - defer s.mu.Unlock() - s.message = message -} - -// Stop stops the spinner silently (clears the line). -func (s *SpinnerHandle) Stop() { - s.mu.Lock() - defer s.mu.Unlock() - if s.done { - return - } - s.done = true - s.ticker.Stop() - close(s.stopCh) - fmt.Print("\033[2K\r") -} - -// Done stops the spinner with a success message. -func (s *SpinnerHandle) Done(message string) { - s.mu.Lock() - alreadyDone := s.done - s.done = true - s.mu.Unlock() - - if alreadyDone { - return - } - s.ticker.Stop() - close(s.stopCh) - fmt.Printf("\033[2K\r%s\n", SuccessStyle.Render(Glyph(":check:")+" "+message)) -} - -// Fail stops the spinner with an error message. -func (s *SpinnerHandle) Fail(message string) { - s.mu.Lock() - alreadyDone := s.done - s.done = true - s.mu.Unlock() - - if alreadyDone { - return - } - s.ticker.Stop() - close(s.stopCh) - fmt.Printf("\033[2K\r%s\n", ErrorStyle.Render(Glyph(":cross:")+" "+message)) -} -``` - -Note: This initial implementation uses a goroutine + ticker rather than bubbletea, keeping it simple and non-blocking. The bubbletea spinner can replace the internals later without changing the public API. - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestSpinner ./pkg/cli/... -v` -Expected: All 5 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/spinner.go pkg/cli/spinner_test.go && git commit -m "feat(cli): add Spinner with async handle (Update, Done, Fail) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: ProgressBar - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/progressbar.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/progressbar_test.go` - -A progress bar that renders inline. Shows percentage, bar, and optional message. - -**Step 1: Write the tests** - -```go -// progressbar_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestProgressBar_Good_Create(t *testing.T) { - pb := NewProgressBar(100) - require.NotNil(t, pb) - assert.Equal(t, 0, pb.Current()) - assert.Equal(t, 100, pb.Total()) -} - -func TestProgressBar_Good_Increment(t *testing.T) { - pb := NewProgressBar(10) - pb.Increment() - assert.Equal(t, 1, pb.Current()) - pb.Increment() - assert.Equal(t, 2, pb.Current()) -} - -func TestProgressBar_Good_SetMessage(t *testing.T) { - pb := NewProgressBar(10) - pb.SetMessage("Processing file.go") - assert.Equal(t, "Processing file.go", pb.message) -} - -func TestProgressBar_Good_Set(t *testing.T) { - pb := NewProgressBar(100) - pb.Set(50) - assert.Equal(t, 50, pb.Current()) -} - -func TestProgressBar_Good_Done(t *testing.T) { - pb := NewProgressBar(5) - for i := 0; i < 5; i++ { - pb.Increment() - } - pb.Done() - // After Done, Current == Total - assert.Equal(t, 5, pb.Current()) -} - -func TestProgressBar_Bad_ExceedsTotal(t *testing.T) { - pb := NewProgressBar(2) - pb.Increment() - pb.Increment() - pb.Increment() // Should clamp to total - assert.Equal(t, 2, pb.Current()) -} - -func TestProgressBar_Good_Render(t *testing.T) { - pb := NewProgressBar(10) - pb.Set(5) - rendered := pb.String() - assert.Contains(t, rendered, "50%") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestProgressBar ./pkg/cli/... -v` -Expected: FAIL — `NewProgressBar` undefined. - -**Step 3: Write the implementation** - -```go -// progressbar.go -package cli - -import ( - "fmt" - "strings" - "sync" -) - -// ProgressHandle controls a progress bar. -type ProgressHandle struct { - mu sync.Mutex - current int - total int - message string - width int -} - -// NewProgressBar creates a new progress bar with the given total. -func NewProgressBar(total int) *ProgressHandle { - return &ProgressHandle{ - total: total, - width: 30, - } -} - -// Current returns the current progress value. -func (p *ProgressHandle) Current() int { - p.mu.Lock() - defer p.mu.Unlock() - return p.current -} - -// Total returns the total value. -func (p *ProgressHandle) Total() int { - return p.total -} - -// Increment advances the progress by 1. -func (p *ProgressHandle) Increment() { - p.mu.Lock() - defer p.mu.Unlock() - if p.current < p.total { - p.current++ - } - p.render() -} - -// Set sets the progress to a specific value. -func (p *ProgressHandle) Set(n int) { - p.mu.Lock() - defer p.mu.Unlock() - if n > p.total { - n = p.total - } - if n < 0 { - n = 0 - } - p.current = n - p.render() -} - -// SetMessage sets the message displayed alongside the bar. -func (p *ProgressHandle) SetMessage(msg string) { - p.mu.Lock() - defer p.mu.Unlock() - p.message = msg - p.render() -} - -// Done completes the progress bar and moves to a new line. -func (p *ProgressHandle) Done() { - p.mu.Lock() - defer p.mu.Unlock() - p.current = p.total - p.render() - fmt.Println() -} - -// String returns the rendered progress bar without ANSI cursor control. -func (p *ProgressHandle) String() string { - pct := 0 - if p.total > 0 { - pct = (p.current * 100) / p.total - } - - filled := (p.width * p.current) / p.total - if filled > p.width { - filled = p.width - } - empty := p.width - filled - - bar := "[" + strings.Repeat("█", filled) + strings.Repeat("░", empty) + "]" - - if p.message != "" { - return fmt.Sprintf("%s %3d%% %s", bar, pct, p.message) - } - return fmt.Sprintf("%s %3d%%", bar, pct) -} - -// render outputs the progress bar, overwriting the current line. -func (p *ProgressHandle) render() { - fmt.Printf("\033[2K\r%s", p.String()) -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestProgressBar ./pkg/cli/... -v` -Expected: All 7 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/progressbar.go pkg/cli/progressbar_test.go && git commit -m "feat(cli): add ProgressBar with Increment, Set, SetMessage, Done - -Co-Authored-By: Virgil " -``` - ---- - -### Task 4: TUI runner (RunTUI + Model interface) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/tui.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/tui_test.go` - -The escape hatch for complex interactive UIs. Wraps `bubbletea.Program` behind our own `Model` interface so domain packages never import bubbletea directly. - -**Step 1: Write the tests** - -```go -// tui_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -// testModel is a minimal Model that quits immediately. -type testModel struct { - initCalled bool - updateCalled bool - viewCalled bool -} - -func (m *testModel) Init() Cmd { - m.initCalled = true - return Quit -} - -func (m *testModel) Update(msg Msg) (Model, Cmd) { - m.updateCalled = true - return m, nil -} - -func (m *testModel) View() string { - m.viewCalled = true - return "test view" -} - -func TestModel_Good_InterfaceSatisfied(t *testing.T) { - var m Model = &testModel{} - assert.NotNil(t, m) -} - -func TestQuitCmd_Good_ReturnsQuitMsg(t *testing.T) { - cmd := Quit - assert.NotNil(t, cmd) -} - -func TestKeyMsg_Good_String(t *testing.T) { - k := KeyMsg{Type: KeyEnter} - assert.Equal(t, KeyEnter, k.Type) -} - -func TestKeyTypes_Good_Constants(t *testing.T) { - // Verify key type constants exist - assert.NotEmpty(t, string(KeyEnter)) - assert.NotEmpty(t, string(KeyEsc)) - assert.NotEmpty(t, string(KeyCtrlC)) - assert.NotEmpty(t, string(KeyUp)) - assert.NotEmpty(t, string(KeyDown)) - assert.NotEmpty(t, string(KeyTab)) - assert.NotEmpty(t, string(KeyBackspace)) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestModel ./pkg/cli/... -v && go test -run TestQuit ./pkg/cli/... -v && go test -run TestKey ./pkg/cli/... -v` -Expected: FAIL — types undefined. - -**Step 3: Write the implementation** - -```go -// tui.go -package cli - -import ( - tea "github.com/charmbracelet/bubbletea/v2" -) - -// Model is the interface for interactive TUI applications. -// It mirrors bubbletea's Model but uses our own types so domain -// packages never import bubbletea directly. -type Model interface { - // Init returns an initial command to run. - Init() Cmd - - // Update handles a message and returns the updated model and command. - Update(msg Msg) (Model, Cmd) - - // View returns the string representation of the UI. - View() string -} - -// Msg is a message passed to Update. Can be any type. -type Msg = tea.Msg - -// Cmd is a function that returns a message. Nil means no command. -type Cmd = tea.Cmd - -// Quit is a command that tells the TUI to exit. -var Quit = tea.Quit - -// KeyMsg represents a key press event. -type KeyMsg = tea.KeyMsg - -// KeyType represents the type of key pressed. -type KeyType = tea.KeyType - -// Key type constants. -const ( - KeyEnter KeyType = tea.KeyEnter - KeyEsc KeyType = tea.KeyEscape - KeyCtrlC KeyType = tea.KeyCtrlC - KeyUp KeyType = tea.KeyUp - KeyDown KeyType = tea.KeyDown - KeyLeft KeyType = tea.KeyLeft - KeyRight KeyType = tea.KeyRight - KeyTab KeyType = tea.KeyTab - KeyBackspace KeyType = tea.KeyBackspace - KeySpace KeyType = tea.KeySpace - KeyHome KeyType = tea.KeyHome - KeyEnd KeyType = tea.KeyEnd - KeyPgUp KeyType = tea.KeyPgUp - KeyPgDown KeyType = tea.KeyPgDown - KeyDelete KeyType = tea.KeyDelete -) - -// adapter wraps our Model interface into a bubbletea.Model. -type adapter struct { - inner Model -} - -func (a adapter) Init() tea.Cmd { - return a.inner.Init() -} - -func (a adapter) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - m, cmd := a.inner.Update(msg) - return adapter{inner: m}, cmd -} - -func (a adapter) View() string { - return a.inner.View() -} - -// RunTUI runs an interactive TUI application using the provided Model. -// This is the escape hatch for complex interactive UIs that need the -// full bubbletea event loop. For simple spinners, progress bars, and -// lists, use the dedicated helpers instead. -// -// err := cli.RunTUI(&myModel{items: items}) -func RunTUI(m Model) error { - p := tea.NewProgram(adapter{inner: m}) - _, err := p.Run() - return err -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestModel|TestQuit|TestKey" ./pkg/cli/... -v` -Expected: All 4 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/tui.go pkg/cli/tui_test.go && git commit -m "feat(cli): add RunTUI escape hatch with Model/Msg/Cmd/KeyMsg types - -Wraps bubbletea behind our own interface so domain packages -never import charmbracelet directly. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 5: Interactive List - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/list.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/list_test.go` - -An interactive scrollable list for terminal selection. Uses our `RunTUI` internally. Falls back to numbered `Select()` when stdin is not a terminal. - -**Step 1: Write the tests** - -```go -// list_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestListModel_Good_Create(t *testing.T) { - items := []string{"alpha", "beta", "gamma"} - m := newListModel(items, "Pick one:") - assert.Equal(t, 3, len(m.items)) - assert.Equal(t, 0, m.cursor) - assert.Equal(t, "Pick one:", m.title) -} - -func TestListModel_Good_MoveDown(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - assert.Equal(t, 1, m.cursor) - m.moveDown() - assert.Equal(t, 2, m.cursor) -} - -func TestListModel_Good_MoveUp(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - m.moveDown() - m.moveUp() - assert.Equal(t, 1, m.cursor) -} - -func TestListModel_Good_WrapAround(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveUp() // Should wrap to bottom - assert.Equal(t, 2, m.cursor) -} - -func TestListModel_Good_View(t *testing.T) { - m := newListModel([]string{"alpha", "beta"}, "Choose:") - view := m.View() - assert.Contains(t, view, "Choose:") - assert.Contains(t, view, "alpha") - assert.Contains(t, view, "beta") -} - -func TestListModel_Good_Selected(t *testing.T) { - m := newListModel([]string{"a", "b", "c"}, "") - m.moveDown() - m.selected = true - assert.Equal(t, "b", m.items[m.cursor]) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestListModel ./pkg/cli/... -v` -Expected: FAIL — `newListModel` undefined. - -**Step 3: Write the implementation** - -```go -// list.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// listModel is the internal bubbletea model for interactive list selection. -type listModel struct { - items []string - cursor int - title string - selected bool - quitted bool -} - -func newListModel(items []string, title string) *listModel { - return &listModel{ - items: items, - title: title, - } -} - -func (m *listModel) moveDown() { - m.cursor++ - if m.cursor >= len(m.items) { - m.cursor = 0 - } -} - -func (m *listModel) moveUp() { - m.cursor-- - if m.cursor < 0 { - m.cursor = len(m.items) - 1 - } -} - -func (m *listModel) Init() tea.Cmd { - return nil -} - -func (m *listModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyUp, tea.KeyShiftTab: - m.moveUp() - case tea.KeyDown, tea.KeyTab: - m.moveDown() - case tea.KeyEnter: - m.selected = true - return m, tea.Quit - case tea.KeyEscape, tea.KeyCtrlC: - m.quitted = true - return m, tea.Quit - default: - // Handle j/k vim-style navigation - if msg.String() == "j" { - m.moveDown() - } else if msg.String() == "k" { - m.moveUp() - } - } - } - return m, nil -} - -func (m *listModel) View() string { - var sb strings.Builder - - if m.title != "" { - sb.WriteString(BoldStyle.Render(m.title) + "\n\n") - } - - for i, item := range m.items { - cursor := " " - style := DimStyle - if i == m.cursor { - cursor = AccentStyle.Render(Glyph(":pointer:")) + " " - style = BoldStyle - } - sb.WriteString(fmt.Sprintf("%s%s\n", cursor, style.Render(item))) - } - - sb.WriteString("\n" + DimStyle.Render("↑/↓ navigate • enter select • esc cancel")) - - return sb.String() -} - -// ListOption configures List behaviour. -type ListOption func(*listConfig) - -type listConfig struct { - height int -} - -// WithHeight sets the visible height of the list (number of items shown). -func WithHeight(n int) ListOption { - return func(c *listConfig) { - c.height = n - } -} - -// InteractiveList presents an interactive scrollable list and returns the -// selected item's index and value. Returns -1 and empty string if cancelled. -// -// Falls back to numbered Select() when stdin is not a terminal (e.g. piped input). -// -// idx, value := cli.InteractiveList("Pick a repo:", repos) -func InteractiveList(title string, items []string, opts ...ListOption) (int, string) { - if len(items) == 0 { - return -1, "" - } - - // Fall back to simple Select if not a terminal - if !term.IsTerminal(int(StdinFd())) { - result, err := Select(title, items) - if err != nil { - return -1, "" - } - for i, item := range items { - if item == result { - return i, result - } - } - return -1, "" - } - - m := newListModel(items, title) - p := tea.NewProgram(m) - finalModel, err := p.Run() - if err != nil { - return -1, "" - } - - final := finalModel.(*listModel) - if final.quitted || !final.selected { - return -1, "" - } - return final.cursor, final.items[final.cursor] -} - -// StdinFd returns the file descriptor for stdin. -// Extracted for testing. -func StdinFd() uintptr { - return uintptr(0) // stdin -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestListModel ./pkg/cli/... -v` -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/list.go pkg/cli/list_test.go && git commit -m "feat(cli): add InteractiveList with keyboard navigation and terminal fallback - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: TextInput - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/textinput.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/textinput_test.go` - -A styled single-line text input with placeholder, validation, and optional masking (for passwords). Falls back to `Question()` when stdin is not a terminal. - -**Step 1: Write the tests** - -```go -// textinput_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestTextInputModel_Good_Create(t *testing.T) { - m := newTextInputModel("Enter name:", "") - assert.Equal(t, "Enter name:", m.title) - assert.Equal(t, "", m.value) -} - -func TestTextInputModel_Good_WithPlaceholder(t *testing.T) { - m := newTextInputModel("Name:", "John") - assert.Equal(t, "John", m.placeholder) -} - -func TestTextInputModel_Good_TypeCharacters(t *testing.T) { - m := newTextInputModel("Name:", "") - m.insertChar('H') - m.insertChar('i') - assert.Equal(t, "Hi", m.value) -} - -func TestTextInputModel_Good_Backspace(t *testing.T) { - m := newTextInputModel("Name:", "") - m.insertChar('A') - m.insertChar('B') - m.backspace() - assert.Equal(t, "A", m.value) -} - -func TestTextInputModel_Good_BackspaceEmpty(t *testing.T) { - m := newTextInputModel("Name:", "") - m.backspace() // Should not panic - assert.Equal(t, "", m.value) -} - -func TestTextInputModel_Good_Masked(t *testing.T) { - m := newTextInputModel("Password:", "") - m.masked = true - m.insertChar('s') - m.insertChar('e') - m.insertChar('c') - assert.Equal(t, "sec", m.value) // Internal value is real - view := m.View() - assert.NotContains(t, view, "sec") // Display is masked - assert.Contains(t, view, "***") -} - -func TestTextInputModel_Good_View(t *testing.T) { - m := newTextInputModel("Enter:", "") - m.insertChar('X') - view := m.View() - assert.Contains(t, view, "Enter:") - assert.Contains(t, view, "X") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestTextInputModel ./pkg/cli/... -v` -Expected: FAIL — `newTextInputModel` undefined. - -**Step 3: Write the implementation** - -```go -// textinput.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// textInputModel is the internal bubbletea model for text input. -type textInputModel struct { - title string - placeholder string - value string - masked bool - submitted bool - cancelled bool - cursorPos int - validator func(string) error - err error -} - -func newTextInputModel(title, placeholder string) *textInputModel { - return &textInputModel{ - title: title, - placeholder: placeholder, - } -} - -func (m *textInputModel) insertChar(ch rune) { - m.value = m.value[:m.cursorPos] + string(ch) + m.value[m.cursorPos:] - m.cursorPos++ -} - -func (m *textInputModel) backspace() { - if m.cursorPos > 0 { - m.value = m.value[:m.cursorPos-1] + m.value[m.cursorPos:] - m.cursorPos-- - } -} - -func (m *textInputModel) Init() tea.Cmd { - return nil -} - -func (m *textInputModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyEnter: - if m.validator != nil { - if err := m.validator(m.value); err != nil { - m.err = err - return m, nil - } - } - if m.value == "" && m.placeholder != "" { - m.value = m.placeholder - } - m.submitted = true - return m, tea.Quit - case tea.KeyEscape, tea.KeyCtrlC: - m.cancelled = true - return m, tea.Quit - case tea.KeyBackspace: - m.backspace() - m.err = nil - case tea.KeyLeft: - if m.cursorPos > 0 { - m.cursorPos-- - } - case tea.KeyRight: - if m.cursorPos < len(m.value) { - m.cursorPos++ - } - default: - if msg.Text != "" { - for _, ch := range msg.Text { - m.insertChar(ch) - } - m.err = nil - } - } - } - return m, nil -} - -func (m *textInputModel) View() string { - var sb strings.Builder - - sb.WriteString(BoldStyle.Render(m.title) + "\n\n") - - display := m.value - if m.masked { - display = strings.Repeat("*", len(m.value)) - } - - if display == "" && m.placeholder != "" { - sb.WriteString(DimStyle.Render(m.placeholder)) - } else { - sb.WriteString(display) - } - sb.WriteString(AccentStyle.Render("█")) // Cursor - - if m.err != nil { - sb.WriteString("\n" + ErrorStyle.Render(fmt.Sprintf(" %s", m.err))) - } - - sb.WriteString("\n\n" + DimStyle.Render("enter submit • esc cancel")) - - return sb.String() -} - -// TextInputOption configures TextInput behaviour. -type TextInputOption func(*textInputConfig) - -type textInputConfig struct { - placeholder string - masked bool - validator func(string) error -} - -// WithPlaceholder sets placeholder text shown when input is empty. -func WithPlaceholder(text string) TextInputOption { - return func(c *textInputConfig) { - c.placeholder = text - } -} - -// WithMask hides input characters (for passwords). -func WithMask() TextInputOption { - return func(c *textInputConfig) { - c.masked = true - } -} - -// WithInputValidator adds a validation function for the input. -func WithInputValidator(fn func(string) error) TextInputOption { - return func(c *textInputConfig) { - c.validator = fn - } -} - -// TextInput presents a styled text input prompt and returns the entered value. -// Returns empty string if cancelled. -// -// Falls back to Question() when stdin is not a terminal. -// -// name, err := cli.TextInput("Enter your name:", WithPlaceholder("Anonymous")) -// pass, err := cli.TextInput("Password:", WithMask()) -func TextInput(title string, opts ...TextInputOption) (string, error) { - cfg := &textInputConfig{} - for _, opt := range opts { - opt(cfg) - } - - // Fall back to simple Question if not a terminal - if !term.IsTerminal(int(StdinFd())) { - var qopts []QuestionOption - if cfg.placeholder != "" { - qopts = append(qopts, WithDefault(cfg.placeholder)) - } - if cfg.validator != nil { - qopts = append(qopts, WithValidator(cfg.validator)) - } - return Question(title, qopts...), nil - } - - m := newTextInputModel(title, cfg.placeholder) - m.masked = cfg.masked - m.validator = cfg.validator - - p := tea.NewProgram(m) - finalModel, err := p.Run() - if err != nil { - return "", err - } - - final := finalModel.(*textInputModel) - if final.cancelled { - return "", nil - } - return final.value, nil -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestTextInputModel ./pkg/cli/... -v` -Expected: All 7 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/textinput.go pkg/cli/textinput_test.go && git commit -m "feat(cli): add TextInput with placeholder, masking, validation - -Co-Authored-By: Virgil " -``` - ---- - -### Task 7: Viewport (scrollable content) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/viewport.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/viewport_test.go` - -A scrollable content pane for displaying long output (logs, diffs, docs). Uses bubbletea internally. - -**Step 1: Write the tests** - -```go -// viewport_test.go -package cli - -import ( - "strings" - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestViewportModel_Good_Create(t *testing.T) { - content := "line 1\nline 2\nline 3" - m := newViewportModel(content, "Title", 5) - assert.Equal(t, "Title", m.title) - assert.Equal(t, 3, len(m.lines)) - assert.Equal(t, 0, m.offset) -} - -func TestViewportModel_Good_ScrollDown(t *testing.T) { - lines := make([]string, 20) - for i := range lines { - lines[i] = strings.Repeat("x", 10) - } - m := newViewportModel(strings.Join(lines, "\n"), "", 5) - m.scrollDown() - assert.Equal(t, 1, m.offset) -} - -func TestViewportModel_Good_ScrollUp(t *testing.T) { - lines := make([]string, 20) - for i := range lines { - lines[i] = strings.Repeat("x", 10) - } - m := newViewportModel(strings.Join(lines, "\n"), "", 5) - m.scrollDown() - m.scrollDown() - m.scrollUp() - assert.Equal(t, 1, m.offset) -} - -func TestViewportModel_Good_NoScrollPastTop(t *testing.T) { - m := newViewportModel("a\nb\nc", "", 5) - m.scrollUp() // Already at top - assert.Equal(t, 0, m.offset) -} - -func TestViewportModel_Good_NoScrollPastBottom(t *testing.T) { - m := newViewportModel("a\nb\nc", "", 5) - for i := 0; i < 10; i++ { - m.scrollDown() - } - // Should clamp — can't scroll past content - assert.GreaterOrEqual(t, m.offset, 0) -} - -func TestViewportModel_Good_View(t *testing.T) { - m := newViewportModel("line 1\nline 2", "My Title", 10) - view := m.View() - assert.Contains(t, view, "My Title") - assert.Contains(t, view, "line 1") - assert.Contains(t, view, "line 2") -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestViewportModel ./pkg/cli/... -v` -Expected: FAIL — `newViewportModel` undefined. - -**Step 3: Write the implementation** - -```go -// viewport.go -package cli - -import ( - "fmt" - "strings" - - tea "github.com/charmbracelet/bubbletea/v2" - "golang.org/x/term" -) - -// viewportModel is the internal bubbletea model for scrollable content. -type viewportModel struct { - title string - lines []string - offset int - height int - quitted bool -} - -func newViewportModel(content, title string, height int) *viewportModel { - lines := strings.Split(content, "\n") - return &viewportModel{ - title: title, - lines: lines, - height: height, - } -} - -func (m *viewportModel) scrollDown() { - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - if m.offset < maxOffset { - m.offset++ - } -} - -func (m *viewportModel) scrollUp() { - if m.offset > 0 { - m.offset-- - } -} - -func (m *viewportModel) Init() tea.Cmd { - return nil -} - -func (m *viewportModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - switch msg := msg.(type) { - case tea.KeyMsg: - switch msg.Type { - case tea.KeyUp: - m.scrollUp() - case tea.KeyDown: - m.scrollDown() - case tea.KeyPgUp: - for i := 0; i < m.height; i++ { - m.scrollUp() - } - case tea.KeyPgDown: - for i := 0; i < m.height; i++ { - m.scrollDown() - } - case tea.KeyHome: - m.offset = 0 - case tea.KeyEnd: - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - m.offset = maxOffset - case tea.KeyEscape, tea.KeyCtrlC: - m.quitted = true - return m, tea.Quit - default: - switch msg.String() { - case "q": - m.quitted = true - return m, tea.Quit - case "j": - m.scrollDown() - case "k": - m.scrollUp() - case "g": - m.offset = 0 - case "G": - maxOffset := len(m.lines) - m.height - if maxOffset < 0 { - maxOffset = 0 - } - m.offset = maxOffset - } - } - } - return m, nil -} - -func (m *viewportModel) View() string { - var sb strings.Builder - - if m.title != "" { - sb.WriteString(BoldStyle.Render(m.title) + "\n") - sb.WriteString(DimStyle.Render(strings.Repeat("─", len(m.title))) + "\n") - } - - // Visible window - end := m.offset + m.height - if end > len(m.lines) { - end = len(m.lines) - } - for _, line := range m.lines[m.offset:end] { - sb.WriteString(line + "\n") - } - - // Scroll indicator - total := len(m.lines) - if total > m.height { - pct := (m.offset * 100) / (total - m.height) - sb.WriteString(DimStyle.Render(fmt.Sprintf("\n%d%% (%d/%d lines)", pct, m.offset+m.height, total))) - } - - sb.WriteString("\n" + DimStyle.Render("↑/↓ scroll • PgUp/PgDn page • q quit")) - - return sb.String() -} - -// ViewportOption configures Viewport behaviour. -type ViewportOption func(*viewportConfig) - -type viewportConfig struct { - title string - height int -} - -// WithViewportTitle sets the title shown above the viewport. -func WithViewportTitle(title string) ViewportOption { - return func(c *viewportConfig) { - c.title = title - } -} - -// WithViewportHeight sets the visible height in lines. -func WithViewportHeight(n int) ViewportOption { - return func(c *viewportConfig) { - c.height = n - } -} - -// Viewport displays scrollable content in the terminal. -// Falls back to printing the full content when stdin is not a terminal. -// -// cli.Viewport(longContent, WithViewportTitle("Build Log"), WithViewportHeight(20)) -func Viewport(content string, opts ...ViewportOption) error { - cfg := &viewportConfig{ - height: 20, - } - for _, opt := range opts { - opt(cfg) - } - - // Fall back to plain output if not a terminal - if !term.IsTerminal(int(StdinFd())) { - if cfg.title != "" { - fmt.Println(BoldStyle.Render(cfg.title)) - fmt.Println(DimStyle.Render(strings.Repeat("─", len(cfg.title)))) - } - fmt.Println(content) - return nil - } - - m := newViewportModel(content, cfg.title, cfg.height) - p := tea.NewProgram(m) - _, err := p.Run() - return err -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run TestViewportModel ./pkg/cli/... -v` -Expected: All 6 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/viewport.go pkg/cli/viewport_test.go && git commit -m "feat(cli): add Viewport for scrollable content (logs, diffs, docs) - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: Future stubs (Form, FilePicker, Tabs) - -**Files:** -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/stubs.go` -- Create: `/Users/snider/Code/host-uk/core/pkg/cli/stubs_test.go` - -Interface definitions for features we'll build later. Simple fallback implementations so the API is usable today. - -**Step 1: Write the tests** - -```go -// stubs_test.go -package cli - -import ( - "testing" - - "github.com/stretchr/testify/assert" -) - -func TestFormField_Good_Types(t *testing.T) { - fields := []FormField{ - {Label: "Name", Key: "name", Type: FieldText}, - {Label: "Password", Key: "pass", Type: FieldPassword}, - {Label: "Accept", Key: "ok", Type: FieldConfirm}, - } - assert.Equal(t, 3, len(fields)) - assert.Equal(t, FieldText, fields[0].Type) - assert.Equal(t, FieldPassword, fields[1].Type) - assert.Equal(t, FieldConfirm, fields[2].Type) -} - -func TestFieldType_Good_Constants(t *testing.T) { - assert.Equal(t, FieldType("text"), FieldText) - assert.Equal(t, FieldType("password"), FieldPassword) - assert.Equal(t, FieldType("confirm"), FieldConfirm) - assert.Equal(t, FieldType("select"), FieldSelect) -} - -func TestTabItem_Good_Structure(t *testing.T) { - tabs := []TabItem{ - {Title: "Overview", Content: "overview content"}, - {Title: "Details", Content: "detail content"}, - } - assert.Equal(t, 2, len(tabs)) - assert.Equal(t, "Overview", tabs[0].Title) -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestFormField|TestFieldType|TestTabItem" ./pkg/cli/... -v` -Expected: FAIL — types undefined. - -**Step 3: Write the implementation** - -```go -// stubs.go -package cli - -// ────────────────────────────────────────────────────────────────────────────── -// Form (stubbed — simple fallback, will use charmbracelet/huh later) -// ────────────────────────────────────────────────────────────────────────────── - -// FieldType defines the type of a form field. -type FieldType string - -const ( - FieldText FieldType = "text" - FieldPassword FieldType = "password" - FieldConfirm FieldType = "confirm" - FieldSelect FieldType = "select" -) - -// FormField describes a single field in a form. -type FormField struct { - Label string - Key string - Type FieldType - Default string - Placeholder string - Options []string // For FieldSelect - Required bool - Validator func(string) error -} - -// Form presents a multi-field form and returns the values keyed by FormField.Key. -// Currently falls back to sequential Question()/Confirm()/Select() calls. -// Will be replaced with charmbracelet/huh interactive form later. -// -// results, err := cli.Form([]cli.FormField{ -// {Label: "Name", Key: "name", Type: cli.FieldText, Required: true}, -// {Label: "Password", Key: "pass", Type: cli.FieldPassword}, -// {Label: "Accept terms?", Key: "terms", Type: cli.FieldConfirm}, -// }) -func Form(fields []FormField) (map[string]string, error) { - results := make(map[string]string, len(fields)) - - for _, f := range fields { - switch f.Type { - case FieldPassword: - val := Question(f.Label+":", WithDefault(f.Default)) - results[f.Key] = val - case FieldConfirm: - if Confirm(f.Label) { - results[f.Key] = "true" - } else { - results[f.Key] = "false" - } - case FieldSelect: - val, err := Select(f.Label, f.Options) - if err != nil { - return nil, err - } - results[f.Key] = val - default: // FieldText - var opts []QuestionOption - if f.Default != "" { - opts = append(opts, WithDefault(f.Default)) - } - if f.Required { - opts = append(opts, RequiredInput()) - } - if f.Validator != nil { - opts = append(opts, WithValidator(f.Validator)) - } - results[f.Key] = Question(f.Label+":", opts...) - } - } - - return results, nil -} - -// ────────────────────────────────────────────────────────────────────────────── -// FilePicker (stubbed — will use charmbracelet/filepicker later) -// ────────────────────────────────────────────────────────────────────────────── - -// FilePickerOption configures FilePicker behaviour. -type FilePickerOption func(*filePickerConfig) - -type filePickerConfig struct { - dir string - extensions []string -} - -// InDirectory sets the starting directory for the file picker. -func InDirectory(dir string) FilePickerOption { - return func(c *filePickerConfig) { - c.dir = dir - } -} - -// WithExtensions filters to specific file extensions (e.g. ".go", ".yaml"). -func WithExtensions(exts ...string) FilePickerOption { - return func(c *filePickerConfig) { - c.extensions = exts - } -} - -// FilePicker presents a file browser and returns the selected path. -// Currently falls back to a text prompt. Will be replaced with an -// interactive file browser later. -// -// path, err := cli.FilePicker(cli.InDirectory("."), cli.WithExtensions(".go")) -func FilePicker(opts ...FilePickerOption) (string, error) { - cfg := &filePickerConfig{dir: "."} - for _, opt := range opts { - opt(cfg) - } - - hint := "File path" - if cfg.dir != "." { - hint += " (from " + cfg.dir + ")" - } - return Question(hint + ":"), nil -} - -// ────────────────────────────────────────────────────────────────────────────── -// Tabs (stubbed — will use bubbletea model later) -// ────────────────────────────────────────────────────────────────────────────── - -// TabItem describes a tab with a title and content. -type TabItem struct { - Title string - Content string -} - -// Tabs displays tabbed content. Currently prints all tabs sequentially. -// Will be replaced with an interactive tab switcher later. -// -// cli.Tabs([]cli.TabItem{ -// {Title: "Overview", Content: summaryText}, -// {Title: "Details", Content: detailText}, -// }) -func Tabs(items []TabItem) error { - for i, tab := range items { - if i > 0 { - Blank() - } - Section(tab.Title) - Println("%s", tab.Content) - } - return nil -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cd /Users/snider/Code/host-uk/core && go test -run "TestFormField|TestFieldType|TestTabItem" ./pkg/cli/... -v` -Expected: All 3 tests PASS. - -**Step 5: Commit** - -```bash -cd /Users/snider/Code/host-uk/core && git add pkg/cli/stubs.go pkg/cli/stubs_test.go && git commit -m "feat(cli): stub Form, FilePicker, Tabs with simple fallbacks - -Interfaces defined for future charmbracelet/huh upgrade. -Current implementations use sequential prompts. - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Run full test suite and verify - -**Step 1: Run all cli package tests** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./pkg/cli/... -v -count=1` -Expected: All tests pass (existing + new). - -**Step 2: Run full module tests** - -Run: `cd /Users/snider/Code/host-uk/core && go test ./... 2>&1 | tail -30` -Expected: No regressions. - -**Step 3: Verify no charmbracelet imports leaked outside pkg/cli** - -Run: `cd /Users/snider/Code/host-uk/core && grep -r "charmbracelet" --include="*.go" . | grep -v pkg/cli/ | grep -v vendor/` -Expected: No output (charmbracelet only imported inside pkg/cli/). - ---- - -## Verification - -After all tasks: - -1. `go test ./pkg/cli/... -v` — all pass (existing + ~34 new tests) -2. `go test ./...` — no regressions across the module -3. `grep -r "charmbracelet" --include="*.go" . | grep -v pkg/cli/` — empty (no leaks) -4. New public API surface: - - `NewSpinner(msg)` → `*SpinnerHandle` (Update, Done, Fail, Stop) - - `NewProgressBar(total)` → `*ProgressHandle` (Increment, Set, SetMessage, Done) - - `InteractiveList(title, items)` → `(int, string)` - - `TextInput(title, opts...)` → `(string, error)` - - `Viewport(content, opts...)` → `error` - - `RunTUI(model)` → `error` (escape hatch) - - `Form(fields)` → `(map[string]string, error)` (stub) - - `FilePicker(opts...)` → `(string, error)` (stub) - - `Tabs(items)` → `error` (stub) - - `Model`, `Msg`, `Cmd`, `KeyMsg`, `KeyType` + key constants - -## Dependency Sequencing - -``` -Task 1 (add deps) ← Task 2 (Spinner) -Task 1 ← Task 3 (ProgressBar) -Task 1 ← Task 4 (TUI runner) ← Task 5 (List) -Task 4 ← Task 6 (TextInput) -Task 4 ← Task 7 (Viewport) -Task 1 ← Task 8 (Stubs) -Tasks 2-8 ← Task 9 (Verification) -``` - -Tasks 2, 3, and 8 are independent of each other (can run in parallel after Task 1). Tasks 5, 6, 7 depend on Task 4 (RunTUI) but are independent of each other. diff --git a/docs/plans/completed/2026-02-21-go-forge-design.md b/docs/plans/completed/2026-02-21-go-forge-design.md deleted file mode 100644 index b718629..0000000 --- a/docs/plans/completed/2026-02-21-go-forge-design.md +++ /dev/null @@ -1,286 +0,0 @@ -# go-forge Design Document - -## Overview - -**go-forge** is a full-coverage Go client for the Forgejo API (450 endpoints, 284 paths, 229 types). It uses a generic `Resource[T, C, U]` pattern for CRUD operations (91% of endpoints) and hand-written methods for 39 unique action endpoints. Types are generated from Forgejo's `swagger.v1.json` spec. - -**Module path:** `forge.lthn.ai/core/go-forge` - -**Origin:** Extracted from `go-scm/forge/` (45 methods covering 10% of API), expanded to full coverage. - -## Architecture - -``` -forge.lthn.ai/core/go-forge -├── client.go # HTTP client: auth, headers, rate limiting, context.Context -├── pagination.go # Generic paginated request helper -├── resource.go # Resource[T, C, U] generic CRUD (List/Get/Create/Update/Delete) -├── errors.go # Typed error handling (APIError, NotFound, Forbidden, etc.) -├── forge.go # Top-level Forge client aggregating all services -│ -├── types/ # Generated from swagger.v1.json -│ ├── generate.go # //go:generate directive -│ ├── repo.go # Repository, CreateRepoOption, EditRepoOption -│ ├── issue.go # Issue, CreateIssueOption, EditIssueOption -│ ├── pr.go # PullRequest, CreatePullRequestOption -│ ├── user.go # User, CreateUserOption -│ ├── org.go # Organisation, CreateOrgOption -│ ├── team.go # Team, CreateTeamOption -│ ├── label.go # Label, CreateLabelOption -│ ├── release.go # Release, CreateReleaseOption -│ ├── branch.go # Branch, BranchProtection -│ ├── milestone.go # Milestone, CreateMilestoneOption -│ ├── hook.go # Hook, CreateHookOption -│ ├── key.go # DeployKey, PublicKey, GPGKey -│ ├── notification.go # NotificationThread, NotificationSubject -│ ├── package.go # Package, PackageFile -│ ├── action.go # ActionRunner, ActionSecret, ActionVariable -│ ├── commit.go # Commit, CommitStatus, CombinedStatus -│ ├── content.go # ContentsResponse, FileOptions -│ ├── wiki.go # WikiPage, WikiPageMetaData -│ ├── review.go # PullReview, PullReviewComment -│ ├── reaction.go # Reaction -│ ├── topic.go # TopicResponse -│ ├── misc.go # Markdown, License, GitignoreTemplate, NodeInfo -│ ├── admin.go # Cron, QuotaGroup, QuotaRule -│ ├── activity.go # Activity, Feed -│ └── common.go # Shared types: Permission, ExternalTracker, etc. -│ -├── repos.go # RepoService: CRUD + fork, mirror, transfer, template -├── issues.go # IssueService: CRUD + pin, deadline, reactions, stopwatch -├── pulls.go # PullService: CRUD + merge, update, reviews, dismiss -├── orgs.go # OrgService: CRUD + members, avatar, block, hooks -├── users.go # UserService: CRUD + keys, followers, starred, settings -├── teams.go # TeamService: CRUD + members, repos -├── admin.go # AdminService: users, orgs, cron, runners, quota, unadopted -├── branches.go # BranchService: CRUD + protection rules -├── releases.go # ReleaseService: CRUD + assets -├── labels.go # LabelService: repo + org + issue labels -├── webhooks.go # WebhookService: CRUD + test hook -├── notifications.go # NotificationService: list, mark read -├── packages.go # PackageService: list, get, delete -├── actions.go # ActionsService: runners, secrets, variables, workflow dispatch -├── contents.go # ContentService: file read/write/delete via API -├── wiki.go # WikiService: pages -├── commits.go # CommitService: status, notes, diff -├── misc.go # MiscService: markdown, licenses, gitignore, nodeinfo -│ -├── config.go # URL/token resolution: env → config file → flags -│ -├── cmd/forgegen/ # Code generator: swagger.v1.json → types/*.go -│ ├── main.go -│ ├── parser.go # Parse OpenAPI 2.0 definitions -│ ├── generator.go # Render Go source files -│ └── templates/ # Go text/template files for codegen -│ -└── testdata/ - └── swagger.v1.json # Pinned spec for testing + generation -``` - -## Key Design Decisions - -### 1. Generic Resource[T, C, U] - -Three type parameters: T (resource type), C (create options), U (update options). - -```go -type Resource[T any, C any, U any] struct { - client *Client - path string // e.g. "/api/v1/repos/{owner}/{repo}/issues" -} - -func (r *Resource[T, C, U]) List(ctx context.Context, params Params, opts ListOptions) ([]T, error) -func (r *Resource[T, C, U]) Get(ctx context.Context, params Params, id string) (*T, error) -func (r *Resource[T, C, U]) Create(ctx context.Context, params Params, body *C) (*T, error) -func (r *Resource[T, C, U]) Update(ctx context.Context, params Params, id string, body *U) (*T, error) -func (r *Resource[T, C, U]) Delete(ctx context.Context, params Params, id string) error -``` - -`Params` is `map[string]string` resolving path variables: `{"owner": "core", "repo": "go-forge"}`. - -This covers 411 of 450 endpoints (91%). - -### 2. Service Structs Embed Resource - -```go -type IssueService struct { - Resource[types.Issue, types.CreateIssueOption, types.EditIssueOption] -} - -// CRUD comes free. Actions are hand-written: -func (s *IssueService) Pin(ctx context.Context, owner, repo string, index int64) error -func (s *IssueService) SetDeadline(ctx context.Context, owner, repo string, index int64, deadline *time.Time) error -``` - -### 3. Top-Level Forge Client - -```go -type Forge struct { - client *Client - Repos *RepoService - Issues *IssueService - Pulls *PullService - Orgs *OrgService - Users *UserService - Teams *TeamService - Admin *AdminService - Branches *BranchService - Releases *ReleaseService - Labels *LabelService - Webhooks *WebhookService - Notifications *NotificationService - Packages *PackageService - Actions *ActionsService - Contents *ContentService - Wiki *WikiService - Commits *CommitService - Misc *MiscService -} - -func NewForge(url, token string, opts ...Option) *Forge -``` - -### 4. Codegen from swagger.v1.json - -The `cmd/forgegen/` tool reads the OpenAPI 2.0 spec and generates: -- Go struct definitions with JSON tags and doc comments -- Enum constants -- Type mapping (OpenAPI → Go) - -229 type definitions → ~25 grouped Go files in `types/`. - -Type mapping rules: -| OpenAPI | Go | -|---------|-----| -| `string` | `string` | -| `string` + `date-time` | `time.Time` | -| `integer` + `int64` | `int64` | -| `integer` | `int` | -| `boolean` | `bool` | -| `array` of T | `[]T` | -| `$ref` | `*T` (pointer) | -| nullable | pointer type | -| `binary` | `[]byte` | - -### 5. HTTP Client - -```go -type Client struct { - baseURL string - token string - httpClient *http.Client - userAgent string -} - -func New(url, token string, opts ...Option) *Client - -func (c *Client) Get(ctx context.Context, path string, out any) error -func (c *Client) Post(ctx context.Context, path string, body, out any) error -func (c *Client) Patch(ctx context.Context, path string, body, out any) error -func (c *Client) Put(ctx context.Context, path string, body, out any) error -func (c *Client) Delete(ctx context.Context, path string) error -``` - -Options: `WithHTTPClient`, `WithUserAgent`, `WithRateLimit`, `WithLogger`. - -### 6. Pagination - -Forgejo uses `page` + `limit` query params and `X-Total-Count` response header. - -```go -type ListOptions struct { - Page int - Limit int // default 50, max configurable -} - -type PagedResult[T any] struct { - Items []T - TotalCount int - Page int - HasMore bool -} - -// ListAll fetches all pages automatically. -func (r *Resource[T, C, U]) ListAll(ctx context.Context, params Params) ([]T, error) -``` - -### 7. Error Handling - -```go -type APIError struct { - StatusCode int - Message string - URL string -} - -func IsNotFound(err error) bool -func IsForbidden(err error) bool -func IsConflict(err error) bool -``` - -### 8. Config Resolution (from go-scm/forge) - -Priority: flags → environment → config file. - -```go -func NewFromConfig(flagURL, flagToken string) (*Forge, error) -func ResolveConfig(flagURL, flagToken string) (url, token string, err error) -func SaveConfig(url, token string) error -``` - -Env vars: `FORGE_URL`, `FORGE_TOKEN`. Config file: `~/.config/forge/config.json`. - -## API Coverage - -| Category | Endpoints | CRUD | Actions | -|----------|-----------|------|---------| -| Repository | 175 | 165 | 10 (fork, mirror, transfer, template, avatar, diffpatch) | -| User | 74 | 70 | 4 (avatar, GPG verify) | -| Issue | 67 | 57 | 10 (pin, deadline, reactions, stopwatch, labels) | -| Organisation | 63 | 59 | 4 (avatar, block/unblock) | -| Admin | 39 | 35 | 4 (cron run, rename, adopt, quota set) | -| Miscellaneous | 12 | 7 | 5 (markdown render, markup, nodeinfo) | -| Notification | 7 | 7 | 0 | -| ActivityPub | 6 | 3 | 3 (inbox POST) | -| Package | 4 | 4 | 0 | -| Settings | 4 | 4 | 0 | -| **Total** | **450** | **411** | **39** | - -## Integration Points - -### go-api - -Services implement `DescribableGroup` from go-api Phase 3, enabling: -- REST endpoint generation via ToolBridge -- Auto-generated OpenAPI spec -- Multi-language SDK codegen - -### go-scm - -go-scm/forge/ becomes a thin adapter importing go-forge types. Existing go-scm users are unaffected — the multi-provider abstraction layer stays. - -### go-ai/mcp - -The MCP subsystem can register go-forge operations as MCP tools, giving AI agents full Forgejo API access. - -## 39 Unique Action Methods - -These require hand-written implementation: - -**Repository:** migrate, fork, generate (template), transfer, accept/reject transfer, mirror sync, push mirror sync, avatar, diffpatch, contents (multi-file modify) - -**Pull Requests:** merge, update (rebase), submit review, dismiss/undismiss review - -**Issues:** pin, set deadline, add reaction, start/stop stopwatch, add issue labels - -**Comments:** add reaction - -**Admin:** run cron task, adopt unadopted, rename user, set quota groups - -**Misc:** render markdown, render raw markdown, render markup, GPG key verify - -**ActivityPub:** inbox POST (actor, repo, user) - -**Actions:** dispatch workflow - -**Git:** set note on commit, test webhook diff --git a/docs/plans/completed/2026-02-21-go-forge-plan.md b/docs/plans/completed/2026-02-21-go-forge-plan.md deleted file mode 100644 index c6b8240..0000000 --- a/docs/plans/completed/2026-02-21-go-forge-plan.md +++ /dev/null @@ -1,2549 +0,0 @@ -# go-forge Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Build a full-coverage Go client for the Forgejo API (450 endpoints) using a generic Resource[T,C,U] pattern and types generated from swagger.v1.json. - -**Architecture:** A code generator (`cmd/forgegen/`) parses Forgejo's Swagger 2.0 spec and emits typed Go structs. A generic `Resource[T,C,U]` provides List/Get/Create/Update/Delete for 411 CRUD endpoints. 18 service structs embed the generic resource and add 39 hand-written action methods. An HTTP client handles auth, pagination, rate limiting, and context.Context. - -**Tech Stack:** Go 1.25, `net/http`, `text/template`, generics, Swagger 2.0 (JSON) - ---- - -## Context - -**This is a NEW repo** at `forge.lthn.ai/core/go-forge`. Create it locally at `/Users/snider/Code/go-forge`. - -**Extracted from:** `/Users/snider/Code/go-scm/forge/` (45 methods covering 10% of API). The config resolution pattern (env → file → flags) comes from there. - -**Swagger spec:** Download from `https://forge.lthn.ai/swagger.v1.json` — Swagger 2.0 format, 229 type definitions, 450 operations across 284 paths. Pin it at `testdata/swagger.v1.json`. - -**Forgejo version:** 10.0.3 (Gitea 1.22.0 compatible) - -**Dependencies:** None (pure `net/http`). Config uses `forge.lthn.ai/core/go` for `pkg/config` and `pkg/log` — same as go-scm. - -**Key insight:** 91% of endpoints are generic CRUD (List/Get/Create/Update/Delete). The generic `Resource[T,C,U]` pattern means each service is a struct definition + path constant + optional action methods. The code generator handles 229 type definitions. - -**Test command:** `go test ./...` from the repo root. - -**The forge remote for this repo will be:** `ssh://git@forge.lthn.ai:2223/core/go-forge.git` - ---- - -## Wave 1: Foundation (Tasks 1-6) - -### Task 1: Repo scaffolding + go.mod - -**Files:** -- Create: `go.mod` -- Create: `go.sum` (auto-generated) -- Create: `doc.go` -- Create: `testdata/swagger.v1.json` (downloaded) - -**Step 1: Create directory and initialise module** - -```bash -mkdir -p /Users/snider/Code/go-forge/testdata -cd /Users/snider/Code/go-forge -git init -go mod init forge.lthn.ai/core/go-forge -``` - -**Step 2: Download and pin swagger spec** - -```bash -curl -s https://forge.lthn.ai/swagger.v1.json > testdata/swagger.v1.json -``` - -Verify: `python3 -c "import json; d=json.load(open('testdata/swagger.v1.json')); print(f'{len(d[\"definitions\"])} types, {len(d[\"paths\"])} paths')"` -Expected: `229 types, 284 paths` - -**Step 3: Write doc.go** - -```go -// Package forge provides a full-coverage Go client for the Forgejo API. -// -// Usage: -// -// f := forge.NewForge("https://forge.lthn.ai", "your-token") -// repos, err := f.Repos.List(ctx, forge.Params{"org": "core"}, forge.DefaultList) -// -// Types are generated from Forgejo's swagger.v1.json spec via cmd/forgegen/. -// Run `go generate ./types/...` to regenerate after a Forgejo upgrade. -package forge -``` - -**Step 4: Commit** - -```bash -git add -A -git commit -m "feat: scaffold go-forge repo with pinned swagger spec - -Co-Authored-By: Virgil " -``` - ---- - -### Task 2: HTTP Client - -**Files:** -- Create: `client.go` -- Create: `client_test.go` - -**Step 1: Write client tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" -) - -func TestClient_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodGet { - t.Errorf("expected GET, got %s", r.Method) - } - if r.Header.Get("Authorization") != "token test-token" { - t.Errorf("missing auth header") - } - if r.URL.Path != "/api/v1/user" { - t.Errorf("wrong path: %s", r.URL.Path) - } - json.NewEncoder(w).Encode(map[string]string{"login": "virgil"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - var out map[string]string - err := c.Get(context.Background(), "/api/v1/user", &out) - if err != nil { - t.Fatal(err) - } - if out["login"] != "virgil" { - t.Errorf("got login=%q", out["login"]) - } -} - -func TestClient_Good_Post(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - var body map[string]string - json.NewDecoder(r.Body).Decode(&body) - if body["name"] != "test-repo" { - t.Errorf("wrong body: %v", body) - } - w.WriteHeader(http.StatusCreated) - json.NewEncoder(w).Encode(map[string]any{"id": 1, "name": "test-repo"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - body := map[string]string{"name": "test-repo"} - var out map[string]any - err := c.Post(context.Background(), "/api/v1/orgs/core/repos", body, &out) - if err != nil { - t.Fatal(err) - } - if out["name"] != "test-repo" { - t.Errorf("got name=%v", out["name"]) - } -} - -func TestClient_Good_Delete(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodDelete { - t.Errorf("expected DELETE, got %s", r.Method) - } - w.WriteHeader(http.StatusNoContent) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Delete(context.Background(), "/api/v1/repos/core/test") - if err != nil { - t.Fatal(err) - } -} - -func TestClient_Bad_ServerError(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(http.StatusInternalServerError) - json.NewEncoder(w).Encode(map[string]string{"message": "internal error"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Get(context.Background(), "/api/v1/user", nil) - if err == nil { - t.Fatal("expected error") - } - var apiErr *APIError - if !errors.As(err, &apiErr) { - t.Fatalf("expected APIError, got %T", err) - } - if apiErr.StatusCode != 500 { - t.Errorf("got status=%d", apiErr.StatusCode) - } -} - -func TestClient_Bad_NotFound(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(http.StatusNotFound) - json.NewEncoder(w).Encode(map[string]string{"message": "not found"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - err := c.Get(context.Background(), "/api/v1/repos/x/y", nil) - if !IsNotFound(err) { - t.Fatalf("expected not found, got %v", err) - } -} - -func TestClient_Good_ContextCancellation(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - <-r.Context().Done() - })) - defer srv.Close() - - c := NewClient(srv.URL, "test-token") - ctx, cancel := context.WithCancel(context.Background()) - cancel() // cancel immediately - err := c.Get(ctx, "/api/v1/user", nil) - if err == nil { - t.Fatal("expected error from cancelled context") - } -} - -func TestClient_Good_Options(t *testing.T) { - c := NewClient("https://forge.lthn.ai", "tok", - WithUserAgent("go-forge/1.0"), - ) - if c.userAgent != "go-forge/1.0" { - t.Errorf("got user agent=%q", c.userAgent) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestClient` -Expected: Compilation errors (types don't exist yet) - -**Step 3: Write client.go** - -```go -package forge - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "io" - "net/http" - "strings" -) - -// APIError represents an error response from the Forgejo API. -type APIError struct { - StatusCode int - Message string - URL string -} - -func (e *APIError) Error() string { - return fmt.Sprintf("forge: %s %d: %s", e.URL, e.StatusCode, e.Message) -} - -// IsNotFound returns true if the error is a 404 response. -func IsNotFound(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusNotFound -} - -// IsForbidden returns true if the error is a 403 response. -func IsForbidden(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusForbidden -} - -// IsConflict returns true if the error is a 409 response. -func IsConflict(err error) bool { - var apiErr *APIError - return errors.As(err, &apiErr) && apiErr.StatusCode == http.StatusConflict -} - -// Option configures the Client. -type Option func(*Client) - -// WithHTTPClient sets a custom http.Client. -func WithHTTPClient(hc *http.Client) Option { - return func(c *Client) { c.httpClient = hc } -} - -// WithUserAgent sets the User-Agent header. -func WithUserAgent(ua string) Option { - return func(c *Client) { c.userAgent = ua } -} - -// Client is a low-level HTTP client for the Forgejo API. -type Client struct { - baseURL string - token string - httpClient *http.Client - userAgent string -} - -// NewClient creates a new Forgejo API client. -func NewClient(url, token string, opts ...Option) *Client { - c := &Client{ - baseURL: strings.TrimRight(url, "/"), - token: token, - httpClient: http.DefaultClient, - userAgent: "go-forge/0.1", - } - for _, opt := range opts { - opt(c) - } - return c -} - -// Get performs a GET request. -func (c *Client) Get(ctx context.Context, path string, out any) error { - return c.do(ctx, http.MethodGet, path, nil, out) -} - -// Post performs a POST request. -func (c *Client) Post(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPost, path, body, out) -} - -// Patch performs a PATCH request. -func (c *Client) Patch(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPatch, path, body, out) -} - -// Put performs a PUT request. -func (c *Client) Put(ctx context.Context, path string, body, out any) error { - return c.do(ctx, http.MethodPut, path, body, out) -} - -// Delete performs a DELETE request. -func (c *Client) Delete(ctx context.Context, path string) error { - return c.do(ctx, http.MethodDelete, path, nil, nil) -} - -func (c *Client) do(ctx context.Context, method, path string, body, out any) error { - url := c.baseURL + path - - var bodyReader io.Reader - if body != nil { - data, err := json.Marshal(body) - if err != nil { - return fmt.Errorf("forge: marshal body: %w", err) - } - bodyReader = bytes.NewReader(data) - } - - req, err := http.NewRequestWithContext(ctx, method, url, bodyReader) - if err != nil { - return fmt.Errorf("forge: create request: %w", err) - } - - req.Header.Set("Authorization", "token "+c.token) - req.Header.Set("Accept", "application/json") - if body != nil { - req.Header.Set("Content-Type", "application/json") - } - if c.userAgent != "" { - req.Header.Set("User-Agent", c.userAgent) - } - - resp, err := c.httpClient.Do(req) - if err != nil { - return fmt.Errorf("forge: request %s %s: %w", method, path, err) - } - defer resp.Body.Close() - - if resp.StatusCode >= 400 { - return c.parseError(resp, path) - } - - if out != nil && resp.StatusCode != http.StatusNoContent { - if err := json.NewDecoder(resp.Body).Decode(out); err != nil { - return fmt.Errorf("forge: decode response: %w", err) - } - } - - return nil -} - -func (c *Client) parseError(resp *http.Response, path string) error { - var errBody struct { - Message string `json:"message"` - } - _ = json.NewDecoder(resp.Body).Decode(&errBody) - return &APIError{ - StatusCode: resp.StatusCode, - Message: errBody.Message, - URL: path, - } -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestClient` -Expected: All 7 tests PASS - -**Step 5: Commit** - -```bash -git add client.go client_test.go -git commit -m "feat: HTTP client with auth, context, error handling - -Co-Authored-By: Virgil " -``` - ---- - -### Task 3: Pagination - -**Files:** -- Create: `pagination.go` -- Create: `pagination_test.go` - -**Step 1: Write pagination tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "fmt" - "net/http" - "net/http/httptest" - "strconv" - "testing" -) - -func TestPagination_Good_SinglePage(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "2") - json.NewEncoder(w).Encode([]map[string]int{{"id": 1}, {"id": 2}}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 2 { - t.Errorf("got %d items", len(result)) - } -} - -func TestPagination_Good_MultiPage(t *testing.T) { - page := 0 - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - page++ - w.Header().Set("X-Total-Count", "100") - items := make([]map[string]int, 50) - for i := range items { - items[i] = map[string]int{"id": (page-1)*50 + i + 1} - } - json.NewEncoder(w).Encode(items) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 100 { - t.Errorf("got %d items, want 100", len(result)) - } -} - -func TestPagination_Good_EmptyResult(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "0") - json.NewEncoder(w).Encode([]map[string]int{}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - result, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err != nil { - t.Fatal(err) - } - if len(result) != 0 { - t.Errorf("got %d items", len(result)) - } -} - -func TestListPage_Good_QueryParams(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - p := r.URL.Query().Get("page") - l := r.URL.Query().Get("limit") - s := r.URL.Query().Get("state") - if p != "2" || l != "25" || s != "open" { - t.Errorf("wrong params: page=%s limit=%s state=%s", p, l, s) - } - w.Header().Set("X-Total-Count", "50") - json.NewEncoder(w).Encode([]map[string]int{}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - _, err := ListPage[map[string]int](context.Background(), c, "/api/v1/repos", - map[string]string{"state": "open"}, ListOptions{Page: 2, Limit: 25}) - if err != nil { - t.Fatal(err) - } -} - -func TestPagination_Bad_ServerError(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.WriteHeader(500) - json.NewEncoder(w).Encode(map[string]string{"message": "fail"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - _, err := ListAll[map[string]int](context.Background(), c, "/api/v1/repos", nil) - if err == nil { - t.Fatal("expected error") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestPagination -run TestListPage` -Expected: Compilation errors - -**Step 3: Write pagination.go** - -```go -package forge - -import ( - "context" - "encoding/json" - "fmt" - "net/http" - "net/url" - "strconv" -) - -// ListOptions controls pagination. -type ListOptions struct { - Page int // 1-based page number - Limit int // items per page (default 50) -} - -// DefaultList returns sensible default pagination. -var DefaultList = ListOptions{Page: 1, Limit: 50} - -// PagedResult holds a single page of results with metadata. -type PagedResult[T any] struct { - Items []T - TotalCount int - Page int - HasMore bool -} - -// ListPage fetches a single page of results. -// Extra query params can be passed via the query map. -func ListPage[T any](ctx context.Context, c *Client, path string, query map[string]string, opts ListOptions) (*PagedResult[T], error) { - if opts.Page < 1 { - opts.Page = 1 - } - if opts.Limit < 1 { - opts.Limit = 50 - } - - u, err := url.Parse(c.baseURL + path) - if err != nil { - return nil, fmt.Errorf("forge: parse url: %w", err) - } - - q := u.Query() - q.Set("page", strconv.Itoa(opts.Page)) - q.Set("limit", strconv.Itoa(opts.Limit)) - for k, v := range query { - q.Set(k, v) - } - u.RawQuery = q.Encode() - - req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil) - if err != nil { - return nil, fmt.Errorf("forge: create request: %w", err) - } - - req.Header.Set("Authorization", "token "+c.token) - req.Header.Set("Accept", "application/json") - if c.userAgent != "" { - req.Header.Set("User-Agent", c.userAgent) - } - - resp, err := c.httpClient.Do(req) - if err != nil { - return nil, fmt.Errorf("forge: request GET %s: %w", path, err) - } - defer resp.Body.Close() - - if resp.StatusCode >= 400 { - return nil, c.parseError(resp, path) - } - - var items []T - if err := json.NewDecoder(resp.Body).Decode(&items); err != nil { - return nil, fmt.Errorf("forge: decode response: %w", err) - } - - totalCount, _ := strconv.Atoi(resp.Header.Get("X-Total-Count")) - - return &PagedResult[T]{ - Items: items, - TotalCount: totalCount, - Page: opts.Page, - HasMore: len(items) >= opts.Limit && opts.Page*opts.Limit < totalCount, - }, nil -} - -// ListAll fetches all pages of results. -func ListAll[T any](ctx context.Context, c *Client, path string, query map[string]string) ([]T, error) { - var all []T - page := 1 - - for { - result, err := ListPage[T](ctx, c, path, query, ListOptions{Page: page, Limit: 50}) - if err != nil { - return nil, err - } - all = append(all, result.Items...) - if !result.HasMore { - break - } - page++ - } - - return all, nil -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestPagination|TestListPage"` -Expected: All 5 tests PASS - -**Step 5: Commit** - -```bash -git add pagination.go pagination_test.go -git commit -m "feat: generic pagination with ListAll and ListPage - -Co-Authored-By: Virgil " -``` - ---- - -### Task 4: Params and path resolution - -**Files:** -- Create: `params.go` -- Create: `params_test.go` - -**Step 1: Write tests** - -```go -package forge - -import "testing" - -func TestResolvePath_Good_Simple(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}", Params{"owner": "core", "repo": "go-forge"}) - want := "/api/v1/repos/core/go-forge" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} - -func TestResolvePath_Good_NoParams(t *testing.T) { - got := ResolvePath("/api/v1/user", nil) - if got != "/api/v1/user" { - t.Errorf("got %q", got) - } -} - -func TestResolvePath_Good_WithID(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}/issues/{index}", Params{ - "owner": "core", "repo": "go-forge", "index": "42", - }) - want := "/api/v1/repos/core/go-forge/issues/42" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} - -func TestResolvePath_Good_URLEncoding(t *testing.T) { - got := ResolvePath("/api/v1/repos/{owner}/{repo}", Params{"owner": "my org", "repo": "my repo"}) - want := "/api/v1/repos/my%20org/my%20repo" - if got != want { - t.Errorf("got %q, want %q", got, want) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolvePath` -Expected: Compilation errors - -**Step 3: Write params.go** - -```go -package forge - -import ( - "net/url" - "strings" -) - -// Params maps path variable names to values. -// Example: Params{"owner": "core", "repo": "go-forge"} -type Params map[string]string - -// ResolvePath substitutes {placeholders} in path with values from params. -func ResolvePath(path string, params Params) string { - for k, v := range params { - path = strings.ReplaceAll(path, "{"+k+"}", url.PathEscape(v)) - } - return path -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolvePath` -Expected: All 4 tests PASS - -**Step 5: Commit** - -```bash -git add params.go params_test.go -git commit -m "feat: path parameter resolution with URL encoding - -Co-Authored-By: Virgil " -``` - ---- - -### Task 5: Generic Resource[T, C, U] - -**Files:** -- Create: `resource.go` -- Create: `resource_test.go` - -**Step 1: Write resource tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" -) - -// Test types -type testItem struct { - ID int `json:"id"` - Name string `json:"name"` -} - -type testCreate struct { - Name string `json:"name"` -} - -type testUpdate struct { - Name *string `json:"name,omitempty"` -} - -func TestResource_Good_List(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/v1/orgs/core/repos" { - t.Errorf("wrong path: %s", r.URL.Path) - } - w.Header().Set("X-Total-Count", "2") - json.NewEncoder(w).Encode([]testItem{{1, "a"}, {2, "b"}}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/orgs/{org}/repos") - - items, err := res.List(context.Background(), Params{"org": "core"}, DefaultList) - if err != nil { - t.Fatal(err) - } - if len(items.Items) != 2 { - t.Errorf("got %d items", len(items.Items)) - } -} - -func TestResource_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.URL.Path != "/api/v1/repos/core/go-forge" { - t.Errorf("wrong path: %s", r.URL.Path) - } - json.NewEncoder(w).Encode(testItem{1, "go-forge"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - item, err := res.Get(context.Background(), Params{"owner": "core", "repo": "go-forge"}) - if err != nil { - t.Fatal(err) - } - if item.Name != "go-forge" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Create(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - var body testCreate - json.NewDecoder(r.Body).Decode(&body) - w.WriteHeader(http.StatusCreated) - json.NewEncoder(w).Encode(testItem{1, body.Name}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/orgs/{org}/repos") - - item, err := res.Create(context.Background(), Params{"org": "core"}, &testCreate{Name: "new-repo"}) - if err != nil { - t.Fatal(err) - } - if item.Name != "new-repo" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Update(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPatch { - t.Errorf("expected PATCH, got %s", r.Method) - } - json.NewEncoder(w).Encode(testItem{1, "updated"}) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - name := "updated" - item, err := res.Update(context.Background(), Params{"owner": "core", "repo": "old"}, &testUpdate{Name: &name}) - if err != nil { - t.Fatal(err) - } - if item.Name != "updated" { - t.Errorf("got name=%q", item.Name) - } -} - -func TestResource_Good_Delete(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodDelete { - t.Errorf("expected DELETE, got %s", r.Method) - } - w.WriteHeader(http.StatusNoContent) - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos/{owner}/{repo}") - - err := res.Delete(context.Background(), Params{"owner": "core", "repo": "old"}) - if err != nil { - t.Fatal(err) - } -} - -func TestResource_Good_ListAll(t *testing.T) { - page := 0 - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - page++ - w.Header().Set("X-Total-Count", "3") - if page == 1 { - json.NewEncoder(w).Encode([]testItem{{1, "a"}, {2, "b"}}) - } else { - json.NewEncoder(w).Encode([]testItem{{3, "c"}}) - } - })) - defer srv.Close() - - c := NewClient(srv.URL, "tok") - res := NewResource[testItem, testCreate, testUpdate](c, "/api/v1/repos") - - items, err := res.ListAll(context.Background(), nil) - if err != nil { - t.Fatal(err) - } - if len(items) != 3 { - t.Errorf("got %d items, want 3", len(items)) - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResource` -Expected: Compilation errors - -**Step 3: Write resource.go** - -```go -package forge - -import "context" - -// Resource provides generic CRUD operations for a Forgejo API resource. -// T is the resource type, C is the create options type, U is the update options type. -type Resource[T any, C any, U any] struct { - client *Client - path string -} - -// NewResource creates a new Resource for the given path pattern. -// The path may contain {placeholders} that are resolved via Params. -func NewResource[T any, C any, U any](c *Client, path string) *Resource[T, C, U] { - return &Resource[T, C, U]{client: c, path: path} -} - -// List returns a single page of resources. -func (r *Resource[T, C, U]) List(ctx context.Context, params Params, opts ListOptions) (*PagedResult[T], error) { - return ListPage[T](ctx, r.client, ResolvePath(r.path, params), nil, opts) -} - -// ListAll returns all resources across all pages. -func (r *Resource[T, C, U]) ListAll(ctx context.Context, params Params) ([]T, error) { - return ListAll[T](ctx, r.client, ResolvePath(r.path, params), nil) -} - -// Get returns a single resource by appending id to the path. -func (r *Resource[T, C, U]) Get(ctx context.Context, params Params) (*T, error) { - var out T - if err := r.client.Get(ctx, ResolvePath(r.path, params), &out); err != nil { - return nil, err - } - return &out, nil -} - -// Create creates a new resource. -func (r *Resource[T, C, U]) Create(ctx context.Context, params Params, body *C) (*T, error) { - var out T - if err := r.client.Post(ctx, ResolvePath(r.path, params), body, &out); err != nil { - return nil, err - } - return &out, nil -} - -// Update modifies an existing resource. -func (r *Resource[T, C, U]) Update(ctx context.Context, params Params, body *U) (*T, error) { - var out T - if err := r.client.Patch(ctx, ResolvePath(r.path, params), body, &out); err != nil { - return nil, err - } - return &out, nil -} - -// Delete removes a resource. -func (r *Resource[T, C, U]) Delete(ctx context.Context, params Params) error { - return r.client.Delete(ctx, ResolvePath(r.path, params)) -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResource` -Expected: All 6 tests PASS - -**Step 5: Commit** - -```bash -git add resource.go resource_test.go -git commit -m "feat: generic Resource[T,C,U] for CRUD operations - -Co-Authored-By: Virgil " -``` - ---- - -### Task 6: Config resolution (extracted from go-scm) - -**Files:** -- Create: `config.go` -- Create: `config_test.go` - -**Step 1: Write config tests** - -```go -package forge - -import ( - "os" - "testing" -) - -func TestResolveConfig_Good_EnvOverrides(t *testing.T) { - t.Setenv("FORGE_URL", "https://forge.example.com") - t.Setenv("FORGE_TOKEN", "env-token") - - url, token, err := ResolveConfig("", "") - if err != nil { - t.Fatal(err) - } - if url != "https://forge.example.com" { - t.Errorf("got url=%q", url) - } - if token != "env-token" { - t.Errorf("got token=%q", token) - } -} - -func TestResolveConfig_Good_FlagOverridesEnv(t *testing.T) { - t.Setenv("FORGE_URL", "https://env.example.com") - t.Setenv("FORGE_TOKEN", "env-token") - - url, token, err := ResolveConfig("https://flag.example.com", "flag-token") - if err != nil { - t.Fatal(err) - } - if url != "https://flag.example.com" { - t.Errorf("got url=%q", url) - } - if token != "flag-token" { - t.Errorf("got token=%q", token) - } -} - -func TestResolveConfig_Good_DefaultURL(t *testing.T) { - // Clear env vars to test defaults - os.Unsetenv("FORGE_URL") - os.Unsetenv("FORGE_TOKEN") - - url, _, err := ResolveConfig("", "") - if err != nil { - t.Fatal(err) - } - if url != DefaultURL { - t.Errorf("got url=%q, want %q", url, DefaultURL) - } -} - -func TestNewForgeFromConfig_Bad_NoToken(t *testing.T) { - os.Unsetenv("FORGE_URL") - os.Unsetenv("FORGE_TOKEN") - - _, err := NewForgeFromConfig("", "") - if err == nil { - t.Fatal("expected error for missing token") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run TestResolveConfig -run TestNewForgeFromConfig` -Expected: Compilation errors - -**Step 3: Write config.go** - -```go -package forge - -import ( - "fmt" - "os" -) - -const ( - // DefaultURL is used when no URL is configured. - DefaultURL = "http://localhost:3000" -) - -// ResolveConfig resolves Forge URL and token from multiple sources. -// Priority (highest to lowest): flags → environment → defaults. -func ResolveConfig(flagURL, flagToken string) (url, token string, err error) { - // Environment variables - url = os.Getenv("FORGE_URL") - token = os.Getenv("FORGE_TOKEN") - - // Flag overrides - if flagURL != "" { - url = flagURL - } - if flagToken != "" { - token = flagToken - } - - // Default URL - if url == "" { - url = DefaultURL - } - - return url, token, nil -} - -// NewForgeFromConfig creates a Forge client using resolved configuration. -func NewForgeFromConfig(flagURL, flagToken string, opts ...Option) (*Forge, error) { - url, token, err := ResolveConfig(flagURL, flagToken) - if err != nil { - return nil, err - } - if token == "" { - return nil, fmt.Errorf("forge: no API token configured (set FORGE_TOKEN or pass --token)") - } - return NewForge(url, token, opts...), nil -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestResolveConfig|TestNewForgeFromConfig"` -Expected: All 4 tests PASS (Note: `NewForge` doesn't exist yet — if this fails, create a stub `NewForge` function that just returns `&Forge{client: NewClient(url, token, opts...)}`) - -**Step 5: Commit** - -```bash -git add config.go config_test.go -git commit -m "feat: config resolution from env vars and flags - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 2: Code Generator (Tasks 7-9) - -### Task 7: Swagger spec parser - -**Files:** -- Create: `cmd/forgegen/main.go` -- Create: `cmd/forgegen/parser.go` -- Create: `cmd/forgegen/parser_test.go` - -The parser reads swagger.v1.json and extracts type definitions into an intermediate representation. - -**Step 1: Write parser tests** - -```go -package main - -import ( - "os" - "testing" -) - -func TestParser_Good_LoadSpec(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - if spec.Swagger != "2.0" { - t.Errorf("got swagger=%q", spec.Swagger) - } - if len(spec.Definitions) < 200 { - t.Errorf("got %d definitions, expected 200+", len(spec.Definitions)) - } -} - -func TestParser_Good_ExtractTypes(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - if len(types) < 200 { - t.Errorf("got %d types", len(types)) - } - - // Check a known type - repo, ok := types["Repository"] - if !ok { - t.Fatal("Repository type not found") - } - if len(repo.Fields) < 50 { - t.Errorf("Repository has %d fields, expected 50+", len(repo.Fields)) - } -} - -func TestParser_Good_FieldTypes(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - repo := types["Repository"] - - // Check specific field mappings - for _, f := range repo.Fields { - switch f.JSONName { - case "id": - if f.GoType != "int64" { - t.Errorf("id: got %q, want int64", f.GoType) - } - case "name": - if f.GoType != "string" { - t.Errorf("name: got %q, want string", f.GoType) - } - case "private": - if f.GoType != "bool" { - t.Errorf("private: got %q, want bool", f.GoType) - } - case "created_at": - if f.GoType != "time.Time" { - t.Errorf("created_at: got %q, want time.Time", f.GoType) - } - case "owner": - if f.GoType != "*User" { - t.Errorf("owner: got %q, want *User", f.GoType) - } - } - } -} - -func TestParser_Good_DetectCreateEditPairs(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - pairs := DetectCRUDPairs(spec) - // Should find Repository, Issue, PullRequest, etc. - if len(pairs) < 10 { - t.Errorf("got %d pairs, expected 10+", len(pairs)) - } - - found := false - for _, p := range pairs { - if p.Base == "Repository" { - found = true - if p.Create != "CreateRepoOption" { - t.Errorf("repo create=%q", p.Create) - } - } - } - if !found { - t.Fatal("Repository pair not found") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestParser` -Expected: Compilation errors - -**Step 3: Write parser.go** - -```go -package main - -import ( - "encoding/json" - "fmt" - "os" - "sort" - "strings" -) - -// Spec represents a Swagger 2.0 specification. -type Spec struct { - Swagger string `json:"swagger"` - Info SpecInfo `json:"info"` - Definitions map[string]SchemaDefinition `json:"definitions"` - Paths map[string]map[string]any `json:"paths"` -} - -type SpecInfo struct { - Title string `json:"title"` - Version string `json:"version"` -} - -// SchemaDefinition represents a type definition in the spec. -type SchemaDefinition struct { - Description string `json:"description"` - Type string `json:"type"` - Properties map[string]SchemaProperty `json:"properties"` - Required []string `json:"required"` - Enum []any `json:"enum"` - XGoName string `json:"x-go-name"` -} - -// SchemaProperty represents a field in a type definition. -type SchemaProperty struct { - Type string `json:"type"` - Format string `json:"format"` - Description string `json:"description"` - Ref string `json:"$ref"` - Items *SchemaProperty `json:"items"` - Enum []any `json:"enum"` - XGoName string `json:"x-go-name"` -} - -// GoType represents a Go type extracted from the spec. -type GoType struct { - Name string - Description string - Fields []GoField - IsEnum bool - EnumValues []string -} - -// GoField represents a field in a Go struct. -type GoField struct { - GoName string - GoType string - JSONName string - Comment string - Required bool -} - -// CRUDPair maps a base type to its Create and Edit option types. -type CRUDPair struct { - Base string // e.g. "Repository" - Create string // e.g. "CreateRepoOption" - Edit string // e.g. "EditRepoOption" -} - -// LoadSpec reads and parses a Swagger 2.0 JSON file. -func LoadSpec(path string) (*Spec, error) { - data, err := os.ReadFile(path) - if err != nil { - return nil, fmt.Errorf("read spec: %w", err) - } - var spec Spec - if err := json.Unmarshal(data, &spec); err != nil { - return nil, fmt.Errorf("parse spec: %w", err) - } - return &spec, nil -} - -// ExtractTypes converts spec definitions to Go types. -func ExtractTypes(spec *Spec) map[string]*GoType { - result := make(map[string]*GoType) - - for name, def := range spec.Definitions { - gt := &GoType{ - Name: name, - Description: def.Description, - } - - if len(def.Enum) > 0 { - gt.IsEnum = true - for _, v := range def.Enum { - gt.EnumValues = append(gt.EnumValues, fmt.Sprintf("%v", v)) - } - sort.Strings(gt.EnumValues) - result[name] = gt - continue - } - - required := make(map[string]bool) - for _, r := range def.Required { - required[r] = true - } - - for fieldName, prop := range def.Properties { - goName := prop.XGoName - if goName == "" { - goName = pascalCase(fieldName) - } - - gf := GoField{ - GoName: goName, - GoType: resolveGoType(prop), - JSONName: fieldName, - Comment: prop.Description, - Required: required[fieldName], - } - gt.Fields = append(gt.Fields, gf) - } - - // Sort fields alphabetically for stable output - sort.Slice(gt.Fields, func(i, j int) bool { - return gt.Fields[i].GoName < gt.Fields[j].GoName - }) - - result[name] = gt - } - - return result -} - -// DetectCRUDPairs finds Create/Edit option pairs. -func DetectCRUDPairs(spec *Spec) []CRUDPair { - var pairs []CRUDPair - - for name := range spec.Definitions { - if !strings.HasPrefix(name, "Create") || !strings.HasSuffix(name, "Option") { - continue - } - - // CreateXxxOption → Xxx → EditXxxOption - inner := strings.TrimPrefix(name, "Create") - inner = strings.TrimSuffix(inner, "Option") - - editName := "Edit" + inner + "Option" - - pair := CRUDPair{ - Base: inner, - Create: name, - } - - if _, ok := spec.Definitions[editName]; ok { - pair.Edit = editName - } - - pairs = append(pairs, pair) - } - - sort.Slice(pairs, func(i, j int) bool { - return pairs[i].Base < pairs[j].Base - }) - - return pairs -} - -func resolveGoType(prop SchemaProperty) string { - if prop.Ref != "" { - parts := strings.Split(prop.Ref, "/") - return "*" + parts[len(parts)-1] - } - - switch prop.Type { - case "string": - switch prop.Format { - case "date-time": - return "time.Time" - case "binary": - return "[]byte" - default: - return "string" - } - case "integer": - switch prop.Format { - case "int64": - return "int64" - case "int32": - return "int32" - default: - return "int" - } - case "number": - switch prop.Format { - case "float": - return "float32" - default: - return "float64" - } - case "boolean": - return "bool" - case "array": - if prop.Items != nil { - itemType := resolveGoType(*prop.Items) - return "[]" + itemType - } - return "[]any" - case "object": - return "map[string]any" - default: - if prop.Type == "" && prop.Ref == "" { - return "any" - } - return "any" - } -} - -func pascalCase(s string) string { - parts := strings.FieldsFunc(s, func(r rune) bool { - return r == '_' || r == '-' - }) - for i, p := range parts { - if len(p) == 0 { - continue - } - // Handle common acronyms - upper := strings.ToUpper(p) - switch upper { - case "ID", "URL", "HTML", "SSH", "HTTP", "HTTPS", "API", "URI", "GPG", "IP", "CSS", "JS": - parts[i] = upper - default: - parts[i] = strings.ToUpper(p[:1]) + p[1:] - } - } - return strings.Join(parts, "") -} -``` - -**Step 4: Write main.go stub** - -```go -package main - -import ( - "flag" - "fmt" - "os" -) - -func main() { - specPath := flag.String("spec", "testdata/swagger.v1.json", "path to swagger.v1.json") - outDir := flag.String("out", "types", "output directory for generated types") - flag.Parse() - - spec, err := LoadSpec(*specPath) - if err != nil { - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - os.Exit(1) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - fmt.Printf("Loaded %d types, %d CRUD pairs\n", len(types), len(pairs)) - fmt.Printf("Output dir: %s\n", *outDir) - - // Generation happens in Task 8 - if err := Generate(types, pairs, *outDir); err != nil { - fmt.Fprintf(os.Stderr, "Error: %v\n", err) - os.Exit(1) - } -} -``` - -**Step 5: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestParser` -Expected: All 4 tests PASS (Note: `Generate` doesn't exist yet — add a stub: `func Generate(...) error { return nil }`) - -**Step 6: Commit** - -```bash -git add cmd/forgegen/ -git commit -m "feat: swagger spec parser for type extraction - -Co-Authored-By: Virgil " -``` - ---- - -### Task 8: Code generator — Go source emission - -**Files:** -- Create: `cmd/forgegen/generator.go` -- Create: `cmd/forgegen/generator_test.go` - -**Step 1: Write generator tests** - -```go -package main - -import ( - "os" - "path/filepath" - "strings" - "testing" -) - -func TestGenerate_Good_CreatesFiles(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Should create at least one .go file - entries, _ := os.ReadDir(outDir) - goFiles := 0 - for _, e := range entries { - if strings.HasSuffix(e.Name(), ".go") { - goFiles++ - } - } - if goFiles == 0 { - t.Fatal("no .go files generated") - } -} - -func TestGenerate_Good_ValidGoSyntax(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Read a generated file and verify basic Go syntax markers - data, err := os.ReadFile(filepath.Join(outDir, "repo.go")) - if err != nil { - // Try another name - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - if strings.HasSuffix(e.Name(), ".go") { - data, err = os.ReadFile(filepath.Join(outDir, e.Name())) - break - } - } - } - if err != nil { - t.Fatal(err) - } - - content := string(data) - if !strings.Contains(content, "package types") { - t.Error("missing package declaration") - } - if !strings.Contains(content, "// Code generated") { - t.Error("missing generated comment") - } -} - -func TestGenerate_Good_RepositoryType(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Find file containing Repository type - var content string - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - data, _ := os.ReadFile(filepath.Join(outDir, e.Name())) - if strings.Contains(string(data), "type Repository struct") { - content = string(data) - break - } - } - - if content == "" { - t.Fatal("Repository type not found in any generated file") - } - - // Check essential fields exist - checks := []string{ - "`json:\"id\"`", - "`json:\"name\"`", - "`json:\"full_name\"`", - "`json:\"private\"`", - } - for _, check := range checks { - if !strings.Contains(content, check) { - t.Errorf("missing field with tag %s", check) - } - } -} - -func TestGenerate_Good_TimeImport(t *testing.T) { - spec, err := LoadSpec("../../testdata/swagger.v1.json") - if err != nil { - t.Fatal(err) - } - - types := ExtractTypes(spec) - pairs := DetectCRUDPairs(spec) - - outDir := t.TempDir() - if err := Generate(types, pairs, outDir); err != nil { - t.Fatal(err) - } - - // Files with time.Time fields should import "time" - entries, _ := os.ReadDir(outDir) - for _, e := range entries { - data, _ := os.ReadFile(filepath.Join(outDir, e.Name())) - content := string(data) - if strings.Contains(content, "time.Time") && !strings.Contains(content, "\"time\"") { - t.Errorf("file %s uses time.Time but doesn't import time", e.Name()) - } - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestGenerate` -Expected: Failures (Generate is stub) - -**Step 3: Write generator.go** - -The generator groups types by logical domain and writes one `.go` file per group. Type grouping uses name prefixes and the CRUD pairs. - -```go -package main - -import ( - "fmt" - "os" - "path/filepath" - "sort" - "strings" - "text/template" -) - -// typeGrouping maps types to their output file. -var typeGrouping = map[string]string{ - "Repository": "repo", - "Repo": "repo", - "Issue": "issue", - "PullRequest": "pr", - "Pull": "pr", - "User": "user", - "Organization": "org", - "Org": "org", - "Team": "team", - "Label": "label", - "Milestone": "milestone", - "Release": "release", - "Tag": "tag", - "Branch": "branch", - "Hook": "hook", - "Deploy": "key", - "PublicKey": "key", - "GPGKey": "key", - "Key": "key", - "Notification": "notification", - "Package": "package", - "Action": "action", - "Commit": "commit", - "Git": "git", - "Contents": "content", - "File": "content", - "Wiki": "wiki", - "Comment": "comment", - "Review": "review", - "Reaction": "reaction", - "Topic": "topic", - "Status": "status", - "Combined": "status", - "Cron": "admin", - "Quota": "quota", - "OAuth2": "oauth", - "AccessToken": "oauth", - "API": "error", - "Forbidden": "error", - "NotFound": "error", - "NodeInfo": "federation", - "Activity": "activity", - "Feed": "activity", - "StopWatch": "time_tracking", - "TrackedTime": "time_tracking", - "Blocked": "user", - "Email": "user", - "Settings": "settings", - "GeneralAPI": "settings", - "GeneralAttachment": "settings", - "GeneralRepo": "settings", - "GeneralUI": "settings", - "Markdown": "misc", - "Markup": "misc", - "License": "misc", - "Gitignore": "misc", - "Annotated": "git", - "Note": "git", - "ChangedFile": "git", - "ExternalTracker": "repo", - "ExternalWiki": "repo", - "InternalTracker": "repo", - "Permission": "common", - "RepoTransfer": "repo", - "PayloadCommit": "hook", - "Dispatch": "action", - "Secret": "action", - "Variable": "action", - "Push": "repo", - "Mirror": "repo", - "Attachment": "common", - "EditDeadline": "issue", - "IssueDeadline": "issue", - "IssueLabels": "issue", - "IssueMeta": "issue", - "IssueTemplate": "issue", - "StateType": "common", - "TimeStamp": "common", - "Rename": "admin", - "Unadopted": "admin", -} - -// classifyType determines which file a type belongs in. -func classifyType(name string) string { - // Direct match - if group, ok := typeGrouping[name]; ok { - return group - } - - // Prefix match (longest first) - for prefix, group := range typeGrouping { - if strings.HasPrefix(name, prefix) { - return group - } - } - - // Try common suffixes - if strings.HasSuffix(name, "Option") || strings.HasSuffix(name, "Options") { - // Strip Create/Edit prefix to find base - trimmed := name - trimmed = strings.TrimPrefix(trimmed, "Create") - trimmed = strings.TrimPrefix(trimmed, "Edit") - trimmed = strings.TrimPrefix(trimmed, "Delete") - trimmed = strings.TrimPrefix(trimmed, "Update") - trimmed = strings.TrimSuffix(trimmed, "Option") - trimmed = strings.TrimSuffix(trimmed, "Options") - if group, ok := typeGrouping[trimmed]; ok { - return group - } - } - - return "misc" -} - -// Generate writes Go source files for all types. -func Generate(types map[string]*GoType, pairs []CRUDPair, outDir string) error { - if err := os.MkdirAll(outDir, 0755); err != nil { - return fmt.Errorf("create output dir: %w", err) - } - - // Group types by file - groups := make(map[string][]*GoType) - for _, gt := range types { - file := classifyType(gt.Name) - groups[file] = append(groups[file], gt) - } - - // Sort types within each group - for file := range groups { - sort.Slice(groups[file], func(i, j int) bool { - return groups[file][i].Name < groups[file][j].Name - }) - } - - // Write each file - for file, fileTypes := range groups { - if err := writeFile(filepath.Join(outDir, file+".go"), fileTypes); err != nil { - return fmt.Errorf("write %s.go: %w", file, err) - } - } - - return nil -} - -var fileTmpl = template.Must(template.New("file").Parse(`// Code generated by forgegen from swagger.v1.json — DO NOT EDIT. - -package types -{{if .NeedsTime}} -import "time" -{{end}} -{{range .Types}} -{{if .Description}}// {{.Name}} — {{.Description}}{{else}}// {{.Name}} represents a Forgejo API type.{{end}} -{{if .IsEnum}}type {{.Name}} string - -const ( -{{range .EnumValues}} {{$.EnumConst .Name .}} {{$.EnumType .Name}} = "{{.}}" -{{end}}) -{{else}}type {{.Name}} struct { -{{range .Fields}} {{.GoName}} {{.GoType}} ` + "`" + `json:"{{.JSONName}}{{if not .Required}},omitempty{{end}}"` + "`" + `{{if .Comment}} // {{.Comment}}{{end}} -{{end}}} -{{end}} -{{end}}`)) - -type fileData struct { - Types []*GoType - NeedsTime bool -} - -func (fd fileData) EnumConst(typeName, value string) string { - return typeName + pascalCase(value) -} - -func (fd fileData) EnumType(typeName string) string { - return typeName -} - -func writeFile(path string, types []*GoType) error { - needsTime := false - for _, gt := range types { - for _, f := range gt.Fields { - if strings.Contains(f.GoType, "time.Time") { - needsTime = true - break - } - } - if needsTime { - break - } - } - - f, err := os.Create(path) - if err != nil { - return err - } - defer f.Close() - - return fileTmpl.Execute(f, fileData{ - Types: types, - NeedsTime: needsTime, - }) -} -``` - -**Step 4: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v ./cmd/forgegen/ -run TestGenerate` -Expected: All 4 tests PASS - -**Step 5: Commit** - -```bash -git add cmd/forgegen/generator.go cmd/forgegen/generator_test.go -git commit -m "feat: Go source code generator from Swagger types - -Co-Authored-By: Virgil " -``` - ---- - -### Task 9: Generate types + verify compilation - -**Files:** -- Create: `types/` directory with generated files -- Create: `types/generate.go` (go:generate directive) - -**Step 1: Run the generator** - -```bash -cd /Users/snider/Code/go-forge -mkdir -p types -go run ./cmd/forgegen/ -spec testdata/swagger.v1.json -out types/ -``` - -**Step 2: Add go:generate directive** - -Create `types/generate.go`: -```go -package types - -//go:generate go run ../cmd/forgegen/ -spec ../testdata/swagger.v1.json -out . -``` - -**Step 3: Verify compilation** - -Run: `cd /Users/snider/Code/go-forge && go build ./types/` -Expected: Compiles without errors - -If there are compilation errors, fix the generator (`cmd/forgegen/generator.go`) and regenerate. Common issues: -- Missing imports (time) -- Duplicate field names (GoName collision) -- Invalid Go identifiers (reserved words, starting with numbers) - -**Step 4: Run all tests** - -Run: `cd /Users/snider/Code/go-forge && go test ./...` -Expected: All tests pass - -**Step 5: Commit** - -```bash -git add types/ -git commit -m "feat: generate all 229 Forgejo API types from swagger spec - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 3: Core Services (Tasks 10-13) - -Each service follows the same pattern: embed `Resource[T,C,U]`, add action methods. The first service (Task 10) is fully detailed as a template. Subsequent services follow the same structure with less repetition. - -### Task 10: Forge client + RepoService (template service) - -**Files:** -- Create: `forge.go` -- Create: `repos.go` -- Create: `forge_test.go` - -**Step 1: Write tests** - -```go -package forge - -import ( - "context" - "encoding/json" - "net/http" - "net/http/httptest" - "testing" - - "forge.lthn.ai/core/go-forge/types" -) - -func TestForge_Good_NewForge(t *testing.T) { - f := NewForge("https://forge.lthn.ai", "tok") - if f.Repos == nil { - t.Fatal("Repos service is nil") - } - if f.Issues == nil { - t.Fatal("Issues service is nil") - } -} - -func TestRepoService_Good_List(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - w.Header().Set("X-Total-Count", "1") - json.NewEncoder(w).Encode([]types.Repository{{Name: "go-forge"}}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - result, err := f.Repos.List(context.Background(), Params{"org": "core"}, DefaultList) - if err != nil { - t.Fatal(err) - } - if len(result.Items) != 1 || result.Items[0].Name != "go-forge" { - t.Errorf("unexpected result: %+v", result) - } -} - -func TestRepoService_Good_Get(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - json.NewEncoder(w).Encode(types.Repository{Name: "go-forge", FullName: "core/go-forge"}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - repo, err := f.Repos.Get(context.Background(), Params{"owner": "core", "repo": "go-forge"}) - if err != nil { - t.Fatal(err) - } - if repo.Name != "go-forge" { - t.Errorf("got name=%q", repo.Name) - } -} - -func TestRepoService_Good_Fork(t *testing.T) { - srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { - if r.Method != http.MethodPost { - t.Errorf("expected POST, got %s", r.Method) - } - w.WriteHeader(http.StatusAccepted) - json.NewEncoder(w).Encode(types.Repository{Name: "go-forge", Fork: true}) - })) - defer srv.Close() - - f := NewForge(srv.URL, "tok") - repo, err := f.Repos.Fork(context.Background(), "core", "go-forge", "my-org") - if err != nil { - t.Fatal(err) - } - if !repo.Fork { - t.Error("expected fork=true") - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestForge|TestRepoService"` -Expected: Compilation errors - -**Step 3: Write forge.go** - -```go -package forge - -import "forge.lthn.ai/core/go-forge/types" - -// Forge is the top-level client for the Forgejo API. -type Forge struct { - client *Client - - Repos *RepoService - Issues *IssueService - Pulls *PullService - Orgs *OrgService - Users *UserService - Teams *TeamService - Admin *AdminService - Branches *BranchService - Releases *ReleaseService - Labels *LabelService - Webhooks *WebhookService - Notifications *NotificationService - Packages *PackageService - Actions *ActionsService - Contents *ContentService - Wiki *WikiService - Misc *MiscService -} - -// NewForge creates a new Forge client. -func NewForge(url, token string, opts ...Option) *Forge { - c := NewClient(url, token, opts...) - f := &Forge{client: c} - f.Repos = newRepoService(c) - // Other services initialised in their respective tasks. - // Stub them here so tests compile: - f.Issues = &IssueService{} - f.Pulls = &PullService{} - f.Orgs = &OrgService{} - f.Users = &UserService{} - f.Teams = &TeamService{} - f.Admin = &AdminService{} - f.Branches = &BranchService{} - f.Releases = &ReleaseService{} - f.Labels = &LabelService{} - f.Webhooks = &WebhookService{} - f.Notifications = &NotificationService{} - f.Packages = &PackageService{} - f.Actions = &ActionsService{} - f.Contents = &ContentService{} - f.Wiki = &WikiService{} - f.Misc = &MiscService{} - return f -} - -// Client returns the underlying HTTP client. -func (f *Forge) Client() *Client { return f.client } -``` - -**Step 4: Write repos.go** - -```go -package forge - -import ( - "context" - - "forge.lthn.ai/core/go-forge/types" -) - -// RepoService handles repository operations. -type RepoService struct { - Resource[types.Repository, types.CreateRepoOption, types.EditRepoOption] -} - -func newRepoService(c *Client) *RepoService { - return &RepoService{ - Resource: *NewResource[types.Repository, types.CreateRepoOption, types.EditRepoOption]( - c, "/api/v1/repos/{owner}/{repo}", - ), - } -} - -// ListOrgRepos returns all repositories for an organisation. -func (s *RepoService) ListOrgRepos(ctx context.Context, org string) ([]types.Repository, error) { - return ListAll[types.Repository](ctx, s.client, "/api/v1/orgs/"+org+"/repos", nil) -} - -// ListUserRepos returns all repositories for the authenticated user. -func (s *RepoService) ListUserRepos(ctx context.Context) ([]types.Repository, error) { - return ListAll[types.Repository](ctx, s.client, "/api/v1/user/repos", nil) -} - -// Fork forks a repository. If org is non-empty, forks into that organisation. -func (s *RepoService) Fork(ctx context.Context, owner, repo, org string) (*types.Repository, error) { - body := map[string]string{} - if org != "" { - body["organization"] = org - } - var out types.Repository - err := s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/forks", body, &out) - if err != nil { - return nil, err - } - return &out, nil -} - -// Migrate imports a repository from an external service. -func (s *RepoService) Migrate(ctx context.Context, opts *types.MigrateRepoOptions) (*types.Repository, error) { - var out types.Repository - err := s.client.Post(ctx, "/api/v1/repos/migrate", opts, &out) - if err != nil { - return nil, err - } - return &out, nil -} - -// Transfer initiates a repository transfer. -func (s *RepoService) Transfer(ctx context.Context, owner, repo string, opts map[string]any) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer", opts, nil) -} - -// AcceptTransfer accepts a pending repository transfer. -func (s *RepoService) AcceptTransfer(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer/accept", nil, nil) -} - -// RejectTransfer rejects a pending repository transfer. -func (s *RepoService) RejectTransfer(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/transfer/reject", nil, nil) -} - -// MirrorSync triggers a mirror sync. -func (s *RepoService) MirrorSync(ctx context.Context, owner, repo string) error { - return s.client.Post(ctx, "/api/v1/repos/"+owner+"/"+repo+"/mirror-sync", nil, nil) -} -``` - -**Step 5: Write stub service types** so `forge.go` compiles. Create `services_stub.go`: - -```go -package forge - -// Stub service types — replaced as each service is implemented. - -type IssueService struct{} -type PullService struct{} -type OrgService struct{} -type UserService struct{} -type TeamService struct{} -type AdminService struct{} -type BranchService struct{} -type ReleaseService struct{} -type LabelService struct{} -type WebhookService struct{} -type NotificationService struct{} -type PackageService struct{} -type ActionsService struct{} -type ContentService struct{} -type WikiService struct{} -type MiscService struct{} -``` - -**Step 6: Run tests** - -Run: `cd /Users/snider/Code/go-forge && go test -v -run "TestForge|TestRepoService"` -Expected: All tests PASS (if generated types compile — if `types.CreateRepoOption` or `types.MigrateRepoOptions` don't exist, adjust field names to match generated types) - -**Step 7: Commit** - -```bash -git add forge.go repos.go services_stub.go forge_test.go -git commit -m "feat: Forge client + RepoService with CRUD and actions - -Co-Authored-By: Virgil " -``` - ---- - -### Task 11: IssueService + PullService - -**Files:** -- Create: `issues.go` -- Create: `pulls.go` -- Create: `issues_test.go` -- Create: `pulls_test.go` -- Modify: `forge.go` (wire up services) -- Modify: `services_stub.go` (remove IssueService, PullService stubs) - -Follow the same pattern as Task 10. Key points: - -**IssueService** embeds `Resource[types.Issue, types.CreateIssueOption, types.EditIssueOption]`. -Path: `/api/v1/repos/{owner}/{repo}/issues/{index}` - -Action methods (9): -- `Pin(ctx, owner, repo, index)` — POST `.../issues/{index}/pin` -- `Unpin(ctx, owner, repo, index)` — DELETE `.../issues/{index}/pin` -- `SetDeadline(ctx, owner, repo, index, deadline)` — POST `.../issues/{index}/deadline` -- `AddReaction(ctx, owner, repo, index, reaction)` — POST `.../issues/{index}/reactions` -- `DeleteReaction(ctx, owner, repo, index, reaction)` — DELETE `.../issues/{index}/reactions` -- `StartStopwatch(ctx, owner, repo, index)` — POST `.../issues/{index}/stopwatch/start` -- `StopStopwatch(ctx, owner, repo, index)` — POST `.../issues/{index}/stopwatch/stop` -- `AddLabels(ctx, owner, repo, index, labelIDs)` — POST `.../issues/{index}/labels` -- `RemoveLabel(ctx, owner, repo, index, labelID)` — DELETE `.../issues/{index}/labels/{id}` -- `ListComments(ctx, owner, repo, index)` — GET `.../issues/{index}/comments` -- `CreateComment(ctx, owner, repo, index, body)` — POST `.../issues/{index}/comments` - -**PullService** embeds `Resource[types.PullRequest, types.CreatePullRequestOption, types.EditPullRequestOption]`. -Path: `/api/v1/repos/{owner}/{repo}/pulls/{index}` - -Action methods (6): -- `Merge(ctx, owner, repo, index, method)` — POST `.../pulls/{index}/merge` -- `Update(ctx, owner, repo, index)` — POST `.../pulls/{index}/update` -- `ListReviews(ctx, owner, repo, index)` — GET `.../pulls/{index}/reviews` -- `SubmitReview(ctx, owner, repo, index, reviewID)` — POST `.../pulls/{index}/reviews/{id}` -- `DismissReview(ctx, owner, repo, index, reviewID, msg)` — POST `.../pulls/{index}/reviews/{id}/dismissals` -- `UndismissReview(ctx, owner, repo, index, reviewID)` — POST `.../pulls/{index}/reviews/{id}/undismissals` - -Write tests for at least: List, Get, Create for each service + one action method each. - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: IssueService and PullService with actions"` - ---- - -### Task 12: OrgService + TeamService + UserService - -**Files:** -- Create: `orgs.go`, `teams.go`, `users.go` -- Create: `orgs_test.go`, `teams_test.go`, `users_test.go` -- Modify: `forge.go` (wire up) -- Modify: `services_stub.go` (remove stubs) - -**OrgService** — `Resource[types.Organization, types.CreateOrgOption, types.EditOrgOption]` -Path: `/api/v1/orgs/{org}` -Actions: ListMembers, AddMember, RemoveMember, SetAvatar, Block, Unblock - -**TeamService** — `Resource[types.Team, types.CreateTeamOption, types.EditTeamOption]` -Path: `/api/v1/teams/{id}` -Actions: ListMembers, AddMember, RemoveMember, ListRepos, AddRepo, RemoveRepo - -**UserService** — `Resource[types.User, struct{}, struct{}]` (no create/edit via this path) -Path: `/api/v1/users/{username}` -Custom: `GetCurrent(ctx)`, `ListFollowers(ctx)`, `ListStarred(ctx)`, keys, GPG keys, settings - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: OrgService, TeamService, UserService"` - ---- - -### Task 13: AdminService - -**Files:** -- Create: `admin.go` -- Create: `admin_test.go` -- Modify: `forge.go` (wire up) -- Modify: `services_stub.go` (remove stub) - -**AdminService** — No generic Resource (admin endpoints are heterogeneous). -Direct methods: -- `ListUsers(ctx)` — GET `/api/v1/admin/users` -- `CreateUser(ctx, opts)` — POST `/api/v1/admin/users` -- `EditUser(ctx, username, opts)` — PATCH `/api/v1/admin/users/{username}` -- `DeleteUser(ctx, username)` — DELETE `/api/v1/admin/users/{username}` -- `RenameUser(ctx, username, newName)` — POST `.../users/{username}/rename` -- `ListOrgs(ctx)` — GET `/api/v1/admin/orgs` -- `RunCron(ctx, task)` — POST `/api/v1/admin/cron/{task}` -- `ListCron(ctx)` — GET `/api/v1/admin/cron` -- `AdoptRepo(ctx, owner, repo)` — POST `.../unadopted/{owner}/{repo}` -- `GenerateRunnerToken(ctx)` — POST `/api/v1/admin/runners/registration-token` - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v` -Commit: `git commit -m "feat: AdminService with user, org, cron, runner operations"` - ---- - -## Wave 4: Extended Services (Tasks 14-17) - -### Task 14: BranchService + ReleaseService - -**BranchService** — `Resource[types.Branch, types.CreateBranchRepoOption, struct{}]` -Path: `/api/v1/repos/{owner}/{repo}/branches/{branch}` -Additional: BranchProtection CRUD at `.../branch_protections/{name}` - -**ReleaseService** — `Resource[types.Release, types.CreateReleaseOption, types.EditReleaseOption]` -Path: `/api/v1/repos/{owner}/{repo}/releases/{id}` -Additional: Asset upload/download at `.../releases/{id}/assets` - -### Task 15: LabelService + WebhookService + ContentService - -**LabelService** — Handles repo labels, org labels, and issue labels. -- `ListRepoLabels(ctx, owner, repo)` -- `CreateRepoLabel(ctx, owner, repo, opts)` -- `ListOrgLabels(ctx, org)` - -**WebhookService** — `Resource[types.Hook, types.CreateHookOption, types.EditHookOption]` -Actions: `TestHook(ctx, owner, repo, id)` - -**ContentService** — File read/write via API -- `GetFile(ctx, owner, repo, path)` — GET `.../contents/{path}` -- `CreateFile(ctx, owner, repo, path, opts)` — POST `.../contents/{path}` -- `UpdateFile(ctx, owner, repo, path, opts)` — PUT `.../contents/{path}` -- `DeleteFile(ctx, owner, repo, path, opts)` — DELETE `.../contents/{path}` - -### Task 16: ActionsService + NotificationService + PackageService - -**ActionsService** — runners, secrets, variables, workflow dispatch -- Repo-level: `.../repos/{owner}/{repo}/actions/{secrets,variables,runners}` -- Org-level: `.../orgs/{org}/actions/{secrets,variables,runners}` -- `DispatchWorkflow(ctx, owner, repo, workflow, opts)` - -**NotificationService** — list, mark read -- `List(ctx)` — GET `/api/v1/notifications` -- `MarkRead(ctx)` — PUT `/api/v1/notifications` -- `GetThread(ctx, id)` — GET `.../notifications/threads/{id}` - -**PackageService** — list, get, delete -- `List(ctx, owner)` — GET `/api/v1/packages/{owner}` -- `Get(ctx, owner, type, name, version)` — GET `.../packages/{owner}/{type}/{name}/{version}` - -### Task 17: WikiService + MiscService + CommitService - -**WikiService** — pages -- `ListPages(ctx, owner, repo)` -- `GetPage(ctx, owner, repo, pageName)` -- `CreatePage(ctx, owner, repo, opts)` -- `EditPage(ctx, owner, repo, pageName, opts)` -- `DeletePage(ctx, owner, repo, pageName)` - -**MiscService** — markdown, licenses, gitignore, nodeinfo -- `RenderMarkdown(ctx, text, mode)` — POST `/api/v1/markdown` -- `ListLicenses(ctx)` — GET `/api/v1/licenses` -- `ListGitignoreTemplates(ctx)` — GET `/api/v1/gitignore/templates` -- `NodeInfo(ctx)` — GET `/api/v1/nodeinfo` - -**CommitService** — status and notes -- `GetCombinedStatus(ctx, owner, repo, ref)` -- `CreateStatus(ctx, owner, repo, sha, opts)` -- `SetNote(ctx, owner, repo, sha, opts)` - -For each task in Wave 4: write tests first, implement, verify all tests pass, commit. - -Run after each task: `cd /Users/snider/Code/go-forge && go test ./... -v` - ---- - -## Wave 5: Clean Up + Services Stub Removal (Task 18) - -### Task 18: Remove stubs + final wiring - -**Files:** -- Delete: `services_stub.go` -- Modify: `forge.go` — replace all stub initialisations with real `newXxxService(c)` calls - -**Step 1: Remove services_stub.go** - -Delete the file. All service types should now be defined in their own files. - -**Step 2: Wire all services in forge.go** - -Update `NewForge()` to call `newXxxService(c)` for every service. - -**Step 3: Run all tests** - -Run: `cd /Users/snider/Code/go-forge && go test ./... -v -count=1` -Expected: All tests pass - -**Step 4: Commit** - -```bash -git add -A -git commit -m "feat: wire all 17 services, remove stubs - -Co-Authored-By: Virgil " -``` - ---- - -## Wave 6: Integration + Forge Repo Setup (Tasks 19-20) - -### Task 19: Create Forge repo + push - -**Step 1: Create repo on Forge** - -Use the Forgejo API or web UI to create `core/go-forge` on `forge.lthn.ai`. - -**Step 2: Add remote and push** - -```bash -cd /Users/snider/Code/go-forge -git remote add forge ssh://git@forge.lthn.ai:2223/core/go-forge.git -git push -u forge main -``` - -### Task 20: Wiki documentation (go-ai treatment) - -Create wiki pages for go-forge on Forge, matching the go-ai documentation pattern: - -1. **Home** — Overview, install, quick start -2. **Architecture** — Generic Resource[T,C,U], codegen pipeline, service pattern -3. **Services** — All 17 services with example usage -4. **Code Generation** — How to regenerate types, upgrade Forgejo version -5. **Configuration** — Env vars, config file, flags -6. **Error Handling** — APIError, IsNotFound, IsForbidden -7. **Development** — Contributing, testing, releasing - -Use the Forge wiki API: `POST /api/v1/repos/core/go-forge/wiki/new` with `{"content_base64":"...","title":"..."}`. - ---- - -## Dependency Sequencing - -``` -Task 1 (scaffold) ← Task 2 (client) ← Task 3 (pagination) ← Task 4 (params) ← Task 5 (resource) -Task 1 ← Task 7 (parser) ← Task 8 (generator) ← Task 9 (generate types) -Task 5 + Task 9 ← Task 6 (config) ← Task 10 (forge + repos) -Task 10 ← Task 11 (issues + PRs) -Task 10 ← Task 12 (orgs + teams + users) -Task 10 ← Task 13 (admin) -Task 10 ← Task 14-17 (extended services) -Task 14-17 ← Task 18 (remove stubs) -Task 18 ← Task 19 (forge push) -Task 19 ← Task 20 (wiki) -``` - -**Wave 1 (Tasks 1-6)**: Foundation — all independent once scaffolded -**Wave 2 (Tasks 7-9)**: Codegen — sequential (parser → generator → run) -**Wave 3 (Tasks 10-13)**: Core services — Task 10 first (creates Forge + stubs), then 11-13 parallel -**Wave 4 (Tasks 14-17)**: Extended services — all parallel after Task 10 -**Wave 5 (Task 18)**: Clean up — after all services done -**Wave 6 (Tasks 19-20)**: Ship — after clean up - -## Verification - -After all tasks: - -1. `cd /Users/snider/Code/go-forge && go test ./... -count=1` — all pass -2. `go build ./...` — compiles cleanly -3. `go vet ./...` — no issues -4. Verify `types/` contains generated files with `Repository`, `Issue`, `PullRequest`, etc. -5. Verify `NewForge()` creates client with all 17 services populated -6. Verify action methods exist (Fork, Merge, Pin, etc.) diff --git a/docs/plans/completed/2026-02-22-frame-bubbletea-design-original.md b/docs/plans/completed/2026-02-22-frame-bubbletea-design-original.md deleted file mode 100644 index 7f55bd7..0000000 --- a/docs/plans/completed/2026-02-22-frame-bubbletea-design-original.md +++ /dev/null @@ -1,209 +0,0 @@ -# Frame Bubbletea Upgrade Design - -**Issue:** core/go#15 -**Date:** 2026-02-22 -**Status:** Approved - -**Goal:** Upgrade `cli.Frame` from raw ANSI + `golang.org/x/term` to bubbletea internally, adding keyboard navigation, focus management, and lipgloss layout composition while preserving the existing public API. - ---- - -## Architecture - -Single ownership model. Frame becomes the sole `tea.Model` wrapping a `tea.Program`. It owns the terminal (alt-screen, raw mode, resize events, input). Region models never touch the terminal directly. - -Message routing: -- **Key messages** — routed to the focused region's `FrameModel.Update()` only -- **Tick/resize messages** — broadcast to all region `FrameModel.Update()` calls -- **Custom messages** — broadcast to all (enables cross-region communication) - -Dual interface pattern: - -```go -// Existing — view-only, no changes -type Model interface { - View(width, height int) string -} - -// New — interactive components -type FrameModel interface { - Model - Init() tea.Cmd - Update(tea.Msg) (FrameModel, tea.Cmd) -} -``` - -Frame wraps plain `Model` in a no-op adapter internally, so existing code (StatusLine, KeyHints, Breadcrumb, StaticModel, ModelFunc) works without changes. - -Layout composition replaces the manual ANSI cursor/clear dance in `runLive()` with lipgloss `JoinVertical` and `JoinHorizontal`. The existing HLCRF variant parser and region size calculations stay, but rendering uses lipgloss instead of raw escape codes. - ---- - -## Focus Management - -Focus ring. Frame maintains an ordered list of focusable regions (only regions with `FrameModel` components). Focus cycles through them. - -Navigation: -- `Tab` / `Shift-Tab` — cycle focus forward/backward through the ring -- Arrow keys — spatial navigation (up to Header, down to Footer, left to Left sidebar, right to Right sidebar) -- Configurable via `KeyMap` struct with sensible defaults - -```go -type KeyMap struct { - FocusNext key.Binding // Tab - FocusPrev key.Binding // Shift-Tab - FocusUp key.Binding // Up (to Header from Content) - FocusDown key.Binding // Down (to Footer from Content) - FocusLeft key.Binding // Left (to Left sidebar) - FocusRight key.Binding // Right (to Right sidebar) - Quit key.Binding // q, Ctrl-C - Back key.Binding // Esc (triggers Navigate back) -} -``` - -Visual feedback: focused region gets a subtle border highlight (configurable via lipgloss border styling). Unfocused regions render normally. - -Key filtering: focus keys are consumed by Frame and never forwarded to region models. All other keys go to the focused region's `Update()`. - ---- - -## Public API - -### Preserved (no changes) - -- `NewFrame(variant string) *Frame` -- `Header(m Model)`, `Left(m Model)`, `Content(m Model)`, `Right(m Model)`, `Footer(m Model)` -- `Navigate(m Model)`, `Back() bool` -- `Run()`, `RunFor(d time.Duration)`, `Stop()` -- `String()` — static render for non-TTY -- `ModelFunc`, `StaticModel`, `StatusLine`, `KeyHints`, `Breadcrumb` - -### New additions - -```go -// WithKeyMap sets custom key bindings for Frame navigation. -func (f *Frame) WithKeyMap(km KeyMap) *Frame - -// Focused returns the currently focused region. -func (f *Frame) Focused() Region - -// Focus sets focus to a specific region. -func (f *Frame) Focus(r Region) - -// Send injects a message into the Frame's tea.Program. -// Useful for triggering updates from external goroutines. -func (f *Frame) Send(msg tea.Msg) -``` - -### Behavioural changes - -- `Run()` now starts a `tea.Program` in TTY mode (instead of raw ticker loop) -- Non-TTY path unchanged — still calls `String()` and returns -- `RunFor()` unchanged — uses `Stop()` after timer - -### New dependencies - -- `github.com/charmbracelet/bubbletea` (already in core/go) -- `github.com/charmbracelet/lipgloss` (already in core/go) -- `github.com/charmbracelet/bubbles/key` (key bindings) - ---- - -## Internal Implementation - -Frame implements `tea.Model`: - -```go -func (f *Frame) Init() tea.Cmd -func (f *Frame) Update(tea.Msg) (tea.Model, tea.Cmd) -func (f *Frame) View() string -``` - -`Init()` collects `Init()` from all `FrameModel` regions via `tea.Batch()`. - -`Update()` handles: -1. `tea.WindowSizeMsg` — update dimensions, broadcast to all FrameModels -2. `tea.KeyMsg` matching focus keys — advance/retreat focus ring -3. `tea.KeyMsg` matching quit — return `tea.Quit` -4. `tea.KeyMsg` matching back — call `Back()`, return nil -5. All other `tea.KeyMsg` — forward to focused region's `Update()` -6. All other messages — broadcast to all FrameModels - -`View()` uses lipgloss composition: - -``` -header = renderRegion(H, width, 1) -footer = renderRegion(F, width, 1) -middleH = height - headerH - footerH - -left = renderRegion(L, width/4, middleH) -right = renderRegion(R, width/4, middleH) -content = renderRegion(C, contentW, middleH) - -middle = lipgloss.JoinHorizontal(Top, left, content, right) -output = lipgloss.JoinVertical(Left, header, middle, footer) -``` - -`Run()` change: - -```go -func (f *Frame) Run() { - if !f.isTTY() { - fmt.Fprint(f.out, f.String()) - return - } - p := tea.NewProgram(f, tea.WithAltScreen()) - f.program = p - if _, err := p.Run(); err != nil { - Fatal(err) - } -} -``` - -Plain `Model` adapter: - -```go -type modelAdapter struct{ m Model } -func (a *modelAdapter) Init() tea.Cmd { return nil } -func (a *modelAdapter) Update(tea.Msg) (FrameModel, tea.Cmd) { return a, nil } -func (a *modelAdapter) View(w, h int) string { return a.m.View(w, h) } -``` - ---- - -## Testing Strategy - -Existing 14 tests preserved. They use `bytes.Buffer` (non-TTY path), bypassing bubbletea. - -New tests for interactive features: -- Focus cycling: Tab advances focus, Shift-Tab goes back -- Spatial navigation: arrow keys move focus to correct region -- Message routing: key events only reach focused model -- Tick broadcast: tick events reach all models -- Resize propagation: resize reaches all models -- FrameModel lifecycle: Init() called on Run(), Update() receives messages -- Adapter: plain Model wrapped correctly, receives no Update calls -- Navigate/Back with FrameModel: focus transfers correctly -- KeyMap customization: overridden bindings work -- Send(): external messages delivered to models - -Testing approach: use bubbletea's `teatest` package for interactive tests. Non-TTY tests stay as-is with `bytes.Buffer`. - ---- - -## Files Affected - -| File | Action | Purpose | -|------|--------|---------| -| `pkg/cli/frame.go` | modify | Add bubbletea tea.Model implementation, lipgloss layout, focus management | -| `pkg/cli/frame_model.go` | new | FrameModel interface, modelAdapter, KeyMap | -| `pkg/cli/frame_test.go` | modify | Add interactive tests alongside existing ones | -| `go.mod` | modify | Add bubbletea, lipgloss, bubbles dependencies | - -## Design Decisions - -1. **Frame as tea.Model, not wrapping separate tea.Model** — Frame IS the model, simplest ownership -2. **Dual interface (Model + FrameModel)** — backward compatible, existing components unchanged -3. **Lipgloss for layout** — replaces manual ANSI, consistent with bubbletea ecosystem -4. **Focus ring with spatial override** — Tab for cycling, arrows for direct spatial jumps -5. **Non-TTY path untouched** — `String()` and non-TTY `Run()` stay exactly as-is diff --git a/docs/plans/completed/2026-02-22-frame-bubbletea-plan-original.md b/docs/plans/completed/2026-02-22-frame-bubbletea-plan-original.md deleted file mode 100644 index 4f70c16..0000000 --- a/docs/plans/completed/2026-02-22-frame-bubbletea-plan-original.md +++ /dev/null @@ -1,1335 +0,0 @@ -# Frame Bubbletea Upgrade Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Upgrade `cli.Frame` from raw ANSI + `golang.org/x/term` to bubbletea internally, adding keyboard focus management and lipgloss layout, while preserving the existing public API. - -**Architecture:** Frame implements `tea.Model` internally and owns a single `tea.Program`. A dual interface pattern keeps the existing `Model` (view-only) working alongside a new `FrameModel` (interactive). Lipgloss replaces manual ANSI escape codes for layout composition. - -**Tech Stack:** Go 1.26, bubbletea v1.3.10, lipgloss v1.1.0, existing HLCRF layout parser - ---- - -## Important Context - -**Repo:** `~/Code/core/cli` (module `forge.lthn.ai/core/cli`) - -**Workspace:** `~/Code/go.work` — Go workspace with 29 modules. Run all commands from `~/Code/core/cli/`. - -**Run tests:** `go test -race ./pkg/cli/` (always from `~/Code/core/cli/`) - -**Design doc:** `docs/plans/2026-02-22-frame-bubbletea-design.md` - -**Key files you'll touch:** -- `pkg/cli/frame.go` — current Frame (359 lines, raw ANSI rendering) -- `pkg/cli/frame_model.go` — **new** file for FrameModel interface, KeyMap, adapter -- `pkg/cli/frame_test.go` — existing 14 tests (must all keep passing) -- `go.mod` — add bubbletea + lipgloss deps - -**Key files to read (don't modify):** -- `pkg/cli/layout.go` — HLCRF variant parser (`Region` type, `Composite` struct) -- `pkg/cli/ansi.go` — `AnsiStyle`, `SetColorEnabled()`, `ColorEnabled()` -- `pkg/cli/styles.go` — `BoldStyle`, `DimStyle`, `Truncate()`, `Pad()` - -**bubbletea API (v1.3.10) cheatsheet:** -- `tea.Model` interface: `Init() tea.Cmd`, `Update(tea.Msg) (tea.Model, tea.Cmd)`, `View() string` -- `tea.NewProgram(model, opts...)` — creates program -- `tea.WithAltScreen()` — fullscreen mode -- `tea.WithOutput(io.Writer)` — custom output -- `tea.Batch(cmds...)` — combine commands -- `tea.Quit()` — exit command -- `tea.KeyMsg` — has `.Type` (KeyType) and `.String()` method -- Key constants: `tea.KeyTab`, `tea.KeyShiftTab`, `tea.KeyUp`, `tea.KeyDown`, `tea.KeyLeft`, `tea.KeyRight`, `tea.KeyEsc`, `tea.KeyCtrlC` -- `tea.WindowSizeMsg` — has `.Width`, `.Height` -- `program.Send(msg)` — inject message from outside -- `program.Quit()` — stop program - -**lipgloss API (v1.1.0) cheatsheet:** -- `lipgloss.JoinVertical(pos, strs...)` — stack strings vertically -- `lipgloss.JoinHorizontal(pos, strs...)` — join strings side-by-side -- `lipgloss.Place(w, h, hPos, vPos, str)` — place string in box -- Constants: `lipgloss.Left`, `lipgloss.Right`, `lipgloss.Center`, `lipgloss.Top`, `lipgloss.Bottom` -- `lipgloss.NewStyle().Width(n).Height(n).Render(str)` — constrain to dimensions - ---- - -### Task 1: Add bubbletea and lipgloss dependencies - -**Files:** -- Modify: `go.mod` - -**Step 1: Add the dependencies** - -Run from `~/Code/core/cli/`: - -```bash -go get github.com/charmbracelet/bubbletea@v1.3.10 -go get github.com/charmbracelet/lipgloss@v1.1.0 -``` - -**Step 2: Tidy** - -```bash -go mod tidy -``` - -**Step 3: Verify existing tests still pass** - -Run: `go test -race ./pkg/cli/` -Expected: PASS (all 14 existing tests unchanged) - -**Step 4: Commit** - -```bash -git add go.mod go.sum -git commit -m "deps: add bubbletea and lipgloss for Frame upgrade" -``` - ---- - -### Task 2: Create FrameModel interface and modelAdapter - -**Files:** -- Create: `pkg/cli/frame_model.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the failing test** - -Add to `pkg/cli/frame_test.go` at the bottom: - -```go -func TestFrameModel_Good(t *testing.T) { - t.Run("modelAdapter wraps plain Model", func(t *testing.T) { - m := StaticModel("hello") - adapted := adaptModel(m) - - // Should return nil cmd from Init - cmd := adapted.Init() - assert.Nil(t, cmd) - - // Should return itself from Update - updated, cmd := adapted.Update(nil) - assert.Equal(t, adapted, updated) - assert.Nil(t, cmd) - - // Should delegate View to wrapped model - assert.Equal(t, "hello", adapted.View(80, 24)) - }) - - t.Run("FrameModel passes through without wrapping", func(t *testing.T) { - fm := &testFrameModel{viewText: "interactive"} - adapted := adaptModel(fm) - - // Should be the same object, not wrapped - _, ok := adapted.(*testFrameModel) - assert.True(t, ok, "FrameModel should not be wrapped") - assert.Equal(t, "interactive", adapted.View(80, 24)) - }) -} - -// testFrameModel is a mock FrameModel for testing. -type testFrameModel struct { - viewText string - initCalled bool - updateCalled bool - lastMsg tea.Msg -} - -func (m *testFrameModel) View(w, h int) string { return m.viewText } - -func (m *testFrameModel) Init() tea.Cmd { - m.initCalled = true - return nil -} - -func (m *testFrameModel) Update(msg tea.Msg) (FrameModel, tea.Cmd) { - m.updateCalled = true - m.lastMsg = msg - return m, nil -} -``` - -You'll need to add `tea "github.com/charmbracelet/bubbletea"` to the test file's imports. - -**Step 2: Run test to verify it fails** - -Run: `go test -race -run TestFrameModel ./pkg/cli/` -Expected: FAIL — `adaptModel` undefined, `FrameModel` undefined, `testFrameModel` can't satisfy unwritten interface - -**Step 3: Write the implementation** - -Create `pkg/cli/frame_model.go`: - -```go -package cli - -import tea "github.com/charmbracelet/bubbletea" - -// FrameModel extends Model with bubbletea lifecycle methods. -// Use this for interactive components that handle input. -// Plain Model components work unchanged — Frame wraps them automatically. -type FrameModel interface { - Model - Init() tea.Cmd - Update(tea.Msg) (FrameModel, tea.Cmd) -} - -// adaptModel wraps a plain Model as a FrameModel via modelAdapter. -// If the model already implements FrameModel, it is returned as-is. -func adaptModel(m Model) FrameModel { - if fm, ok := m.(FrameModel); ok { - return fm - } - return &modelAdapter{m: m} -} - -// modelAdapter wraps a plain Model to satisfy FrameModel. -// Init returns nil, Update is a no-op, View delegates to the wrapped Model. -type modelAdapter struct { - m Model -} - -func (a *modelAdapter) View(w, h int) string { return a.m.View(w, h) } -func (a *modelAdapter) Init() tea.Cmd { return nil } -func (a *modelAdapter) Update(tea.Msg) (FrameModel, tea.Cmd) { return a, nil } -``` - -**Step 4: Run test to verify it passes** - -Run: `go test -race -run TestFrameModel ./pkg/cli/` -Expected: PASS - -**Step 5: Run all tests to verify no regressions** - -Run: `go test -race ./pkg/cli/` -Expected: PASS (all existing + new tests) - -**Step 6: Commit** - -```bash -git add pkg/cli/frame_model.go pkg/cli/frame_test.go -git commit -m "feat(frame): add FrameModel interface and modelAdapter" -``` - ---- - -### Task 3: Add KeyMap struct with defaults - -**Files:** -- Modify: `pkg/cli/frame_model.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the failing test** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestKeyMap_Good(t *testing.T) { - t.Run("default keymap has expected bindings", func(t *testing.T) { - km := DefaultKeyMap() - assert.Equal(t, tea.KeyTab, km.FocusNext) - assert.Equal(t, tea.KeyShiftTab, km.FocusPrev) - assert.Equal(t, tea.KeyUp, km.FocusUp) - assert.Equal(t, tea.KeyDown, km.FocusDown) - assert.Equal(t, tea.KeyLeft, km.FocusLeft) - assert.Equal(t, tea.KeyRight, km.FocusRight) - assert.Equal(t, tea.KeyEsc, km.Back) - assert.Equal(t, tea.KeyCtrlC, km.Quit) - }) -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -race -run TestKeyMap ./pkg/cli/` -Expected: FAIL — `DefaultKeyMap` undefined, `KeyMap` undefined - -**Step 3: Write the implementation** - -Add to `pkg/cli/frame_model.go`: - -```go -// KeyMap defines key bindings for Frame navigation. -// Use DefaultKeyMap() for sensible defaults, or build your own. -type KeyMap struct { - FocusNext tea.KeyType // Tab — cycle focus forward - FocusPrev tea.KeyType // Shift-Tab — cycle focus backward - FocusUp tea.KeyType // Up — spatial: move to Header - FocusDown tea.KeyType // Down — spatial: move to Footer - FocusLeft tea.KeyType // Left — spatial: move to Left sidebar - FocusRight tea.KeyType // Right — spatial: move to Right sidebar - Back tea.KeyType // Esc — Navigate back - Quit tea.KeyType // Ctrl-C — quit -} - -// DefaultKeyMap returns the standard Frame key bindings. -func DefaultKeyMap() KeyMap { - return KeyMap{ - FocusNext: tea.KeyTab, - FocusPrev: tea.KeyShiftTab, - FocusUp: tea.KeyUp, - FocusDown: tea.KeyDown, - FocusLeft: tea.KeyLeft, - FocusRight: tea.KeyRight, - Back: tea.KeyEsc, - Quit: tea.KeyCtrlC, - } -} -``` - -**Step 4: Run test to verify it passes** - -Run: `go test -race -run TestKeyMap ./pkg/cli/` -Expected: PASS - -**Step 5: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/cli/frame_model.go pkg/cli/frame_test.go -git commit -m "feat(frame): add KeyMap with default bindings" -``` - ---- - -### Task 4: Add focus management fields to Frame - -**Files:** -- Modify: `pkg/cli/frame.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the failing tests** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameFocus_Good(t *testing.T) { - t.Run("default focus is Content", func(t *testing.T) { - f := NewFrame("HCF") - assert.Equal(t, RegionContent, f.Focused()) - }) - - t.Run("Focus sets focused region", func(t *testing.T) { - f := NewFrame("HCF") - f.Focus(RegionHeader) - assert.Equal(t, RegionHeader, f.Focused()) - }) - - t.Run("Focus ignores invalid region", func(t *testing.T) { - f := NewFrame("HCF") - f.Focus(RegionLeft) // Left not in "HCF" - assert.Equal(t, RegionContent, f.Focused()) // unchanged - }) - - t.Run("WithKeyMap returns frame for chaining", func(t *testing.T) { - km := DefaultKeyMap() - km.Quit = tea.KeyCtrlQ - f := NewFrame("HCF").WithKeyMap(km) - assert.Equal(t, tea.KeyCtrlQ, f.keyMap.Quit) - }) - - t.Run("focusRing builds from variant", func(t *testing.T) { - f := NewFrame("HLCRF") - ring := f.buildFocusRing() - assert.Equal(t, []Region{RegionHeader, RegionLeft, RegionContent, RegionRight, RegionFooter}, ring) - }) - - t.Run("focusRing respects variant order", func(t *testing.T) { - f := NewFrame("HCF") - ring := f.buildFocusRing() - assert.Equal(t, []Region{RegionHeader, RegionContent, RegionFooter}, ring) - }) -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -race -run TestFrameFocus ./pkg/cli/` -Expected: FAIL — `Focused()`, `Focus()`, `WithKeyMap()`, `buildFocusRing()` all undefined - -**Step 3: Write the implementation** - -Modify `pkg/cli/frame.go`. Add new fields to the `Frame` struct: - -```go -type Frame struct { - variant string - layout *Composite - models map[Region]Model - history []Model // content region stack for Navigate/Back - out io.Writer - done chan struct{} - mu sync.Mutex - - // Focus management (bubbletea upgrade) - focused Region - keyMap KeyMap - width int - height int - program *tea.Program -} -``` - -Add `tea "github.com/charmbracelet/bubbletea"` to frame.go imports (alongside existing ones). You do NOT need to import lipgloss yet. - -Update `NewFrame` to initialise new fields: - -```go -func NewFrame(variant string) *Frame { - return &Frame{ - variant: variant, - layout: Layout(variant), - models: make(map[Region]Model), - out: os.Stdout, - done: make(chan struct{}), - focused: RegionContent, - keyMap: DefaultKeyMap(), - width: 80, - height: 24, - } -} -``` - -Add the new public methods: - -```go -// WithKeyMap sets custom key bindings for Frame navigation. -func (f *Frame) WithKeyMap(km KeyMap) *Frame { - f.keyMap = km - return f -} - -// Focused returns the currently focused region. -func (f *Frame) Focused() Region { - f.mu.Lock() - defer f.mu.Unlock() - return f.focused -} - -// Focus sets focus to a specific region. -// Ignores the request if the region is not in this Frame's variant. -func (f *Frame) Focus(r Region) { - f.mu.Lock() - defer f.mu.Unlock() - if _, exists := f.layout.regions[r]; exists { - f.focused = r - } -} - -// buildFocusRing returns the ordered list of regions in this Frame's variant. -// Order follows HLCRF convention. -func (f *Frame) buildFocusRing() []Region { - order := []Region{RegionHeader, RegionLeft, RegionContent, RegionRight, RegionFooter} - var ring []Region - for _, r := range order { - if _, exists := f.layout.regions[r]; exists { - ring = append(ring, r) - } - } - return ring -} -``` - -**Step 4: Run test to verify it passes** - -Run: `go test -race -run TestFrameFocus ./pkg/cli/` -Expected: PASS - -**Step 5: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/cli/frame.go pkg/cli/frame_test.go -git commit -m "feat(frame): add focus management fields, Focused(), Focus(), WithKeyMap()" -``` - ---- - -### Task 5: Implement tea.Model on Frame (Init, Update, View) - -This is the core task. Frame becomes a `tea.Model`. The existing `runLive()` and `renderFrame()` methods get replaced. - -**Files:** -- Modify: `pkg/cli/frame.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the failing tests** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameTeaModel_Good(t *testing.T) { - t.Run("Init collects FrameModel inits", func(t *testing.T) { - f := NewFrame("HCF") - fm := &testFrameModel{viewText: "x"} - f.Content(fm) - - cmd := f.Init() - // Should produce a batch command (non-nil if any FrameModel has Init) - // fm.Init returns nil, so batch of nils = nil - _ = cmd // no panic = success - assert.True(t, fm.initCalled) - }) - - t.Run("Update routes key to focused region", func(t *testing.T) { - f := NewFrame("HCF") - header := &testFrameModel{viewText: "h"} - content := &testFrameModel{viewText: "c"} - f.Header(header) - f.Content(content) - - // Focus is Content by default - keyMsg := tea.KeyMsg{Type: tea.KeyRunes, Runes: []rune{'a'}} - f.Update(keyMsg) - - assert.True(t, content.updateCalled, "focused region should receive key") - assert.False(t, header.updateCalled, "unfocused region should not receive key") - }) - - t.Run("Update broadcasts WindowSizeMsg to all", func(t *testing.T) { - f := NewFrame("HCF") - header := &testFrameModel{viewText: "h"} - content := &testFrameModel{viewText: "c"} - footer := &testFrameModel{viewText: "f"} - f.Header(header) - f.Content(content) - f.Footer(footer) - - sizeMsg := tea.WindowSizeMsg{Width: 120, Height: 40} - f.Update(sizeMsg) - - assert.True(t, header.updateCalled, "header should get resize") - assert.True(t, content.updateCalled, "content should get resize") - assert.True(t, footer.updateCalled, "footer should get resize") - assert.Equal(t, 120, f.width) - assert.Equal(t, 40, f.height) - }) - - t.Run("Update handles quit key", func(t *testing.T) { - f := NewFrame("HCF") - f.Content(StaticModel("c")) - - quitMsg := tea.KeyMsg{Type: tea.KeyCtrlC} - _, cmd := f.Update(quitMsg) - - // cmd should be tea.Quit - assert.NotNil(t, cmd) - }) - - t.Run("Update handles back key", func(t *testing.T) { - f := NewFrame("HCF") - f.Content(StaticModel("page-1")) - f.Navigate(StaticModel("page-2")) - - escMsg := tea.KeyMsg{Type: tea.KeyEsc} - f.Update(escMsg) - - assert.Contains(t, f.String(), "page-1") - }) - - t.Run("Update cycles focus with Tab", func(t *testing.T) { - f := NewFrame("HCF") - f.Header(StaticModel("h")) - f.Content(StaticModel("c")) - f.Footer(StaticModel("f")) - - assert.Equal(t, RegionContent, f.Focused()) - - tabMsg := tea.KeyMsg{Type: tea.KeyTab} - f.Update(tabMsg) - assert.Equal(t, RegionFooter, f.Focused()) - - f.Update(tabMsg) - assert.Equal(t, RegionHeader, f.Focused()) // wraps around - - shiftTabMsg := tea.KeyMsg{Type: tea.KeyShiftTab} - f.Update(shiftTabMsg) - assert.Equal(t, RegionFooter, f.Focused()) // back - }) - - t.Run("View produces non-empty output", func(t *testing.T) { - SetColorEnabled(false) - defer SetColorEnabled(true) - - f := NewFrame("HCF") - f.Header(StaticModel("HEAD")) - f.Content(StaticModel("BODY")) - f.Footer(StaticModel("FOOT")) - - view := f.View() - assert.Contains(t, view, "HEAD") - assert.Contains(t, view, "BODY") - assert.Contains(t, view, "FOOT") - }) - - t.Run("View lipgloss layout: header before content before footer", func(t *testing.T) { - SetColorEnabled(false) - defer SetColorEnabled(true) - - f := NewFrame("HCF") - f.Header(StaticModel("AAA")) - f.Content(StaticModel("BBB")) - f.Footer(StaticModel("CCC")) - f.width = 80 - f.height = 24 - - view := f.View() - posA := indexOf(view, "AAA") - posB := indexOf(view, "BBB") - posC := indexOf(view, "CCC") - assert.Greater(t, posA, -1, "header should be present") - assert.Greater(t, posB, -1, "content should be present") - assert.Greater(t, posC, -1, "footer should be present") - assert.Less(t, posA, posB, "header before content") - assert.Less(t, posB, posC, "content before footer") - }) -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -race -run TestFrameTeaModel ./pkg/cli/` -Expected: FAIL — `Init()`, `Update(tea.Msg)`, `View()` don't exist on Frame (wrong signatures from what tea.Model needs) - -**Step 3: Write the implementation** - -This is the biggest change. Modify `pkg/cli/frame.go`: - -**Add lipgloss import:** - -```go -import ( - "fmt" - "io" - "os" - "strings" - "sync" - "time" - - tea "github.com/charmbracelet/bubbletea" - "github.com/charmbracelet/lipgloss" - "golang.org/x/term" -) -``` - -**Add the three tea.Model methods:** - -```go -// Init implements tea.Model. Collects Init() from all FrameModel regions. -func (f *Frame) Init() tea.Cmd { - f.mu.Lock() - defer f.mu.Unlock() - - var cmds []tea.Cmd - for _, m := range f.models { - fm := adaptModel(m) - if cmd := fm.Init(); cmd != nil { - cmds = append(cmds, cmd) - } - } - return tea.Batch(cmds...) -} - -// Update implements tea.Model. Routes messages based on type and focus. -func (f *Frame) Update(msg tea.Msg) (tea.Model, tea.Cmd) { - f.mu.Lock() - defer f.mu.Unlock() - - switch msg := msg.(type) { - case tea.WindowSizeMsg: - f.width = msg.Width - f.height = msg.Height - return f, f.broadcastLocked(msg) - - case tea.KeyMsg: - switch msg.Type { - case f.keyMap.Quit: - return f, tea.Quit - - case f.keyMap.Back: - f.backLocked() - return f, nil - - case f.keyMap.FocusNext: - f.cycleFocusLocked(1) - return f, nil - - case f.keyMap.FocusPrev: - f.cycleFocusLocked(-1) - return f, nil - - case f.keyMap.FocusUp: - f.spatialFocusLocked(RegionHeader) - return f, nil - - case f.keyMap.FocusDown: - f.spatialFocusLocked(RegionFooter) - return f, nil - - case f.keyMap.FocusLeft: - f.spatialFocusLocked(RegionLeft) - return f, nil - - case f.keyMap.FocusRight: - f.spatialFocusLocked(RegionRight) - return f, nil - - default: - // Forward to focused region - return f, f.updateFocusedLocked(msg) - } - - default: - // Broadcast non-key messages to all regions - return f, f.broadcastLocked(msg) - } -} - -// View implements tea.Model. Composes region views using lipgloss. -func (f *Frame) View() string { - f.mu.Lock() - defer f.mu.Unlock() - return f.viewLocked() -} - -func (f *Frame) viewLocked() string { - w, h := f.width, f.height - if w == 0 || h == 0 { - w, h = f.termSize() - } - - // Calculate region dimensions - headerH, footerH := 0, 0 - if _, ok := f.layout.regions[RegionHeader]; ok { - if _, ok := f.models[RegionHeader]; ok { - headerH = 1 - } - } - if _, ok := f.layout.regions[RegionFooter]; ok { - if _, ok := f.models[RegionFooter]; ok { - footerH = 1 - } - } - middleH := h - headerH - footerH - if middleH < 1 { - middleH = 1 - } - - // Render each region - header := f.renderRegionLocked(RegionHeader, w, headerH) - footer := f.renderRegionLocked(RegionFooter, w, footerH) - - // Calculate sidebar widths - leftW, rightW := 0, 0 - if _, ok := f.layout.regions[RegionLeft]; ok { - if _, ok := f.models[RegionLeft]; ok { - leftW = w / 4 - } - } - if _, ok := f.layout.regions[RegionRight]; ok { - if _, ok := f.models[RegionRight]; ok { - rightW = w / 4 - } - } - contentW := w - leftW - rightW - if contentW < 1 { - contentW = 1 - } - - left := f.renderRegionLocked(RegionLeft, leftW, middleH) - right := f.renderRegionLocked(RegionRight, rightW, middleH) - content := f.renderRegionLocked(RegionContent, contentW, middleH) - - // Compose middle row - var middleParts []string - if leftW > 0 { - middleParts = append(middleParts, left) - } - middleParts = append(middleParts, content) - if rightW > 0 { - middleParts = append(middleParts, right) - } - - middle := content - if len(middleParts) > 1 { - middle = lipgloss.JoinHorizontal(lipgloss.Top, middleParts...) - } - - // Compose full layout - var verticalParts []string - if headerH > 0 { - verticalParts = append(verticalParts, header) - } - verticalParts = append(verticalParts, middle) - if footerH > 0 { - verticalParts = append(verticalParts, footer) - } - - return lipgloss.JoinVertical(lipgloss.Left, verticalParts...) -} - -func (f *Frame) renderRegionLocked(r Region, w, h int) string { - if w <= 0 || h <= 0 { - return "" - } - m, ok := f.models[r] - if !ok { - return "" - } - fm := adaptModel(m) - return fm.View(w, h) -} -``` - -**Add internal focus helpers (inside frame.go):** - -```go -// cycleFocusLocked moves focus forward (+1) or backward (-1) in the focus ring. -// Must be called with f.mu held. -func (f *Frame) cycleFocusLocked(dir int) { - ring := f.buildFocusRing() - if len(ring) == 0 { - return - } - idx := 0 - for i, r := range ring { - if r == f.focused { - idx = i - break - } - } - idx = (idx + dir + len(ring)) % len(ring) - f.focused = ring[idx] -} - -// spatialFocusLocked moves focus to a specific region if it exists in the layout. -// Must be called with f.mu held. -func (f *Frame) spatialFocusLocked(target Region) { - if _, exists := f.layout.regions[target]; exists { - f.focused = target - } -} - -// backLocked pops the content history. Must be called with f.mu held. -func (f *Frame) backLocked() { - if len(f.history) == 0 { - return - } - f.models[RegionContent] = f.history[len(f.history)-1] - f.history = f.history[:len(f.history)-1] -} - -// broadcastLocked sends a message to all FrameModel regions. -// Must be called with f.mu held. -func (f *Frame) broadcastLocked(msg tea.Msg) tea.Cmd { - var cmds []tea.Cmd - for r, m := range f.models { - fm := adaptModel(m) - updated, cmd := fm.Update(msg) - f.models[r] = updated - if cmd != nil { - cmds = append(cmds, cmd) - } - } - return tea.Batch(cmds...) -} - -// updateFocusedLocked sends a message to only the focused region. -// Must be called with f.mu held. -func (f *Frame) updateFocusedLocked(msg tea.Msg) tea.Cmd { - m, ok := f.models[f.focused] - if !ok { - return nil - } - fm := adaptModel(m) - updated, cmd := fm.Update(msg) - f.models[f.focused] = updated - return cmd -} -``` - -**Step 4: Run test to verify it passes** - -Run: `go test -race -run TestFrameTeaModel ./pkg/cli/` -Expected: PASS - -**Step 5: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS — existing tests use `String()` (non-TTY path) which is unchanged - -**Step 6: Commit** - -```bash -git add pkg/cli/frame.go pkg/cli/frame_test.go -git commit -m "feat(frame): implement tea.Model (Init, Update, View) with lipgloss layout" -``` - ---- - -### Task 6: Replace runLive() with tea.Program - -**Files:** -- Modify: `pkg/cli/frame.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the failing test** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameSend_Good(t *testing.T) { - t.Run("Send is safe before Run", func(t *testing.T) { - f := NewFrame("C") - f.out = &bytes.Buffer{} - f.Content(StaticModel("x")) - - // Should not panic when program is nil - assert.NotPanics(t, func() { - f.Send(tea.KeyMsg{Type: tea.KeyRunes, Runes: []rune{'x'}}) - }) - }) -} -``` - -**Step 2: Run test to verify it fails** - -Run: `go test -race -run TestFrameSend ./pkg/cli/` -Expected: FAIL — `Send` method doesn't exist - -**Step 3: Write the implementation** - -Modify `pkg/cli/frame.go`: - -**Replace `runLive()`:** - -```go -func (f *Frame) runLive() { - opts := []tea.ProgramOption{ - tea.WithAltScreen(), - } - if f.out != os.Stdout { - opts = append(opts, tea.WithOutput(f.out)) - } - - p := tea.NewProgram(f, opts...) - f.program = p - - if _, err := p.Run(); err != nil { - Error(err.Error()) - } -} -``` - -**Delete the old `renderFrame()` method** — it's no longer used (View() replaces it). - -**Update `Stop()`:** - -```go -func (f *Frame) Stop() { - if f.program != nil { - f.program.Quit() - return - } - select { - case <-f.done: - default: - close(f.done) - } -} -``` - -**Add `Send()`:** - -```go -// Send injects a message into the Frame's tea.Program. -// Safe to call before Run() (message is discarded). -func (f *Frame) Send(msg tea.Msg) { - if f.program != nil { - f.program.Send(msg) - } -} -``` - -**Update `RunFor()`** to work with tea.Program: - -```go -func (f *Frame) RunFor(d time.Duration) { - go func() { - timer := time.NewTimer(d) - defer timer.Stop() - select { - case <-timer.C: - f.Stop() - case <-f.done: - } - }() - f.Run() -} -``` - -**Step 4: Run test to verify it passes** - -Run: `go test -race -run TestFrameSend ./pkg/cli/` -Expected: PASS - -**Step 5: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS - -**Step 6: Commit** - -```bash -git add pkg/cli/frame.go pkg/cli/frame_test.go -git commit -m "feat(frame): replace raw ANSI runLive with tea.Program" -``` - ---- - -### Task 7: Clean up dead code - -**Files:** -- Modify: `pkg/cli/frame.go` - -**Step 1: Review and remove unused code** - -The old `renderFrame()` method should have been removed in Task 6. Verify it's gone. - -Also check if `f.done` channel is still needed. It's used by `RunFor()` as a fallback and by the non-TTY `Stop()` path. Keep it for now. - -Check if `golang.org/x/term` import is still needed. `isTTY()` and `termSize()` still use it for the non-TTY fallback path. Keep it. - -**Step 2: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS - -**Step 3: Run go vet** - -Run: `go vet ./pkg/cli/` -Expected: no warnings - -**Step 4: Commit (only if there were changes)** - -```bash -git add pkg/cli/frame.go -git commit -m "refactor(frame): remove unused renderFrame method" -``` - ---- - -### Task 8: Update String() to use viewLocked() - -The `String()` method currently has its own rendering logic separate from `View()`. Unify them so `String()` delegates to the same lipgloss-based layout. - -**Files:** -- Modify: `pkg/cli/frame.go` -- Test: `pkg/cli/frame_test.go` - -**Step 1: Verify existing String() tests still define the contract** - -The existing tests assert: -- `"static render HCF"` — contains header, content, footer -- `"region order preserved"` — header before content before footer -- `"empty regions skipped"` — only content → "only content\n" -- `"empty frame"` — no models → "" - -These must still pass after the change. - -**Step 2: Update String()** - -Replace the existing `String()` method: - -```go -func (f *Frame) String() string { - f.mu.Lock() - defer f.mu.Unlock() - - view := f.viewLocked() - if view == "" { - return "" - } - // Ensure trailing newline for non-TTY consistency - if !strings.HasSuffix(view, "\n") { - view += "\n" - } - return view -} -``` - -**Step 3: Run all tests** - -Run: `go test -race ./pkg/cli/` -Expected: PASS — the existing test assertions should still hold because `viewLocked()` produces the same output structure (header, content, footer in order) - -**Important:** If `"empty regions skipped"` fails (expected `"only content\n"` but lipgloss adds padding), you may need to adjust `viewLocked()` to not use `lipgloss.Place()` for content — just return the raw view string when there's only one region. The fix: - -In `viewLocked()`, if only one vertical part exists (no header, no footer, just content), return it directly without lipgloss wrapping. - -**Step 4: Commit** - -```bash -git add pkg/cli/frame.go -git commit -m "refactor(frame): unify String() with View() via viewLocked()" -``` - ---- - -### Task 9: Add spatial focus navigation tests - -**Files:** -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the tests** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameSpatialFocus_Good(t *testing.T) { - t.Run("arrow keys move to target region", func(t *testing.T) { - f := NewFrame("HLCRF") - f.Header(StaticModel("h")) - f.Left(StaticModel("l")) - f.Content(StaticModel("c")) - f.Right(StaticModel("r")) - f.Footer(StaticModel("f")) - - // Start at Content - assert.Equal(t, RegionContent, f.Focused()) - - // Up → Header - f.Update(tea.KeyMsg{Type: tea.KeyUp}) - assert.Equal(t, RegionHeader, f.Focused()) - - // Down → Footer - f.Update(tea.KeyMsg{Type: tea.KeyDown}) - assert.Equal(t, RegionFooter, f.Focused()) - - // Left → Left sidebar - f.Update(tea.KeyMsg{Type: tea.KeyLeft}) - assert.Equal(t, RegionLeft, f.Focused()) - - // Right → Right sidebar - f.Update(tea.KeyMsg{Type: tea.KeyRight}) - assert.Equal(t, RegionRight, f.Focused()) - }) - - t.Run("spatial focus ignores missing regions", func(t *testing.T) { - f := NewFrame("HCF") // no Left or Right - f.Header(StaticModel("h")) - f.Content(StaticModel("c")) - f.Footer(StaticModel("f")) - - assert.Equal(t, RegionContent, f.Focused()) - - // Left arrow → no Left region, focus stays - f.Update(tea.KeyMsg{Type: tea.KeyLeft}) - assert.Equal(t, RegionContent, f.Focused()) - }) -} -``` - -**Step 2: Run tests** - -Run: `go test -race -run TestFrameSpatialFocus ./pkg/cli/` -Expected: PASS (these should work with the Update logic from Task 5) - -**Step 3: Commit** - -```bash -git add pkg/cli/frame_test.go -git commit -m "test(frame): add spatial focus navigation tests" -``` - ---- - -### Task 10: Add Navigate/Back FrameModel focus transfer tests - -**Files:** -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the tests** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameNavigateFrameModel_Good(t *testing.T) { - t.Run("Navigate with FrameModel preserves focus on Content", func(t *testing.T) { - f := NewFrame("HCF") - f.Header(StaticModel("h")) - f.Content(&testFrameModel{viewText: "page-1"}) - f.Footer(StaticModel("f")) - - // Focus something else - f.Focus(RegionHeader) - assert.Equal(t, RegionHeader, f.Focused()) - - // Navigate replaces Content, focus should remain where it was - f.Navigate(&testFrameModel{viewText: "page-2"}) - assert.Equal(t, RegionHeader, f.Focused()) - assert.Contains(t, f.String(), "page-2") - }) - - t.Run("Back restores FrameModel", func(t *testing.T) { - f := NewFrame("HCF") - f.out = &bytes.Buffer{} - fm1 := &testFrameModel{viewText: "page-1"} - fm2 := &testFrameModel{viewText: "page-2"} - f.Header(StaticModel("h")) - f.Content(fm1) - f.Footer(StaticModel("f")) - - f.Navigate(fm2) - assert.Contains(t, f.String(), "page-2") - - ok := f.Back() - assert.True(t, ok) - assert.Contains(t, f.String(), "page-1") - }) -} -``` - -**Step 2: Run tests** - -Run: `go test -race -run TestFrameNavigateFrameModel ./pkg/cli/` -Expected: PASS - -**Step 3: Commit** - -```bash -git add pkg/cli/frame_test.go -git commit -m "test(frame): add Navigate/Back tests with FrameModel" -``` - ---- - -### Task 11: Add message routing edge case tests - -**Files:** -- Test: `pkg/cli/frame_test.go` - -**Step 1: Write the tests** - -Add to `pkg/cli/frame_test.go`: - -```go -func TestFrameMessageRouting_Good(t *testing.T) { - t.Run("custom message broadcasts to all FrameModels", func(t *testing.T) { - f := NewFrame("HCF") - header := &testFrameModel{viewText: "h"} - content := &testFrameModel{viewText: "c"} - footer := &testFrameModel{viewText: "f"} - f.Header(header) - f.Content(content) - f.Footer(footer) - - // Send a custom message (not KeyMsg, not WindowSizeMsg) - type customMsg struct{ data string } - f.Update(customMsg{data: "hello"}) - - assert.True(t, header.updateCalled, "header should receive custom msg") - assert.True(t, content.updateCalled, "content should receive custom msg") - assert.True(t, footer.updateCalled, "footer should receive custom msg") - }) - - t.Run("plain Model regions ignore messages gracefully", func(t *testing.T) { - f := NewFrame("HCF") - f.Header(StaticModel("h")) - f.Content(StaticModel("c")) - f.Footer(StaticModel("f")) - - // Should not panic — modelAdapter ignores all messages - assert.NotPanics(t, func() { - f.Update(tea.WindowSizeMsg{Width: 100, Height: 50}) - f.Update(tea.KeyMsg{Type: tea.KeyRunes, Runes: []rune{'x'}}) - }) - }) -} -``` - -**Step 2: Run tests** - -Run: `go test -race -run TestFrameMessageRouting ./pkg/cli/` -Expected: PASS - -**Step 3: Commit** - -```bash -git add pkg/cli/frame_test.go -git commit -m "test(frame): add message routing edge case tests" -``` - ---- - -### Task 12: Final verification and cleanup - -**Files:** -- All modified files - -**Step 1: Run full test suite with race detector** - -Run: `go test -race ./pkg/cli/` -Expected: PASS with no race conditions - -**Step 2: Run go vet** - -Run: `go vet ./pkg/cli/` -Expected: no warnings - -**Step 3: Check for unused imports** - -Run: `go build ./pkg/cli/` -Expected: clean build, no errors - -**Step 4: Count test coverage** - -Run: `go test -race -cover ./pkg/cli/` -Expected: coverage reported, aim for >85% on frame.go and frame_model.go - -**Step 5: Verify all existing tests still pass with exact same assertions** - -Run: `go test -race -v -run "TestFrame_Good|TestFrame_Bad|TestStatusLine|TestKeyHints|TestBreadcrumb|TestStaticModel" ./pkg/cli/` -Expected: all 14 original tests PASS - -**Step 6: Final commit if any cleanup was needed** - -```bash -git add -A -git commit -m "chore(frame): final cleanup after bubbletea upgrade" -``` - ---- - -## Summary of Files - -| File | Action | Lines (approx) | -|------|--------|----------------| -| `go.mod` | modify | +2 deps | -| `go.sum` | modify | auto-generated | -| `pkg/cli/frame_model.go` | **create** | ~60 lines | -| `pkg/cli/frame.go` | modify | Replace runLive/renderFrame, add Init/Update/View, add focus helpers (~150 lines changed) | -| `pkg/cli/frame_test.go` | modify | Add ~200 lines of new tests | - -## Commit Sequence - -1. `deps: add bubbletea and lipgloss for Frame upgrade` -2. `feat(frame): add FrameModel interface and modelAdapter` -3. `feat(frame): add KeyMap with default bindings` -4. `feat(frame): add focus management fields, Focused(), Focus(), WithKeyMap()` -5. `feat(frame): implement tea.Model (Init, Update, View) with lipgloss layout` -6. `feat(frame): replace raw ANSI runLive with tea.Program` -7. `refactor(frame): remove unused renderFrame method` -8. `refactor(frame): unify String() with View() via viewLocked()` -9. `test(frame): add spatial focus navigation tests` -10. `test(frame): add Navigate/Back tests with FrameModel` -11. `test(frame): add message routing edge case tests` -12. `chore(frame): final cleanup after bubbletea upgrade` diff --git a/docs/plans/completed/bugseti-hub-service.md b/docs/plans/completed/bugseti-hub-service.md deleted file mode 100644 index 1a3ac54..0000000 --- a/docs/plans/completed/bugseti-hub-service.md +++ /dev/null @@ -1,57 +0,0 @@ -# BugSETI HubService — Completion Summary - -**Completed:** 13 February 2026 -**Module:** `forge.lthn.ai/core/cli` (extracted to `core/bugseti` repo on 16 Feb 2026) -**Status:** Complete — all Go-side tasks implemented and wired into app lifecycle - -## What Was Built - -Thin HTTP client service coordinating with the agentic portal's -`/api/bugseti/*` endpoints for issue claiming, stats sync, leaderboard, -and offline-first pending operations queue. - -### Implementation (Tasks 1-8 from plan) - -All 8 Go-side tasks were implemented across commits `a38ce05` through `177ce27`: - -1. **Config fields** — HubURL, HubToken, ClientID, ClientName added to - ConfigService with getters/setters (`a38ce05`) -2. **HubService types + constructor** — HubService, PendingOp, HubClaim, - LeaderboardEntry, GlobalStats, ConflictError, NotFoundError (`a89acfa`) -3. **HTTP request helpers** — `doRequest()`, `doJSON()` with bearer auth, - error classification (401/404/409), and connection tracking (`ab7ef52`) -4. **AutoRegister** — exchange forge token for ak_ hub token via - `/auth/forge` endpoint (`21d5f5f`) -5. **Write operations** — Register, Heartbeat, ClaimIssue, UpdateStatus, - ReleaseClaim, SyncStats (`a6456e2`) -6. **Read operations** — IsIssueClaimed, ListClaims, GetLeaderboard, - GetGlobalStats (`7a92fe0`) -7. **Pending ops queue** — offline-first queue with disk persistence to - `hub_pending.json`, drain-on-reconnect (`a567568`) -8. **main.go integration** — HubService wired as Wails service with - auto-registration at startup (`177ce27`) - -### Tests - -All operations tested with `httptest.NewServer` mocks covering success, -network error, 409 conflict, 401 re-auth, and pending ops persist/reload -scenarios. Hub test file: `internal/bugseti/hub_test.go`. - -### Key files (before extraction) - -- `internal/bugseti/hub.go` — HubService implementation (25 exported methods) -- `internal/bugseti/hub_test.go` — comprehensive httptest-based test suite -- `internal/bugseti/config.go` — hub config fields and accessors -- `cmd/bugseti/main.go` — lifecycle wiring - -### Task 9 (Laravel endpoint) - -The portal-side `/api/bugseti/auth/forge` endpoint (Task 9) lives in the -`agentic` repo, not in `core/cli`. It was designed in this plan but -implemented separately. - -### Extraction - -BugSETI was extracted to its own repo on 16 Feb 2026 (`8167f66`): -`internal/bugseti/` moved to `core/bugseti`, `cmd/bugseti/` moved to -`core/bugseti/cmd/`. diff --git a/docs/plans/completed/cli-meta-package.md b/docs/plans/completed/cli-meta-package.md deleted file mode 100644 index d88672b..0000000 --- a/docs/plans/completed/cli-meta-package.md +++ /dev/null @@ -1,30 +0,0 @@ -# CLI Meta-Package Restructure — Completed - -**Completed:** 22 Feb 2026 - -## What Was Done - -`pkg/cli` was extracted from `core/go` into its own Go module at `forge.lthn.ai/core/cli`. This made the CLI SDK a first-class, independently versioned package rather than a subdirectory of the Go foundation repo. - -Following the extraction, an ecosystem-wide import path migration updated all consumers from the old path to the new one: - -- Old: `forge.lthn.ai/core/go/pkg/cli` -- New: `forge.lthn.ai/core/cli/pkg/cli` - -## Scope - -- **147+ files** updated across **10 repos** -- All repos build clean after migration - -## Repos Migrated - -`core/cli`, `core/go`, `go-devops`, `go-ai`, `go-agentic`, `go-crypt`, `go-rag`, `go-scm`, `go-api`, `go-update` - -## Key Outcomes - -- `forge.lthn.ai/core/cli/pkg/cli` is the single import for all CLI concerns across the ecosystem -- Domain repos are insulated from cobra, lipgloss, and bubbletea — only `pkg/cli` imports them -- Command registration uses the Core framework lifecycle via `cli.WithCommands()` — no `init()`, no global state -- `core/cli` is a thin assembly repo (~2K LOC) with 7 meta packages; all business logic lives in domain repos -- Variant binary pattern established: multiple `main.go` files can wire different `WithCommands` sets for targeted binaries (core-ci, core-mlx, core-ops, etc.) -- Command migration from the old `core/cli` monolith to domain repos was completed in full (13 command groups moved) diff --git a/docs/plans/completed/cli-sdk-expansion.md b/docs/plans/completed/cli-sdk-expansion.md deleted file mode 100644 index a0a84a3..0000000 --- a/docs/plans/completed/cli-sdk-expansion.md +++ /dev/null @@ -1,39 +0,0 @@ -# CLI SDK Expansion — Completion Summary - -**Completed:** 21 February 2026 -**Module:** `forge.lthn.ai/core/go/pkg/cli` (later migrated to `forge.lthn.ai/core/cli`) -**Status:** Complete — all TUI primitives shipped, then extracted to core/cli - -## What Was Built - -Extended `pkg/cli` with charmbracelet TUI primitives so domain repos only -import `core/cli` for all CLI concerns. Charmbracelet dependencies (bubbletea, -bubbles, lipgloss) are encapsulated behind our own types. - -### Components added - -| Component | File | Purpose | -|-----------|------|---------| -| RunTUI | `runtui.go` | Escape hatch with `Model`/`Msg`/`Cmd`/`KeyMsg` types | -| Spinner | `spinner.go` | Async handle with `Update()`, `Done()`, `Fail()` | -| ProgressBar | `progressbar.go` | `Increment()`, `Set()`, `SetMessage()`, `Done()` | -| InteractiveList | `list.go` | Keyboard navigation with terminal fallback | -| TextInput | `textinput.go` | Placeholder, masking, validation | -| Viewport | `viewport.go` | Scrollable content for logs, diffs, docs | -| Form (stub) | `form.go` | Interface defined, bufio fallback | -| FilePicker (stub) | `filepicker.go` | Interface defined, bufio fallback | -| Tabs (stub) | `tabs.go` | Interface defined, simple fallback | - -### Subsequent migration - -On 22 February 2026, `pkg/cli` was extracted from `core/go` into its own -module at `forge.lthn.ai/core/cli` and all imports were updated. The TUI -primitives now live in the standalone CLI module. - -### Frame upgrade (follow-on) - -The Frame layout system was upgraded to implement `tea.Model` directly on -22 February 2026 (in `core/cli`), adding bubbletea lifecycle, `KeyMap` for -configurable bindings, `Navigate()`/`Back()` for panel switching, and -lipgloss-based HLCRF rendering. This was a separate plan -(`frame-bubbletea`) that built on the SDK expansion. diff --git a/docs/plans/completed/core-ide-job-runner.md b/docs/plans/completed/core-ide-job-runner.md deleted file mode 100644 index 10093f5..0000000 --- a/docs/plans/completed/core-ide-job-runner.md +++ /dev/null @@ -1,50 +0,0 @@ -# Core-IDE Job Runner — Completion Summary - -**Completed:** 9 February 2026 -**Module:** `forge.lthn.ai/core/cli` (extracted to `core/ide` repo during monorepo split) -**Status:** Complete — all components built, tested, and operational before extraction - -## What Was Built - -Autonomous job runner for core-ide that polls Forgejo for actionable pipeline -work, executes it via typed handler functions, captures JSONL training data, -and supports both headless (server) and desktop (Wails GUI) modes. - -### Key components - -- **`pkg/jobrunner/types.go`** — JobSource, JobHandler, PipelineSignal, - ActionResult interfaces and structs -- **`pkg/jobrunner/poller.go`** — multi-source poller with configurable - interval, ETag-based conditional requests, and idle backoff -- **`pkg/jobrunner/journal.go`** — append-only JSONL writer for training data - capture (structural signals only, no content) -- **`pkg/jobrunner/forgejo/source.go`** — ForgejoSource adapter (evolved from - original GitHubSource design to use pkg/forge SDK) -- **`pkg/jobrunner/forgejo/signals.go`** — PR/issue state extraction and - signal building from Forgejo API responses - -### Handlers - -All six handlers from the design were implemented with tests: - -- `publish_draft` — mark draft PRs as ready when checks pass -- `send_fix_command` — comment fix instructions for conflicts/reviews -- `resolve_threads` — resolve pre-commit review threads after fix -- `enable_auto_merge` — enable auto-merge when all checks pass -- `tick_parent` — update epic issue checklist when child PR merges -- `dispatch` — SCP ticket delivery to agent machines via SSH (added beyond - original design) - -### Headless / Desktop mode - -- `hasDisplay()` detection for Linux/macOS/Windows -- `--headless` / `--desktop` CLI flag overrides -- Headless: poller + MCP bridge, signal handling, systemd-ready -- Desktop: Wails GUI with system tray, optional poller toggle - -### Extraction - -Code was fully operational and then extracted during the Feb 2026 monorepo -split (`abe74a1`). `pkg/jobrunner/` moved to `core/go`, `cmd/core-ide/` and -`internal/core-ide/` moved to `core/ide`. The agentci dispatch system -(`d9f3b72` through `886c67e`) built on top of the jobrunner before extraction. diff --git a/docs/plans/completed/frame-bubbletea.md b/docs/plans/completed/frame-bubbletea.md deleted file mode 100644 index 9a0d09f..0000000 --- a/docs/plans/completed/frame-bubbletea.md +++ /dev/null @@ -1,39 +0,0 @@ -# Frame Bubbletea Upgrade — Completion Summary - -**Completed:** 22 February 2026 -**Module:** `forge.lthn.ai/core/cli` -**Status:** Complete — Frame implements tea.Model with full bubbletea lifecycle - -## What Was Built - -Upgraded the Frame layout system from a static HLCRF renderer to a full -bubbletea `tea.Model` with lifecycle management, keyboard handling, and -panel navigation. - -### Key changes - -- **Frame implements `tea.Model`** — `Init()`, `Update()`, `View()` lifecycle -- **`KeyMap`** — configurable keybindings with default set (quit, navigate, - help, focus cycling) -- **`Navigate(name)` / `Back()`** — panel switching with history stack -- **Focus management** — Tab/Shift-Tab cycles focus between visible models -- **lipgloss layout** — HLCRF regions (Header, Left, Content, Right, Footer) - rendered with lipgloss instead of raw ANSI -- **`FrameModel` interface** — models register with `Frame.Header()`, - `.Content()`, `.Footer()` etc., receiving focus/blur/resize messages - -### Tests - -Navigate/Back stack tests, focus cycling, key dispatch, resize propagation. -All passing with `-race`. - -### Dependencies - -- `github.com/charmbracelet/bubbletea` -- `github.com/charmbracelet/lipgloss` - -### Consumer - -`go-blockchain/cmd/chain/` is the first consumer — TUI dashboard uses -Frame with StatusModel (header), ExplorerModel (content), KeyHintsModel -(footer). diff --git a/docs/plans/completed/go-api.md b/docs/plans/completed/go-api.md deleted file mode 100644 index 86278a2..0000000 --- a/docs/plans/completed/go-api.md +++ /dev/null @@ -1,57 +0,0 @@ -# go-api — Completion Summary - -**Completed:** 21 February 2026 -**Module:** `forge.lthn.ai/core/go-api` -**Status:** Phases 1–3 complete, 176 tests passing - -## What Was Built - -### Phase 1 — Core Framework (20 Feb 2026) - -Gin-based HTTP engine with extensible middleware via `With*()` options. Key components: - -- `RouteGroup` / `StreamGroup` interfaces — subsystems register their own endpoints -- `Response[T]` envelope — `OK()`, `Fail()`, `Paginated()` generics -- `Engine` — `New()`, `Register()`, `Handler()`, `Serve()` with graceful shutdown -- Bearer auth, request ID, and CORS middleware -- WebSocket endpoint wrapping a `go-ws` Hub -- Swagger UI at `/swagger/` with runtime spec serving -- `/health` endpoint always available without auth -- First integration proof in `go-ml/api/` (3 endpoints, 12 tests) - -### Phase 2 — Gin Plugin Stack (20–21 Feb 2026) - -17 middleware plugins added across four waves, all as drop-in `With*()` options: - -| Wave | Plugins | -|------|---------| -| 1 — Gateway hardening | Authentik (OIDC + forward auth), secure headers, structured slog, timeouts, gzip, static files | -| 2 — Performance + auth | Brotli compression, in-memory response cache, server-side sessions, Casbin RBAC | -| 3 — Network + streaming | HTTP signature verification, SSE broker, reverse proxy detection, i18n locale, GraphQL | -| 4 — Observability | pprof, expvar, OpenTelemetry distributed tracing | - -### Phase 3 — OpenAPI + SDK Codegen (21 Feb 2026) - -Runtime spec generation (not swaggo annotations — incompatible with dynamic RouteGroups and `Response[T]` generics): - -- `DescribableGroup` interface — opt-in OpenAPI metadata for route groups -- `ToolBridge` — converts MCP tool descriptors into `POST /{tool_name}` REST endpoints -- `SpecBuilder` — assembles full OpenAPI 3.1 JSON from registered groups at runtime -- Spec export to JSON and YAML (`core api spec`) -- SDK codegen wrapper for openapi-generator-cli, 11 languages (`core api sdk --lang go`) -- `go-ai` `mcp/registry.go` — generic `addToolRecorded[In,Out]` captures types in closures -- `go-ai` `mcp/bridge.go` — `BridgeToAPI()` populates ToolBridge from MCP tool registry -- CLI commands: `core api spec`, `core api sdk` (in `core/cli` dev branch) - -## Key Outcomes - -- **176 tests** across go-api (143), go-ai bridge (10), and CLI commands (4), all passing -- Zero internal ecosystem dependencies — subsystems import go-api, not the reverse -- Authentik (OIDC) and bearer token auth coexist; Casbin adds RBAC on top -- Four-protocol access pattern established: REST, GraphQL, WebSocket, MCP — same handlers - -## Known Limitations - -- Subsystem MCP tools registered via `mcp.AddTool` directly are excluded from the REST bridge (only the 10 built-in tools appear). Fix: pass `*Service` to `RegisterTools` instead of `*mcp.Server`. -- `structSchema` reflection handles flat structs only; nested structs are not recursed. -- `core api spec` currently emits a spec with only `/health`; full MCP wiring into the CLI command is pending. diff --git a/docs/plans/completed/mcp-integration.md b/docs/plans/completed/mcp-integration.md deleted file mode 100644 index 7edf86e..0000000 --- a/docs/plans/completed/mcp-integration.md +++ /dev/null @@ -1,37 +0,0 @@ -# MCP Integration — Completion Summary - -**Completed:** 2026-02-05 -**Plan:** `docs/plans/2026-02-05-mcp-integration.md` - -## What Was Built - -### RAG Tools (`pkg/mcp/tools_rag.go`) -Three MCP tools added to the existing `pkg/mcp` server: -- `rag_query` — semantic search against Qdrant vector DB -- `rag_ingest` — ingest a file or directory into a named collection -- `rag_collections` — list available Qdrant collections (with optional stats) - -### Metrics Tools (`pkg/mcp/tools_metrics.go`) -Two MCP tools for agent activity tracking: -- `metrics_record` — write a typed event (agent_id, repo, arbitrary data) to JSONL storage -- `metrics_query` — query events with aggregation by type, repo, and agent; supports human-friendly duration strings (7d, 24h) - -Also added `parseDuration()` helper for "Nd"/"Nh"/"Nm" duration strings. - -### `core mcp serve` Command (`internal/cmd/mcpcmd/cmd_mcp.go`) -New CLI sub-command registered via `cli.WithCommands()` (not `init()`). -- Runs `pkg/mcp` server over stdio by default -- TCP mode via `MCP_ADDR=:9000` environment variable -- `--workspace` flag to restrict file operations to a directory - -Registered in the full CLI variant. i18n strings added for all user-facing text. - -### Plugin Configuration -`.mcp.json` created for the `agentic-flows` Claude Code plugin, pointing to `core mcp serve`. Exposes all 15 tools to Claude Code agents via the `core-cli` MCP server name. - -## Key Outcomes - -- `core mcp serve` is the single entry point for all MCP tooling (file ops, RAG, metrics, language detection, process management, WebSocket, webview/CDP) -- MCP command moved to `go-ai/cmd/mcpcmd/` in final form; the plan's `internal/cmd/mcpcmd/` path reflects the pre-extraction location -- Registration pattern updated from `init()` + `RegisterCommands()` to `cli.WithCommands()` lifecycle hooks -- Services required at runtime: Qdrant (localhost:6333), Ollama with nomic-embed-text (localhost:11434) diff --git a/docs/plans/completed/qk-bone-orientation.md b/docs/plans/completed/qk-bone-orientation.md deleted file mode 100644 index 0cfcaa9..0000000 --- a/docs/plans/completed/qk-bone-orientation.md +++ /dev/null @@ -1,62 +0,0 @@ -# Q/K Bone Orientation — Completion Summary - -**Completed:** 23 February 2026 -**Repos:** go-inference, go-mlx, go-ml, LEM -**Status:** All 7 tasks complete, 14 files changed (+917 lines), all tests passing - -## What Was Built - -### go-inference — AttentionSnapshot types (Task 1) - -`AttentionSnapshot` struct and `AttentionInspector` optional interface. Backends expose attention data via type assertion — no breaking changes to `TextModel`. - -### go-mlx — KV cache extraction (Task 2) - -`InspectAttention` on `metalAdapter` runs a single prefill pass and extracts post-RoPE K vectors from each layer's KV cache. Tested against real Gemma3-1B (26 layers, 1 KV head via GQA, 256 head dim). - -### go-ml — Adapter pass-through (Task 3) - -`InspectAttention` on `InferenceAdapter` type-asserts the underlying `TextModel` to `AttentionInspector`. Returns clear error for unsupported backends. - -### LEM — Analysis engine (Task 4) - -Pure Go CPU math in `pkg/lem/attention.go`. Computes 5 BO metrics from raw K tensors: - -- **Mean Coherence** — pairwise cosine similarity of K vectors within each layer -- **Cross-Layer Alignment** — cosine similarity of mean K vectors between adjacent layers -- **Head Entropy** — normalised Shannon entropy of K vector magnitudes across positions -- **Phase-Lock Score** — fraction of head pairs above coherence threshold (0.7) -- **Joint Collapse Count** — layers where cross-alignment drops below threshold (0.5) - -Composite score: 30% coherence + 25% cross-alignment + 20% phase-lock + 15% entropy + 10% joint stability → 0-100 scale. - -### LEM — CLI command (Task 5) - -`lem score attention -model -prompt [-json]` loads a model, runs InspectAttention, and prints BO metrics. - -### LEM — Distill integration (Task 6) - -Opt-in attention scoring in the distill pipeline. Gated behind `scorer.attention: true` and `scorer.attention_min_score` in ai.yaml. Costs one extra prefill per probe. - -### LEM — Feature vectors (Task 7) - -19D full feature vector: 6D grammar + 8D heuristic + 5D attention (`mean_coherence`, `cross_alignment`, `head_entropy`, `phase_lock`, `joint_stability`). Ready for Poindexter KDTree spatial indexing. - -## Key Decisions - -- **Optional interface** — `AttentionInspector` via type assertion, not added to `TextModel` -- **Named `BOResult`** — avoids collision with `metal.AttentionResult` in go-mlx -- **Opt-in for distill** — extra prefill per probe is expensive, off by default -- **Pure Go analysis** — zero CGO deps in the analysis engine; GPU data extracted once via `.Floats()` - -## Commits - -| Repo | SHA | Message | -|------|-----|---------| -| go-inference | `0f7263f` | feat: add AttentionInspector optional interface | -| go-mlx | `c2177f7` | feat: implement AttentionInspector via KV cache extraction | -| go-ml | `45e9fed` | feat: add InspectAttention pass-through | -| LEM | `28309b2` | feat: add Q/K Bone Orientation analysis engine | -| LEM | `e333192` | feat: add 'lem score attention' CLI | -| LEM | `fbc636e` | feat: integrate attention scoring into distill pipeline | -| LEM | `b621baa` | feat: add 19D full feature vector |