feat(runtime): add RuntimeManager for provider loading

RuntimeManager discovers providers in ~/.core/providers/, starts their
binaries via os/exec, waits for health checks, and registers ProxyProviders
in the API engine. Wired into all three IDE modes (MCP stdio, headless,
GUI) with proper startup/shutdown lifecycle. Includes implementation plan.

Co-Authored-By: Virgil <virgil@lethean.io>

docs/superpowers/plans/2026-03-14-runtime-provider-loading.md

@@ -0,0 +1,105 @@
+# Runtime Provider Loading — Implementation Plan
+
+**Date:** 2026-03-14
+**Spec:** ../specs/2026-03-14-runtime-provider-loading-design.md
+**Status:** Complete
+
+## Task 1: ProxyProvider in core/api (`pkg/provider/proxy.go`)
+
+**File:** `/Users/snider/Code/core/api/pkg/provider/proxy.go`
+
+Replace the Phase 3 stub with a working ProxyProvider that:
+
+- Takes a `ProxyConfig` struct: Name, BasePath, Upstream URL, ElementSpec, SpecFile path
+- Implements `Provider` (Name, BasePath, RegisterRoutes)
+- Implements `Renderable` (Element)
+- `RegisterRoutes` creates a catch-all `/*path` handler using `net/http/httputil.ReverseProxy`
+- Strips the base path before proxying so the upstream sees clean paths
+- Upstream is always `127.0.0.1` (local process)
+
+**Test file:** `pkg/provider/proxy_test.go`
+- Proxy routes requests to a test upstream server
+- Health check passthrough
+- Element() returns configured ElementSpec
+- Name/BasePath return configured values
+
+## Task 2: Manifest Extensions in go-scm (`manifest/manifest.go`)
+
+**File:** `/Users/snider/Code/core/go-scm/manifest/manifest.go`
+
+Extend the Manifest struct with provider-specific fields:
+
+- `Namespace` — API route prefix (e.g. `/api/v1/cool-widget`)
+- `Port` — listen port (0 = auto-assign)
+- `Binary` — path to binary relative to provider dir
+- `Args` — additional CLI args
+- `Element` — UI element spec (tag + source)
+- `Spec` — path to OpenAPI spec file
+
+These fields are optional — existing manifests without them remain valid.
+
+**Test:** Add parse test with provider fields.
+
+## Task 3: Provider Discovery in go-scm (`marketplace/discovery.go`)
+
+**File:** `/Users/snider/Code/core/go-scm/marketplace/discovery.go`
+
+- `DiscoverProviders(dir string)` — scans `dir/*/manifest.yaml` (using `os` directly, not Medium, since this is filesystem discovery)
+- Returns `[]DiscoveredProvider` with Dir + Manifest
+- Skips directories without manifests (logs warning, continues)
+
+**File:** `/Users/snider/Code/core/go-scm/marketplace/registry_file.go`
+
+- `ProviderRegistry` — read/write `registry.yaml` tracking installed providers
+- `LoadRegistry(path)`, `SaveRegistry(path)`, `Add()`, `Remove()`, `Get()`, `List()`
+
+**Test file:** `marketplace/discovery_test.go`
+- Discover finds providers in temp dirs with manifests
+- Discover skips dirs without manifests
+- Registry CRUD operations
+
+## Task 4: RuntimeManager in core/ide (`runtime.go`)
+
+**File:** `/Users/snider/Code/core/ide/runtime.go`
+
+- `RuntimeManager` struct holding discovered providers, engine reference, process registry
+- `NewRuntimeManager(engine, processRegistry)` constructor
+- `StartAll(ctx)` — discover providers in `~/.core/providers/`, start each binary via `os/exec.Cmd`, wait for health, register ProxyProvider
+- `StopAll()` — SIGTERM each provider process, clean up
+- `List()` — return running providers
+- Free port allocation via `net.Listen(":0")`
+- Health check polling with timeout
+
+## Task 5: Wire into core/ide main.go
+
+**File:** `/Users/snider/Code/core/ide/main.go`
+
+- After creating the api.Engine (`engine, _ := api.New(...)`), create a RuntimeManager
+- In each mode (mcpOnly, headless, GUI), call `rm.StartAll(ctx)` before serving
+- In shutdown paths, call `rm.StopAll()`
+- Import `forge.lthn.ai/core/go-scm/marketplace` (add dependency)
+
+## Task 6: Build Verification
+
+- `cd core/api && go build ./...`
+- `cd go-scm && go build ./...`
+- `cd ide && go build ./...` (may need go.mod updates)
+
+## Task 7: Tests
+
+- `cd core/api && go test ./pkg/provider/...`
+- `cd go-scm && go test ./marketplace/... ./manifest/...`
+
+## Repos Affected
+
+| Repo | Changes |
+|------|---------|
+| core/api | `pkg/provider/proxy.go` (implement), `pkg/provider/proxy_test.go` (new) |
+| go-scm | `manifest/manifest.go` (extend), `manifest/manifest_test.go` (extend), `marketplace/discovery.go` (new), `marketplace/registry_file.go` (new), `marketplace/discovery_test.go` (new) |
+| core/ide | `runtime.go` (new), `main.go` (wire), `go.mod` (add go-scm dep) |
+
+## Commit Strategy
+
+1. core/api — `feat(provider): implement ProxyProvider reverse proxy`
+2. go-scm — `feat(marketplace): add provider discovery and registry`
+3. core/ide — `feat(runtime): add RuntimeManager for provider loading`

go.mod

@@ -3,9 +3,9 @@ module forge.lthn.ai/core/ide
 go 1.26.0
 
 require (
-	forge.lthn.ai/core/go v0.2.2
 	forge.lthn.ai/core/api v0.1.0
 	forge.lthn.ai/core/config v0.1.0
+	forge.lthn.ai/core/go v0.2.2
 	forge.lthn.ai/core/go-process v0.1.0
 	forge.lthn.ai/core/go-ws v0.1.3
 	forge.lthn.ai/core/gui v0.0.0
@@ -22,6 +22,7 @@ require (
 	forge.lthn.ai/core/go-ml v0.1.0 // indirect
 	forge.lthn.ai/core/go-mlx v0.1.0 // indirect
 	forge.lthn.ai/core/go-rag v0.1.0 // indirect
+	forge.lthn.ai/core/go-scm v0.1.0
 	forge.lthn.ai/core/go-webview v0.1.2 // indirect
 	github.com/99designs/gqlgen v0.17.87 // indirect
 	github.com/KyleBanks/depth v1.2.1 // indirect

go.sum

@@ -1,10 +1,11 @@
 dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
 dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
+forge.lthn.ai/core/api v0.1.0 h1:ZKnQx+L9vxLQSEjwpsD1eNcIQrE4YKV1c2AlMtseM6o=
+forge.lthn.ai/core/config v0.1.0 h1:qj14x/dnOWcsXMBQWAT3FtA+/sy6Qd+1NFTg5Xoil1I=
 forge.lthn.ai/core/go v0.2.2 h1:JCWaFfiG+agb0f7b5DO1g+h40x6nb4UydxJ7D+oZk5k=
 forge.lthn.ai/core/go v0.2.2/go.mod h1:gE6c8h+PJ2287qNhVUJ5SOe1kopEwHEquvinstpuyJc=
 forge.lthn.ai/core/go-ai v0.1.5 h1:iOKW5Wv4Pquc5beDw0QqaKspq3pUvqXxT8IEdCT13Go=
 forge.lthn.ai/core/go-ai v0.1.5/go.mod h1:h1gcfi7l0m+Z9lSOwzcqzSeqRIR/6Qc2vqezBo74Rl0=
-forge.lthn.ai/core/go-api v0.1.2 h1:zGmU2CqCQ0n0cntNvprdc7HoucD4E631wBdZw+taK1w=
 forge.lthn.ai/core/go-inference v0.1.0 h1:pO7etYgqV8LMKFdpW8/2RWncuECZJCIcf8nnezeZ5R4=
 forge.lthn.ai/core/go-inference v0.1.0/go.mod h1:jfWz+IJX55wAH98+ic6FEqqGB6/P31CHlg7VY7pxREw=
 forge.lthn.ai/core/go-io v0.0.5 h1:oSyngKTkB1gR5fEWYKXftTg9FxwnpddSiCq2dlwfImE=

main.go

@@ -78,6 +78,9 @@ func main() {
 	)
 	reg.MountAll(engine)
 
+	// ── Runtime Provider Manager ──────────────────────────────
+	rm := NewRuntimeManager(engine)
+
 	// ── Core framework ─────────────────────────────────────────
 	c, err := core.New(
 		core.WithName("ws", func(c *core.Core) (any, error) {
@@ -116,6 +119,11 @@ func main() {
 		bridge.Start(ctx)
 		go hub.Run(ctx)
 
+		// Start runtime providers
+		if err := rm.StartAll(ctx); err != nil {
+			log.Printf("runtime provider error: %v", err)
+		}
+
 		// Start API server in background for provider endpoints
 		go func() {
 			if err := engine.Serve(ctx); err != nil {
@@ -127,6 +135,7 @@ func main() {
 			log.Printf("MCP stdio error: %v", err)
 		}
 
+		rm.StopAll()
 		_ = mcpSvc.Shutdown(ctx)
 		_ = c.ServiceShutdown(ctx)
 		return
@@ -144,6 +153,11 @@ func main() {
 		bridge.Start(ctx)
 		go hub.Run(ctx)
 
+		// Start runtime providers
+		if err := rm.StartAll(ctx); err != nil {
+			log.Printf("runtime provider error: %v", err)
+		}
+
 		// Start API server
 		go func() {
 			log.Printf("API server listening on %s", apiAddr)
@@ -159,6 +173,7 @@ func main() {
 		}()
 
 		<-ctx.Done()
+		rm.StopAll()
 		shutdownCtx := context.Background()
 		_ = mcpSvc.Shutdown(shutdownCtx)
 		_ = c.ServiceShutdown(shutdownCtx)
@@ -184,6 +199,7 @@ func main() {
 			ActivationPolicy: application.ActivationPolicyAccessory,
 		},
 		OnShutdown: func() {
+			rm.StopAll()
 			ctx := context.Background()
 			_ = mcpSvc.Shutdown(ctx)
 			bridge.Shutdown()
@@ -221,12 +237,17 @@ func main() {
 	})
 	systray.SetMenu(trayMenu)
 
-	// Start MCP transport and API server alongside Wails
+	// Start MCP transport, runtime providers, and API server alongside Wails
 	go func() {
 		ctx := context.Background()
 		bridge.Start(ctx)
 		go hub.Run(ctx)
 
+		// Start runtime providers
+		if err := rm.StartAll(ctx); err != nil {
+			log.Printf("runtime provider error: %v", err)
+		}
+
 		// Start API server
 		go func() {
 			log.Printf("API server listening on %s", apiAddr)

runtime.go

@@ -0,0 +1,267 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"net"
+	"net/http"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strconv"
+	"sync"
+	"time"
+
+	"forge.lthn.ai/core/api"
+	"forge.lthn.ai/core/api/pkg/provider"
+	"forge.lthn.ai/core/go-scm/manifest"
+	"forge.lthn.ai/core/go-scm/marketplace"
+	"github.com/gin-gonic/gin"
+)
+
+// RuntimeProvider represents a running provider process with its proxy.
+type RuntimeProvider struct {
+	Dir      string
+	Manifest *manifest.Manifest
+	Port     int
+	Cmd      *exec.Cmd
+}
+
+// RuntimeManager discovers, starts, and stops runtime provider processes.
+// Each provider runs as a separate binary on 127.0.0.1, reverse-proxied
+// through the IDE's Gin router via ProxyProvider.
+type RuntimeManager struct {
+	engine    *api.Engine
+	providers []*RuntimeProvider
+	mu        sync.Mutex
+}
+
+// NewRuntimeManager creates a RuntimeManager.
+func NewRuntimeManager(engine *api.Engine) *RuntimeManager {
+	return &RuntimeManager{
+		engine: engine,
+	}
+}
+
+// defaultProvidersDir returns ~/.core/providers/.
+func defaultProvidersDir() string {
+	home, err := os.UserHomeDir()
+	if err != nil {
+		home = os.TempDir()
+	}
+	return filepath.Join(home, ".core", "providers")
+}
+
+// StartAll discovers providers in ~/.core/providers/ and starts each one.
+// Providers that fail to start are logged and skipped — they do not prevent
+// other providers from starting.
+func (rm *RuntimeManager) StartAll(ctx context.Context) error {
+	rm.mu.Lock()
+	defer rm.mu.Unlock()
+
+	dir := defaultProvidersDir()
+	discovered, err := marketplace.DiscoverProviders(dir)
+	if err != nil {
+		return fmt.Errorf("runtime: discover providers: %w", err)
+	}
+
+	if len(discovered) == 0 {
+		log.Println("runtime: no providers found in", dir)
+		return nil
+	}
+
+	log.Printf("runtime: discovered %d provider(s) in %s", len(discovered), dir)
+
+	for _, dp := range discovered {
+		rp, err := rm.startProvider(ctx, dp)
+		if err != nil {
+			log.Printf("runtime: failed to start %s: %v", dp.Manifest.Code, err)
+			continue
+		}
+		rm.providers = append(rm.providers, rp)
+		log.Printf("runtime: started %s on port %d", dp.Manifest.Code, rp.Port)
+	}
+
+	return nil
+}
+
+// startProvider starts a single provider binary and registers its proxy.
+func (rm *RuntimeManager) startProvider(ctx context.Context, dp marketplace.DiscoveredProvider) (*RuntimeProvider, error) {
+	m := dp.Manifest
+
+	// Assign a free port.
+	port, err := findFreePort()
+	if err != nil {
+		return nil, fmt.Errorf("find free port: %w", err)
+	}
+
+	// Resolve binary path.
+	binaryPath := m.Binary
+	if !filepath.IsAbs(binaryPath) {
+		binaryPath = filepath.Join(dp.Dir, binaryPath)
+	}
+
+	// Build command args.
+	args := make([]string, len(m.Args))
+	copy(args, m.Args)
+	args = append(args, "--namespace", m.Namespace, "--port", strconv.Itoa(port))
+
+	// Start the process.
+	cmd := exec.CommandContext(ctx, binaryPath, args...)
+	cmd.Dir = dp.Dir
+	cmd.Stdout = os.Stdout
+	cmd.Stderr = os.Stderr
+
+	if err := cmd.Start(); err != nil {
+		return nil, fmt.Errorf("start binary %s: %w", binaryPath, err)
+	}
+
+	// Wait for health check.
+	healthURL := fmt.Sprintf("http://127.0.0.1:%d/health", port)
+	if err := waitForHealth(healthURL, 10*time.Second); err != nil {
+		// Kill the process if health check fails.
+		_ = cmd.Process.Kill()
+		return nil, fmt.Errorf("health check failed for %s: %w", m.Code, err)
+	}
+
+	// Register proxy provider.
+	cfg := provider.ProxyConfig{
+		Name:     m.Code,
+		BasePath: m.Namespace,
+		Upstream: fmt.Sprintf("http://127.0.0.1:%d", port),
+	}
+	if m.Element != nil {
+		cfg.Element = provider.ElementSpec{
+			Tag:    m.Element.Tag,
+			Source: m.Element.Source,
+		}
+	}
+	if m.Spec != "" {
+		cfg.SpecFile = filepath.Join(dp.Dir, m.Spec)
+	}
+
+	proxy := provider.NewProxy(cfg)
+	rm.engine.Register(proxy)
+
+	// Serve JS assets if the provider has an element source.
+	if m.Element != nil && m.Element.Source != "" {
+		assetsDir := filepath.Join(dp.Dir, "assets")
+		if _, err := os.Stat(assetsDir); err == nil {
+			// Assets are served at /assets/{code}/
+			rm.engine.Register(&staticAssetGroup{
+				name:     m.Code + "-assets",
+				basePath: "/assets/" + m.Code,
+				dir:      assetsDir,
+			})
+		}
+	}
+
+	rp := &RuntimeProvider{
+		Dir:      dp.Dir,
+		Manifest: m,
+		Port:     port,
+		Cmd:      cmd,
+	}
+
+	return rp, nil
+}
+
+// StopAll terminates all running provider processes.
+func (rm *RuntimeManager) StopAll() {
+	rm.mu.Lock()
+	defer rm.mu.Unlock()
+
+	for _, rp := range rm.providers {
+		if rp.Cmd != nil && rp.Cmd.Process != nil {
+			log.Printf("runtime: stopping %s (pid %d)", rp.Manifest.Code, rp.Cmd.Process.Pid)
+			_ = rp.Cmd.Process.Signal(os.Interrupt)
+
+			// Give the process 5 seconds to exit gracefully.
+			done := make(chan error, 1)
+			go func() { done <- rp.Cmd.Wait() }()
+
+			select {
+			case <-done:
+				// Exited cleanly.
+			case <-time.After(5 * time.Second):
+				_ = rp.Cmd.Process.Kill()
+			}
+		}
+	}
+
+	rm.providers = nil
+}
+
+// List returns a copy of all running provider info.
+func (rm *RuntimeManager) List() []RuntimeProviderInfo {
+	rm.mu.Lock()
+	defer rm.mu.Unlock()
+
+	infos := make([]RuntimeProviderInfo, 0, len(rm.providers))
+	for _, rp := range rm.providers {
+		infos = append(infos, RuntimeProviderInfo{
+			Code:      rp.Manifest.Code,
+			Name:      rp.Manifest.Name,
+			Version:   rp.Manifest.Version,
+			Namespace: rp.Manifest.Namespace,
+			Port:      rp.Port,
+			Dir:       rp.Dir,
+		})
+	}
+	return infos
+}
+
+// RuntimeProviderInfo is a serialisable summary of a running provider.
+type RuntimeProviderInfo struct {
+	Code      string `json:"code"`
+	Name      string `json:"name"`
+	Version   string `json:"version"`
+	Namespace string `json:"namespace"`
+	Port      int    `json:"port"`
+	Dir       string `json:"dir"`
+}
+
+// findFreePort asks the OS for an available TCP port on 127.0.0.1.
+func findFreePort() (int, error) {
+	l, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		return 0, err
+	}
+	defer l.Close()
+	return l.Addr().(*net.TCPAddr).Port, nil
+}
+
+// waitForHealth polls a health URL until it returns 200 or the timeout expires.
+func waitForHealth(url string, timeout time.Duration) error {
+	deadline := time.Now().Add(timeout)
+	client := &http.Client{Timeout: 2 * time.Second}
+
+	for time.Now().Before(deadline) {
+		resp, err := client.Get(url)
+		if err == nil {
+			resp.Body.Close()
+			if resp.StatusCode == http.StatusOK {
+				return nil
+			}
+		}
+		time.Sleep(100 * time.Millisecond)
+	}
+
+	return fmt.Errorf("health check timed out after %s: %s", timeout, url)
+}
+
+// staticAssetGroup is a simple RouteGroup that serves static files.
+// Used to serve provider JS assets.
+type staticAssetGroup struct {
+	name     string
+	basePath string
+	dir      string
+}
+
+func (s *staticAssetGroup) Name() string     { return s.name }
+func (s *staticAssetGroup) BasePath() string { return s.basePath }
+
+func (s *staticAssetGroup) RegisterRoutes(rg *gin.RouterGroup) {
+	rg.Static("/", s.dir)
+}