cli/docs/pkg-batch6-analysis.md

# Package Analysis — Batch 6

Generated by: gemini-batch-runner.sh
Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview
Date: 2026-02-09
Packages: ansible deploy devops framework mcp plugin unifi webview ws collect i18n cache
Total tokens: 458153

---

# Framework Documentation

This document provides a detailed technical analysis of the core packages within the framework.

---

## === Package: pkg/ansible ===

### 1. Overview
A native Go implementation of an Ansible playbook runner. Unlike wrappers that call the `ansible` CLI, this package parses YAML playbooks and inventories, handles variable interpolation (Jinja2-style), and executes modules directly over SSH or local connections. It is designed for embedding configuration management directly into the application without external dependencies.

### 2. Public API

#### Executor
The primary entry point for running playbooks.
```go
type Executor struct { ... }
func NewExecutor(basePath string) *Executor
func (e *Executor) SetInventory(path string) error
func (e *Executor) SetInventoryDirect(inv *Inventory)
func (e *Executor) SetVar(key string, value any)
func (e *Executor) Run(ctx context.Context, playbookPath string) error
func (e *Executor) Close()
```

#### Callback Hooks
Callbacks to monitor execution progress.
```go
e.OnPlayStart = func(play *Play)
e.OnTaskStart = func(host string, task *Task)
e.OnTaskEnd   = func(host string, task *Task, result *TaskResult)
e.OnPlayEnd   = func(play *Play)
```

#### Types
```go
type Playbook struct { Plays []Play }
type Play struct { ... }
type Task struct { ... }
type Inventory struct { ... }
type TaskResult struct { Changed, Failed bool; Msg, Stdout string; ... }
```

### 3. Internal Design
*   **Parser**: `parser.go` handles recursive YAML parsing, resolving role paths, and `include_tasks`.
*   **Module Dispatch**: `modules.go` contains a switch statement dispatching tasks to native Go functions (e.g., `moduleShell`, `moduleCopy`, `moduleApt`) based on the task name.
*   **Templating**: Implements a custom Jinja2-subset parser in `executor.go` (`templateString`, `resolveExpr`) to handle variables like `{{ var }}` and filters.
*   **SSH Abstraction**: `ssh.go` wraps `golang.org/x/crypto/ssh` to handle connection pooling, key management, and `sudo` escalation (become).

### 4. Dependencies
*   `forge.lthn.ai/core/cli/pkg/log`: structured logging.
*   `golang.org/x/crypto/ssh`: Underlying SSH transport.
*   `gopkg.in/yaml.v3`: YAML parsing.

### 5. Test Coverage Notes
*   **Templating Logic**: Critical to test variable resolution, filters (`default`, `bool`), and nested lookups.
*   **Module Idempotency**: Verify that file/apt modules return `Changed: false` when state matches.
*   **SSH/Sudo**: Test `become` functionality with password handling.

### 6. Integration Points
Used by higher-level orchestration tools or the `pkg/devops` package to provision environments.

---

## === Package: pkg/devops ===

### 1. Overview
Manages a portable, sandboxed development environment using LinuxKit images. It handles the lifecycle (download, install, boot, stop) of a QEMU-based VM and provides utilities to bridge the host and VM (SSH forwarding, file mounting).

### 2. Public API

#### Lifecycle Management
```go
type DevOps struct { ... }
func New(m io.Medium) (*DevOps, error)
func (d *DevOps) Install(ctx, progress func(int64, int64)) error
func (d *DevOps) Boot(ctx, opts BootOptions) error
func (d *DevOps) Stop(ctx) error
func (d *DevOps) Status(ctx) (*DevStatus, error)
```

#### Interaction
```go
func (d *DevOps) Shell(ctx, opts ShellOptions) error
func (d *DevOps) Serve(ctx, projectDir, opts ServeOptions) error
func (d *DevOps) Test(ctx, projectDir, opts TestOptions) error
func (d *DevOps) Claude(ctx, projectDir, opts ClaudeOptions) error
```

### 3. Internal Design
*   **Image Management**: `images.go` handles versioning and downloading QCOW2 images from GitHub/CDN.
*   **Container Abstraction**: Delegates low-level VM execution to `pkg/container` (likely a wrapper around QEMU/LinuxKit).
*   **SSH Bridging**: Heavily relies on `exec.Command("ssh", ...)` to tunnel ports, mount filesystems via SSHFS, and forward agents.
*   **Auto-Detection**: `DetectServeCommand` and `DetectTestCommand` inspect project files (`package.json`, `go.mod`) to determine how to run projects.

### 4. Dependencies
*   `pkg/container`: VM runtime management.
*   `pkg/io`: Filesystem abstraction.
*   `pkg/config`: Configuration loading.

### 5. Test Coverage Notes
*   **Command Detection**: Unit tests for `Detect*Command` with various mock file structures.
*   **SSH Config**: Verify `ensureHostKey` correctly parses and updates `known_hosts`.

### 6. Integration Points
The primary interface for CLI commands (`core dev ...`). Bridges `pkg/mcp` agents into a sandboxed environment.

---

## === Package: pkg/framework ===

### 1. Overview
A facade package that re-exports types and functions from `pkg/framework/core`. It serves as the primary entry point for the Dependency Injection (DI) framework, providing a cleaner import path for consumers.

### 2. Public API
Re-exports `Core`, `Option`, `Message`, `Startable`, `Stoppable`, and constructors like `New`, `WithService`, `ServiceFor`.

### 3. Internal Design
Purely structural; contains type aliases and variable assignments to expose the internal `core` package.

### 4. Dependencies
*   `forge.lthn.ai/core/cli/pkg/framework/core`

### 5. Test Coverage Notes
No logic to test directly; coverage belongs in `pkg/framework/core`.

### 6. Integration Points
Imported by `main.go` and all service packages to register themselves with the DI container.

---

## === Package: pkg/mcp ===

### 1. Overview
Implements a Model Context Protocol (MCP) server. It acts as a bridge between AI models (like Claude) and the system tools, exposing file operations, process management, web browsing, and RAG capabilities as callable tools.

### 2. Public API
```go
type Service struct { ... }
func New(opts ...Option) (*Service, error)
func (s *Service) Run(ctx context.Context) error
func (s *Service) ServeTCP(ctx, addr string) error
func (s *Service) ServeStdio(ctx) error
```

#### Configuration Options
```go
func WithWorkspaceRoot(root string) Option
func WithProcessService(svc *process.Service) Option
func WithWSHub(hub *ws.Hub) Option
```

### 3. Internal Design
*   **Tool Registry**: Registers functions (e.g., `s.readFile`, `s.processStart`) with the MCP SDK.
*   **Sandboxing**: `WithWorkspaceRoot` creates a restricted `io.Medium` to prevent AI from accessing files outside the workspace.
*   **Subsystems**: Segregates tools into files (`tools_rag.go`, `tools_webview.go`, etc.).
*   **Transports**: Supports Stdio (for CLI pipes), TCP, and Unix sockets.

### 4. Dependencies
*   `github.com/modelcontextprotocol/go-sdk`: MCP protocol implementation.
*   `pkg/process`, `pkg/ws`, `pkg/rag`, `pkg/webview`: Capability providers.
*   `pkg/io`: Filesystem access.

### 5. Test Coverage Notes
*   **Security**: Verify `WithWorkspaceRoot` actually prevents accessing `/etc/passwd`.
*   **Tool I/O**: Ensure JSON inputs/outputs for tools map correctly to internal service calls.

### 6. Integration Points
Runs as a standalone server or subprocess for AI agents. Consumes `pkg/process` and `pkg/webview`.

---

## === Package: pkg/plugin ===

### 1. Overview
Provides a plugin system for the CLI, allowing extensions to be installed from GitHub. It manages a local registry of installed plugins and handles their lifecycle (install, update, remove).

### 2. Public API
```go
type Plugin interface { Name(); Version(); Init(); Start(); Stop() }
type Registry struct { ... }
func NewRegistry(m io.Medium, basePath string) *Registry
func (r *Registry) List() []*PluginConfig

type Installer struct { ... }
func (i *Installer) Install(ctx, source string) error // source: "org/repo"
func (i *Installer) Update(ctx, name string) error
```

### 3. Internal Design
*   **Manifest**: Relies on `plugin.json` in the root of the plugin repo.
*   **Git Integration**: Uses the `gh` CLI via `exec` to clone/pull repositories.
*   **Persistence**: Stores plugin metadata in a `registry.json` file.

### 4. Dependencies
*   `pkg/io`: Filesystem access.
*   `pkg/framework/core`: Error handling.
*   External `gh` and `git` binaries.

### 5. Test Coverage Notes
*   **Manifest Validation**: Test valid/invalid `plugin.json` parsing.
*   **Source Parsing**: Test parsing of `org/repo`, `org/repo@v1`, etc.

### 6. Integration Points
Used by the main CLI application to load dynamic commands at startup.

---

## === Package: pkg/unifi ===

### 1. Overview
A strongly-typed client for Ubiquiti UniFi controllers. It wraps the `unpoller` SDK but adds configuration resolution (config file -> env var -> flags) and specific helper methods for data extraction not easily accessible in the raw SDK.

### 2. Public API
```go
func NewFromConfig(...) (*Client, error)
func (c *Client) GetClients(filter ClientFilter) ([]*uf.Client, error)
func (c *Client) GetDeviceList(site, type string) ([]DeviceInfo, error)
func (c *Client) GetRoutes(site string) ([]Route, error)
func (c *Client) GetNetworks(site string) ([]NetworkConf, error)
```

### 3. Internal Design
*   **Config Cascade**: `ResolveConfig` logic ensures hierarchical configuration overrides.
*   **Raw API Fallback**: Methods like `GetRoutes` and `GetNetworks` bypass the SDK's high-level structs to hit specific API endpoints (`/api/s/%s/stat/routing`) and decode into custom structs.

### 4. Dependencies
*   `github.com/unpoller/unifi/v5`: Base SDK.
*   `pkg/config`: Config file management.

### 5. Test Coverage Notes
*   **Config Resolution**: Verify priority order (Flag > Env > Config).
*   **JSON Unmarshalling**: Test `GetRoutes`/`GetNetworks` against sample JSON responses from a controller.

### 6. Integration Points
Used by network management plugins or diagnostic tools.

---

## === Package: pkg/webview ===

### 1. Overview
A browser automation package using the Chrome DevTools Protocol (CDP). It is designed for headless testing, scraping, and AI-driven interaction. It supports advanced features like Angular-specific waiting strategies.

### 2. Public API
```go
type Webview struct { ... }
func New(opts ...Option) (*Webview, error)
func (wv *Webview) Navigate(url string) error
func (wv *Webview) Click(selector string) error
func (wv *Webview) Type(selector, text string) error
func (wv *Webview) Screenshot() ([]byte, error)
func (wv *Webview) Evaluate(script string) (any, error)
```

#### Angular Helpers
```go
func NewAngularHelper(wv *Webview) *AngularHelper
func (ah *AngularHelper) WaitForAngular() error
func (ah *AngularHelper) GetNgModel(selector string) (any, error)
```

### 3. Internal Design
*   **CDP Client**: `cdp.go` implements a raw WebSocket client for the DevTools protocol, managing message IDs and event dispatching.
*   **Action Sequence**: `actions.go` implements the Command pattern (`Action` interface) to chain browser interactions.
*   **Angular Awareness**: `angular.go` injects JS to probe `window.ng` or `getAllAngularRootElements` to interact with Angular's Zone.js and component state.

### 4. Dependencies
*   `github.com/gorilla/websocket`: WebSocket transport.

### 5. Test Coverage Notes
*   **CDP Protocol**: Mock the WebSocket server to ensure correct message serialization/response handling.
*   **Angular Helpers**: Requires an actual Angular app (or mock environment) to verify Zone.js stabilization logic.

### 6. Integration Points
Used by `pkg/mcp` to expose browser tools to AI agents.

---

## === Package: pkg/ws ===

### 1. Overview
A concurrent WebSocket hub implementation. It handles client registration, broadcasting, and channel-based subscriptions (e.g., subscribing only to logs for a specific process).

### 2. Public API
```go
type Hub struct { ... }
func NewHub() *Hub
func (h *Hub) Run(ctx)
func (h *Hub) Handler() http.HandlerFunc
func (h *Hub) SendProcessOutput(id, output string) error
func (h *Hub) SendEvent(type string, data any) error
```

### 3. Internal Design
*   **Hub Pattern**: Central `Hub` struct manages a map of clients and channels. Uses unbuffered channels for registration to avoid race conditions.
*   **Channel Routing**: Maintains a `map[string]map[*Client]bool` to route messages efficiently to subscribers.
*   **Goroutines**: Each client spawns a `readPump` and `writePump` to handle I/O concurrently.

### 4. Dependencies
*   `github.com/gorilla/websocket`

### 5. Test Coverage Notes
*   **Concurrency**: Test registering/unregistering clients while broadcasting heavily.
*   **Subscription**: Verify messages only go to subscribed clients.

### 6. Integration Points
Used by `pkg/mcp` to stream process output to a web UI.

---

## === Package: pkg/collect ===

### 1. Overview
A data collection pipeline for gathering data from various sources (GitHub, Forums, Market Data, Papers). It standardizes the collection process into a `Collector` interface and handles common concerns like rate limiting, state tracking (resume support), and formatting.

### 2. Public API
```go
type Collector interface {
    Name() string
    Collect(ctx, cfg *Config) (*Result, error)
}
type Excavator struct { Collectors []Collector ... }
func (e *Excavator) Run(ctx, cfg) (*Result, error)
```

### 3. Internal Design
*   **Excavator**: The orchestrator that runs collectors sequentially.
*   **RateLimiter**: Implements token bucket-like delays per source type (e.g., GitHub, CoinGecko).
*   **State Persistence**: Saves a JSON cursor file to resume interrupted collections.
*   **Formatters**: `process.go` converts raw HTML/JSON into Markdown for easier consumption by LLMs.

### 4. Dependencies
*   `pkg/io`: File storage.
*   `golang.org/x/net/html`: HTML parsing for forums/papers.
*   `gh` CLI: Used for GitHub data fetching.

### 5. Test Coverage Notes
*   **HTML Parsing**: Test `ParsePostsFromHTML` with sample forum HTML.
*   **Rate Limit**: Verify `Wait` respects context cancellation and time delays.

### 6. Integration Points
Used as a standalone CLI command or by AI agents to gather context.

---

## === Package: pkg/i18n ===

### 1. Overview
A sophisticated internationalization library that goes beyond simple key-value lookups. It includes a grammar engine to handle pluralization, verb conjugation, and semantic sentence generation ("Subject verbed object").

### 2. Public API
```go
func T(key string, args ...any) string // Main translation function
func S(noun string, value any) *Subject // Create a semantic subject
func N(format string, value any) string // Number formatting
func SetLanguage(lang string) error
```

### 3. Internal Design
*   **Grammar Engine**: `grammar.go` applies rules for past tense, gerunds, and pluralization based on language-specific JSON rules or algorithmic fallbacks.
*   **Namespace Handlers**: `handler.go` intercepts keys like `i18n.count.*` or `i18n.done.*` to auto-generate phrases based on the grammar engine.
*   **Loader**: `loader.go` flattens nested JSON translation files and extracts grammar rules (`gram.verb.*`).

### 4. Dependencies
*   `golang.org/x/text/language`: Standard language tag parsing.

### 5. Test Coverage Notes
*   **Pluralization**: Test complex rules (e.g., Slavic/Arabic plural categories).
*   **Grammar generation**: Test `PastTense` and `Gerund` for regular and irregular English verbs.

### 6. Integration Points
Used pervasively across the CLI for all user-facing output.

---

## === Package: pkg/cache ===

### 1. Overview
A simple, file-based JSON cache with Time-To-Live (TTL) support.

### 2. Public API
```go
func New(m io.Medium, baseDir string, ttl time.Duration) (*Cache, error)
func (c *Cache) Get(key string, dest interface{}) (bool, error)
func (c *Cache) Set(key string, data interface{}) error
```

### 3. Internal Design
*   Stores data as JSON files: `{ "data": ..., "expires_at": ... }`.
*   Uses `pkg/io` abstraction for storage independence.

### 4. Dependencies
*   `pkg/io`

### 5. Test Coverage Notes
*   **Expiry**: Verify `Get` returns false after TTL expires.
*   **Serialization**: Ensure struct round-tripping works correctly.

### 6. Integration Points
Used by `pkg/collect` or `pkg/plugin` to cache API responses (e.g., GitHub releases).

---

## Quick Reference (Flash Summary)

### pkg/ansible
**Description:** Implements a Go-based Ansible-lite engine for executing playbooks and roles over SSH with YAML parsing and fact gathering.
- **Executor (Type):** Main runner that manages inventory, variables, and execution state.
- **NewExecutor (Func):** Initialises the executor with a base path for roles and playbooks.
- **Task (Type):** Represents a single Ansible task with module parameters and conditional logic.
- **Run (Func):** Parses and executes a playbook from a file path.
- **Inventory (Type):** Holds the host and group structure for targeting remote machines.
**Dependencies:** `pkg/log`
**Complexity:** Complex

### pkg/devops
**Description:** Manages a portable development environment using LinuxKit VM images and QEMU/SSH integration.
- **DevOps (Type):** Core service for environment lifecycle, mounting, and tool execution.
- **Boot (Func):** Configures and starts the dev environment container.
- **Claude (Func):** Launches a sandboxed AI session with project mounting and auth forwarding.
- **Serve (Func):** Auto-detects project types and runs local development servers inside the VM.
- **ImageManager (Type):** Handles downloading and updating dev environment system images.
**Dependencies:** `pkg/config`, `pkg/container`, `pkg/io`, `pkg/devops/sources`
**Complexity:** Moderate

### pkg/framework
**Description:** Provides a facade for the Core dependency injection and service runtime framework.
- **Core (Type):** The central DI container and service registry.
- **New (Func):** Creates and initialises a new Core instance.
- **ServiceFor (Func):** Retrieves a type-safe service from the container by name.
- **Runtime (Type):** Manages the lifecycle and configuration of application services.
**Dependencies:** `pkg/framework/core`
**Complexity:** Simple

### pkg/mcp
**Description:** Implements a Model Context Protocol (MCP) server providing filesystem, process, and RAG tools to AI agents.
- **Service (Type):** The MCP server instance managing tools and transport.
- **New (Func):** Initialises the server with workspace sandboxing and optional service integration.
- **WithWorkspaceRoot (Option):** Restricts file operations to a specific directory for security.
- **ServeTCP / ServeStdio (Func):** Transport-specific server implementations.
- **Subsystem (Interface):** Allows external packages to register custom toolsets.
**Dependencies:** `pkg/io`, `pkg/io/local`, `pkg/log`, `pkg/process`, `pkg/ws`, `pkg/ai`, `pkg/rag`, `pkg/webview`
**Complexity:** Complex

### pkg/plugin
**Description:** A dynamic plugin system that manages gits-based extensions for the core CLI.
- **Plugin (Interface):** Defines the lifecycle (Init/Start/Stop) for extensions.
- **Registry (Type):** Manages metadata and persistence for installed plugins.
- **Installer (Type):** Handles git-based installation and updates from GitHub.
- **Loader (Type):** Discovers and loads plugin manifests from the filesystem.
**Dependencies:** `pkg/framework/core`, `pkg/io`
**Complexity:** Moderate

### pkg/unifi
**Description:** A wrapper for the UniFi SDK providing simplified access to network controller devices, clients, and routing.
- **Client (Type):** Main API client for interacting with UniFi controllers.
- **NewFromConfig (Func):** Resolves credentials from config/env and initialises a client.
- **GetClients (Func):** Returns a filtered list of connected wired and wireless clients.
- **GetDeviceList (Func):** Returns flat metadata for infrastructure hardware (APs, Switches, Gateways).
**Dependencies:** `pkg/config`, `pkg/log`
**Complexity:** Simple

### pkg/webview
**Description:** Provides browser automation and framework-specific testing (Angular) via the Chrome DevTools Protocol (CDP).
- **Webview (Type):** High-level controller for browser navigation and interaction.
- **CDPClient (Type):** Manages raw WebSocket communication with Chrome.
- **AngularHelper (Type):** Specialized tools for waiting on Zone.js stability and interacting with Angular components.
- **ConsoleWatcher (Type):** Captures and filters browser console logs and exceptions.
- **ActionSequence (Type):** Chains multiple browser interactions (click, type, navigate) into a single execution.
**Dependencies:** None
**Complexity:** Complex

### pkg/ws
**Description:** Implements a WebSocket hub for real-time message broadcasting and channel-based subscriptions.
- **Hub (Type):** Manages client connections, message loops, and channel routing.
- **Run (Func):** Starts the central event loop for broadcasting and registration.
- **Broadcast (Func):** Sends a message to every connected client.
- **SendToChannel (Func):** Targets messages to clients subscribed to specific topics (e.g., process logs).
**Dependencies:** None
**Complexity:** Moderate

### pkg/collect
**Description:** An orchestration subsystem for scraping and processing data from GitHub, forums, market APIs, and academic sources.
- **Collector (Interface):** Standard interface for data sources (e.g., `GitHubCollector`, `BitcoinTalkCollector`).
- **Excavator (Type):** Orchestrates multiple collectors with rate limiting and state resume support.
- **Processor (Type):** Converts raw HTML/JSON data into cleaned Markdown files.
- **RateLimiter (Type):** Manages per-source API delays to prevent IP bans.
- **State (Type):** Persists progress to allow incremental collection runs.
**Dependencies:** `pkg/framework/core`, `pkg/io`
**Complexity:** Complex

### pkg/i18n
**Description:** A localization engine supporting nested translations, grammatical rules (plurals, gender, verbs), and semantic composition.
- **Service (Type):** Manages loaded locales and message resolution logic.
- **T / Raw (Func):** Translates keys with or without automatic grammatical composition.
- **Subject (Type):** Provides context (count, gender, formality) for semantic intent templates.
- **RegisterLocales (Func):** Allows packages to register embedded translation files.
- **GrammarData (Type):** Defines language-specific rules for past tense, gerunds, and articles.
**Dependencies:** None
**Complexity:** Complex

### pkg/cache
**Description:** Provides a persistent, file-based JSON cache with TTL-based expiration.
- **Cache (Type):** Main handler for storing and retrieving cached entries.
- **Entry (Type):** Internal wrapper for data, including cached and expiry timestamps.
- **Get / Set (Func):** Thread-safe operations for managing cached data.
- **Age (Func):** Calculates how long an item has been stored.
**Dependencies:** `pkg/io`
**Complexity:** Simple