cli/docs/pkg-batch3-analysis.md

Package Analysis — Batch 3

Generated by: gemini-batch-runner.sh Models: gemini-2.5-flash-lite → gemini-3-flash-preview → gemini-3-pro-preview Date: 2026-02-09 Packages: build container process jobrunner Total tokens: 96300

---

Here is the technical documentation for the Core framework packages, analyzing the provided source code.

Core Framework Package Documentation

Table of Contents

1. pkg/build
2. pkg/container
3. pkg/process
4. pkg/jobrunner

---

Package: pkg/build

1. Overview

The build package provides a standardized system for detecting project types, loading build configurations, and packaging artifacts. It is designed around an abstraction of the filesystem (io.Medium) to facilitate testing and cross-platform compatibility, handling compression formats (gzip, xz, zip) and SHA256 checksum generation.

2. Public API

Project Detection & Configuration

// Represents a detected project type (e.g., "go", "wails", "node")
type ProjectType string

// Detects project types in a directory based on marker files
func Discover(fs io.Medium, dir string) ([]ProjectType, error)
func PrimaryType(fs io.Medium, dir string) (ProjectType, error)

// Helper predicates for detection
func IsGoProject(fs io.Medium, dir string) bool
func IsWailsProject(fs io.Medium, dir string) bool
func IsNodeProject(fs io.Medium, dir string) bool
func IsPHPProject(fs io.Medium, dir string) bool
func IsCPPProject(fs io.Medium, dir string) bool

// Loads configuration from .core/build.yaml
func LoadConfig(fs io.Medium, dir string) (*BuildConfig, error)
func ConfigExists(fs io.Medium, dir string) bool

Artifact Management

type Artifact struct {
    Path, OS, Arch, Checksum string
}

type ArchiveFormat string // "gz", "xz", "zip"

// Archiving functions
func Archive(fs io.Medium, artifact Artifact) (Artifact, error) // Default gzip
func ArchiveXZ(fs io.Medium, artifact Artifact) (Artifact, error)
func ArchiveWithFormat(fs io.Medium, artifact Artifact, format ArchiveFormat) (Artifact, error)
func ArchiveAll(fs io.Medium, artifacts []Artifact) ([]Artifact, error)

// Checksum functions
func Checksum(fs io.Medium, artifact Artifact) (Artifact, error)
func ChecksumAll(fs io.Medium, artifacts []Artifact) ([]Artifact, error)
func WriteChecksumFile(fs io.Medium, artifacts []Artifact, path string) error

Interfaces

// Interface for project-specific build logic
type Builder interface {
    Name() string
    Detect(fs io.Medium, dir string) (bool, error)
    Build(ctx context.Context, cfg *Config, targets []Target) ([]Artifact, error)
}

3. Internal Design

• Filesystem Abstraction: Heavily relies on dependency injection via io.Medium rather than direct os calls, enabling safe unit testing of file operations.
• Strategy Pattern: The Builder interface allows different build logic (Go, Docker, Node) to be swapped dynamically based on detection.
• Priority Detection: Discovery uses an ordered slice of markers (markers var) to handle hybrid projects (e.g., Wails is detected before Go).
• Configuration Overlay: Uses mapstructure to parse YAML config, applying sensible defaults via applyDefaults if fields are missing.

4. Dependencies

• archive/tar, archive/zip, compress/gzip: Standard library for archiving.
• github.com/Snider/Borg/pkg/compress: External dependency for XZ compression support.
• github.com/host-uk/core/pkg/io: Internal interface for filesystem abstraction.
• github.com/host-uk/core/pkg/config: Internal centralized configuration loading.

5. Test Coverage Notes

• Mocking IO: Tests must implement a mock io.Medium to simulate file existence (Detect) and write operations (Archive) without touching the disk.
• Format Specifics: Verify that Windows builds automatically default to .zip regardless of the requested format in ArchiveWithFormat.
• Config Parsing: Test LoadConfig with malformed YAML and missing fields to ensure defaults are applied correctly.

6. Integration Points

• CLI Build Commands: This package is the backend for any core build CLI command.
• CI Pipelines: Used to generate release artifacts and CHECKSUMS.txt files for releases.

---

Package: pkg/container

1. Overview

This package manages the lifecycle of local LinuxKit virtual machines. It abstracts underlying hypervisors (QEMU on Linux, Hyperkit on macOS) to provide a container-like experience (start, stop, logs, exec) for running VM images.

2. Public API

Manager & Lifecycle

type Manager interface {
    Run(ctx context.Context, image string, opts RunOptions) (*Container, error)
    Stop(ctx context.Context, id string) error
    List(ctx context.Context) ([]*Container, error)
    Logs(ctx context.Context, id string, follow bool) (io.ReadCloser, error)
    Exec(ctx context.Context, id string, cmd []string) error
}

// Factory
func NewLinuxKitManager(m io.Medium) (*LinuxKitManager, error)

Templates

type TemplateManager struct { ... }

func NewTemplateManager(m io.Medium) *TemplateManager
func (tm *TemplateManager) ListTemplates() []Template
func (tm *TemplateManager) GetTemplate(name string) (string, error)
func (tm *TemplateManager) ApplyTemplate(name string, vars map[string]string) (string, error)

Types

type Container struct {
    ID, Name, Image string
    Status Status // "running", "stopped", "error"
    PID int
    // ... ports, memory stats
}

type RunOptions struct {
    Name string
    Detach bool
    Memory, CPUs, SSHPort int
    Ports, Volumes map[string]string
}

3. Internal Design

• Hypervisor Abstraction: The Hypervisor interface hides the complexity of building CLI arguments for qemu-system-x86_64 vs hyperkit.
• State Persistence: Uses a JSON file (.core/containers.json) protected by a sync.RWMutex to track VM state across process restarts.
• Embedded Assets: Uses Go embed to package default LinuxKit YAML templates (templates/*.yml) inside the binary.
• Log Following: Implements a custom followReader to emulate tail -f behavior for VM logs.

4. Dependencies

• os/exec: Essential for spawning the hypervisor processes.
• embed: For built-in templates.
• github.com/host-uk/core/pkg/io: Filesystem access for state and logs.

5. Test Coverage Notes

• Process Management: Difficult to test Run in standard CI. Mocking exec.Command or the Hypervisor interface is required.
• State Integrity: Test LoadState and SaveState handles corruption or concurrent writes.
• Template Interpolation: Verify ApplyVariables correctly handles required vs optional ${VAR:-default} syntax.

6. Integration Points

• Dev Environments: Used to spin up isolated development environments defined by LinuxKit YAMLs.
• Testing: Can be used to launch disposable VMs for integration testing.

---

Package: pkg/process

1. Overview

A sophisticated wrapper around os/exec that integrates with the Core framework's event bus. It features output streaming, ring-buffer capturing, dependency-based task execution (DAG), and a global singleton service for ease of use.

2. Public API

Service & Global Access

// Global singletons (require Init)
func Init(c *framework.Core) error
func Start(ctx, cmd string, args ...string) (*Process, error)
func Run(ctx, cmd string, args ...string) (string, error)
func Kill(id string) error

// Service Factory
func NewService(opts Options) func(*framework.Core) (any, error)

Process Control

type Process struct { ... }

func (p *Process) Wait() error
func (p *Process) Kill() error
func (p *Process) Output() string
func (p *Process) IsRunning() bool
func (p *Process) SendInput(input string) error
func (p *Process) Done() <-chan struct{}

Task Runner

type Runner struct { ... }
type RunSpec struct {
    Name, Command string
    After []string // Dependencies
    // ... args, env
}

func NewRunner(svc *Service) *Runner
func (r *Runner) RunAll(ctx context.Context, specs []RunSpec) (*RunAllResult, error)
func (r *Runner) RunParallel(ctx context.Context, specs []RunSpec) (*RunAllResult, error)

3. Internal Design

• Event Sourcing: Instead of just logging, the service broadcasts events (ActionProcessStarted, ActionProcessOutput) via framework.Core. This allows UI frontends to subscribe to real-time output.
• Ring Buffer: Uses a fixed-size circular buffer (RingBuffer) to store logs, preventing memory exhaustion from long-running processes.
• DAG Execution: The Runner.RunAll method implements a dependency graph resolver to run tasks in parallel waves based on the After field.
• Global Singleton: Uses atomic.Pointer for a thread-safe global Default() service instance.

4. Dependencies

• os/exec: The underlying execution engine.
• github.com/host-uk/core/pkg/framework: Creates the ServiceRuntime and provides the IPC/Action bus.

5. Test Coverage Notes

• Concurrency: The Runner needs tests for race conditions during parallel execution.
• Dependency Resolution: Test circular dependencies (deadlock detection) and skip logic when a dependency fails.
• Buffer Overflow: Verify RingBuffer overwrites old data correctly when full.

6. Integration Points

• Task Runners: The Runner struct is the engine for tools like make or Taskfile.
• UI/TUI: The Action-based output streaming is designed to feed data into a TUI or Web frontend in real-time.

---

Package: pkg/jobrunner

1. Overview

A polling-based workflow engine designed to ingest "signals" (e.g., GitHub Issues/PRs), match them to specific handlers, and record execution results in a structured journal. It implements a "dry-run" capability and detailed audit logging.

2. Public API

Poller

type Poller struct { ... }
type PollerConfig struct {
    Sources []JobSource
    Handlers []JobHandler
    Journal *Journal
    // ... interval, dryRun
}

func NewPoller(cfg PollerConfig) *Poller
func (p *Poller) Run(ctx context.Context) error
func (p *Poller) AddSource(s JobSource)
func (p *Poller) AddHandler(h JobHandler)

Journaling

type Journal struct { ... }
func NewJournal(baseDir string) (*Journal, error)
func (j *Journal) Append(signal *PipelineSignal, result *ActionResult) error

Interfaces

type JobSource interface {
    Poll(ctx context.Context) ([]*PipelineSignal, error)
    Report(ctx context.Context, result *ActionResult) error
}

type JobHandler interface {
    Match(signal *PipelineSignal) bool
    Execute(ctx context.Context, signal *PipelineSignal) (*ActionResult, error)
}

3. Internal Design

• Poller Loop: Runs a continuous loop (ticker-based) that snapshots sources and handlers at the start of every cycle to allow dynamic registration.
• Data Models: Defines rigid structures (PipelineSignal, ActionResult) to decouple data sources (GitHub) from logic handlers.
• Journaling: Writes jsonl (JSON Lines) files partitioned by repository and date (baseDir/owner/repo/YYYY-MM-DD.jsonl), ensuring an append-only audit trail.

4. Dependencies

• github.com/host-uk/core/pkg/log: Internal logging.
• encoding/json: For journal serialization.

5. Test Coverage Notes

• Matching Logic: Test that findHandler picks the correct handler for a given signal.
• Dry Run: Ensure Execute is not called when dryRun is true, but logs are generated.
• Journal Locking: Verify concurrent writes to the journal do not corrupt the JSONL file.

6. Integration Points

• CI Bots: The primary framework for building bots that automate Pull Request management or Issue triage.
• Dashboarding: The generated JSONL journal files are structured to be ingested by analytics tools.

---

Quick Reference (Flash Summary)

Package: pkg/build

Provides project type detection, build configuration management, and cross-compilation utilities.

Key Exported Types and Functions

• Builder (interface): Defines the interface for project-specific build implementations (Go, Node, PHP, etc.).
• Config / BuildConfig (structs): Hold runtime and file-based build parameters.
• Artifact (struct): Represents a build output file with path, OS, architecture, and checksum metadata.
• ProjectType (type): Constants identifying project types (e.g., ProjectTypeGo, ProjectTypeWails).
• Archive, ArchiveXZ, ArchiveWithFormat: Functions to create compressed archives (tar.gz, tar.xz, zip) of build artifacts.
• Checksum, ChecksumAll: Compute SHA256 hashes for build artifacts.
• Discover, PrimaryType: Detect project types based on marker files (e.g., go.mod, package.json).
• LoadConfig: Loads build settings from .core/build.yaml.

Dependencies

• pkg/io
• pkg/config
• pkg/build/signing

Complexity Rating Moderate

---

Package: pkg/container

Manages the lifecycle of LinuxKit virtual machines using platform-native hypervisors.

Key Exported Types and Functions

• Manager (interface): Defines container lifecycle operations (Run, Stop, List, Logs, Exec).
• LinuxKitManager (struct): Core implementation for managing LinuxKit VM instances.
• Container (struct): Represents a running or stopped VM instance with metadata like PID and status.
• Hypervisor (interface): Abstract interface for VM backends (QEMU, Hyperkit).
• TemplateManager (struct): Handles LinuxKit YAML templates and variable substitution.
• State (struct): Manages persistent storage of container metadata in JSON format.
• DetectHypervisor: Automatically selects the appropriate hypervisor for the current OS.
• ApplyVariables: Performs ${VAR:-default} string interpolation in configuration files.

Dependencies

• pkg/io

Complexity Rating Complex

---

Package: pkg/process

Advanced process management system featuring output streaming, circular buffering, and dependency-aware task execution.

Key Exported Types and Functions

• Service (struct): Manages multiple processes with Core framework IPC integration.
• Process (struct): Represents a managed external process with non-blocking output capture.
• Runner (struct): Orchestrates complex task execution with dependency graph support (DAG).
• RingBuffer (struct): A thread-safe circular buffer for efficient process output storage.
• RunOptions (struct): Detailed configuration for spawning processes (env, dir, capture settings).
• ActionProcessOutput, ActionProcessExited: IPC message types for broadcasting process events via the Core framework.
• Start, Run, Kill: Global convenience functions for rapid process control.

Dependencies

• pkg/framework

Complexity Rating Moderate/Complex

---

Package: pkg/jobrunner

A poll-dispatch automation system designed to process structural signals from issues or pull requests.

Key Exported Types and Functions

• Poller (struct): Implements the main loop that discovers work from sources and dispatches to handlers.
• PipelineSignal (struct): A metadata snapshot of a work item (e.g., PR state, thread counts, mergeability).
• JobSource (interface): Interface for external systems (like GitHub) that provide actionable items.
• JobHandler (interface): Interface for logic that matches and executes actions on signals.
• Journal (struct): Provides persistent, date-partitioned JSONL audit logging for all actions.
• ActionResult (struct): Captures the success, failure, and duration of a completed job.

Dependencies

• pkg/log

Complexity Rating Moderate