core/go-process
8
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

docs: add human-friendly documentation

Browse source 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Snider 2026-03-11 13:02:40 +00:00
parent 81de841903
commit ec95200765
3 changed files with 636 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

348
docs/architecture.md Normal file
View file

@ -0,0 +1,348 @@
---
title: Architecture
description: Internals of go-process — key types, data flow, and design decisions.
---
# Architecture
This document explains how `go-process` is structured, how data flows through
the system, and the role of each major component.
## Overview
The package is organised into four layers:
1. **Service** — The Core-integrated service that owns processes and broadcasts events.
2. **Process** — An individual managed process with output capture and lifecycle state.
3. **Runner** — A pipeline orchestrator that runs multiple processes with dependency resolution.
4. **Daemon** — A higher-level abstraction for long-running services with PID files, health checks, and registry integration.
A separate `exec/` sub-package provides a thin, fluent wrapper around `os/exec`
for simple one-shot commands.
## Key Types
### Status
Process lifecycle is tracked as a state machine:
```
pending -> running -> exited
-> failed
-> killed
```
```go
type Status string
const (
StatusPending Status = "pending"
StatusRunning Status = "running"
StatusExited Status = "exited"
StatusFailed Status = "failed"
StatusKilled Status = "killed"
)
```
- **pending** — queued but not yet started (currently unused by the service,
reserved for future scheduling).
- **running** — actively executing.
- **exited** — completed; check `ExitCode` for success (0) or failure.
- **failed** — could not be started (e.g. binary not found).
- **killed** — terminated by signal or context cancellation.
### Service
`Service` is the central type. It embeds `core.ServiceRuntime[Options]` to
participate in the Core DI container and implements both `Startable` and
`Stoppable` lifecycle interfaces.
```go
type Service struct {
*core.ServiceRuntime[Options]
processes map[string]*Process
mu sync.RWMutex
bufSize int
idCounter atomic.Uint64
}
```
Key behaviours:
- **OnStartup** — currently a no-op; reserved for future initialisation.
- **OnShutdown** — iterates all running processes and calls `Kill()` on each,
ensuring no orphaned child processes when the application exits.
- Process IDs are generated as `proc-N` using an atomic counter, guaranteeing
uniqueness without locks.
#### Registration
The service is registered with Core via a factory function:
```go
process.NewService(process.Options{BufferSize: 2 * 1024 * 1024})
```
`NewService` returns a `func(*core.Core) (any, error)` closure — the standard
Core service factory signature. The `Options` struct is captured by the closure
and applied when Core instantiates the service.
### Process
`Process` wraps an `os/exec.Cmd` with:
- Thread-safe state (`sync.RWMutex` guards all mutable fields).
- A `RingBuffer` for output capture (configurable size, default 1 MB).
- A `done` channel that closes when the process exits, enabling `select`-based
coordination.
- Stdin pipe access via `SendInput()` and `CloseStdin()`.
- Context-based cancellation — cancelling the context kills the process.
#### Info Snapshot
`Process.Info()` returns an `Info` struct — a serialisable snapshot of the
process state, suitable for JSON APIs or UI display:
```go
type Info struct {
ID string `json:"id"`
Command string `json:"command"`
Args []string `json:"args"`
Dir string `json:"dir"`
StartedAt time.Time `json:"startedAt"`
Status Status `json:"status"`
ExitCode int `json:"exitCode"`
Duration time.Duration `json:"duration"`
PID int `json:"pid"`
}
```
### RingBuffer
A fixed-size circular buffer that overwrites the oldest data when full.
Thread-safe for concurrent reads and writes.
```go
rb := process.NewRingBuffer(64 * 1024) // 64 KB
rb.Write([]byte("data"))
fmt.Println(rb.String()) // "data"
fmt.Println(rb.Len()) // 4
fmt.Println(rb.Cap()) // 65536
rb.Reset()
```
The ring buffer is used internally to capture process stdout and stderr. When
a process produces more output than the buffer capacity, the oldest data is
silently overwritten. This prevents unbounded memory growth for long-running
or verbose processes.
### ACTION Messages
Four IPC message types are broadcast through `Core.ACTION()`:
| Type | When | Key Fields |
|------|------|------------|
| `ActionProcessStarted` | Process begins execution | `ID`, `Command`, `Args`, `Dir`, `PID` |
| `ActionProcessOutput` | Each line of stdout/stderr | `ID`, `Line`, `Stream` |
| `ActionProcessExited` | Process completes | `ID`, `ExitCode`, `Duration`, `Error` |
| `ActionProcessKilled` | Process is terminated | `ID`, `Signal` |
The `Stream` type distinguishes stdout from stderr:
```go
type Stream string
const (
StreamStdout Stream = "stdout"
StreamStderr Stream = "stderr"
)
```
## Data Flow
When `Service.StartWithOptions()` is called:
```
1. Generate unique ID (atomic counter)
2. Create context with cancel
3. Build os/exec.Cmd with dir, env, pipes
4. Create RingBuffer (unless DisableCapture is set)
5. cmd.Start()
6. Store process in map
7. Broadcast ActionProcessStarted via Core.ACTION
8. Spawn 2 goroutines to stream stdout and stderr
- Each line is written to the RingBuffer
- Each line is broadcast as ActionProcessOutput
9. Spawn 1 goroutine to wait for process exit
- Waits for output goroutines to finish first
- Calls cmd.Wait()
- Updates process status and exit code
- Closes the done channel
- Broadcasts ActionProcessExited
```
The output streaming goroutines use `bufio.Scanner` with a 1 MB line buffer
to handle long lines without truncation.
## Runner
The `Runner` orchestrates multiple processes, defined as `RunSpec` values:
```go
type RunSpec struct {
Name string
Command string
Args []string
Dir string
Env []string
After []string // dependency names
AllowFailure bool
}
```
Three execution strategies are available:
### RunAll (dependency graph)
Processes dependencies in waves. In each wave, all specs whose dependencies
are satisfied run in parallel. If a dependency fails (and `AllowFailure` is
false), its dependents are skipped. Circular dependencies are detected and
reported as skipped with an error.
```
Wave 1: [lint, vet] (no dependencies)
Wave 2: [test] (depends on lint, vet)
Wave 3: [build] (depends on test)
```
### RunSequential
Executes specs one after another. Stops on the first failure unless
`AllowFailure` is set. Remaining specs are marked as skipped.
### RunParallel
Runs all specs concurrently, ignoring the `After` field entirely. Failures
do not affect other specs.
All three strategies return a `RunAllResult` with aggregate counts:
```go
type RunAllResult struct {
Results []RunResult
Duration time.Duration
Passed int
Failed int
Skipped int
}
```
## Daemon
The `Daemon` type manages the full lifecycle of a long-running service:
```
NewDaemon(opts) -> Start() -> Run(ctx) -> Stop()
```
### PID File
`PIDFile` provides single-instance enforcement. `Acquire()` writes the current
process PID to a file; if the file already exists and the recorded PID is still
alive (verified via `syscall.Signal(0)`), it returns an error. Stale PID files
from dead processes are automatically cleaned up.
```go
pid := process.NewPIDFile("/var/run/myapp.pid")
err := pid.Acquire() // writes current PID, fails if another instance is live
defer pid.Release() // removes the file
```
### Health Server
`HealthServer` exposes two HTTP endpoints:
- **`/health`** — runs all registered `HealthCheck` functions. Returns 200 if
all pass, 503 if any fail.
- **`/ready`** — returns 200 or 503 based on the readiness flag, toggled via
`SetReady(bool)`.
The server binds to a configurable address (use port `0` for ephemeral port
allocation in tests). `WaitForHealth()` is a polling utility that waits for
`/health` to return 200 within a timeout.
### Registry
`Registry` tracks running daemons via JSON files in a directory (default:
`~/.core/daemons/`). Each daemon is a `DaemonEntry`:
```go
type DaemonEntry struct {
Code string `json:"code"`
Daemon string `json:"daemon"`
PID int `json:"pid"`
Health string `json:"health,omitempty"`
Project string `json:"project,omitempty"`
Binary string `json:"binary,omitempty"`
Started time.Time `json:"started"`
}
```
The registry automatically prunes entries with dead PIDs on `List()` and
`Get()`. When a `Daemon` is configured with a `Registry`, it auto-registers
on `Start()` and auto-unregisters on `Stop()`.
File naming convention: `{code}-{daemon}.json` (slashes replaced with dashes).
## exec Sub-Package
The `exec` package (`forge.lthn.ai/core/go-process/exec`) provides a fluent
wrapper around `os/exec` for simple, one-shot commands that do not need Core
integration:
```go
import "forge.lthn.ai/core/go-process/exec"
// Fluent API
err := exec.Command(ctx, "go", "build", "./...").
WithDir("/path/to/project").
WithEnv([]string{"CGO_ENABLED=0"}).
WithLogger(myLogger).
Run()
// Get output
out, err := exec.Command(ctx, "git", "status").Output()
// Combined stdout + stderr
out, err := exec.Command(ctx, "make").CombinedOutput()
// Quiet mode (suppresses stdout, includes stderr in error)
err := exec.RunQuiet(ctx, "go", "vet", "./...")
```
### Logging
Commands are automatically logged at debug level before execution and at error
level on failure. The logger interface is minimal:
```go
type Logger interface {
Debug(msg string, keyvals ...any)
Error(msg string, keyvals ...any)
}
```
A `NopLogger` (the default) discards all messages. Use `SetDefaultLogger()` to
set a package-wide logger, or `WithLogger()` for per-command overrides.
## Thread Safety
All public types are safe for concurrent use:
- `Service` — `sync.RWMutex` protects the process map; atomic counter for IDs.
- `Process` — `sync.RWMutex` protects mutable state.
- `RingBuffer` — `sync.RWMutex` on all read/write operations.
- `PIDFile` — `sync.Mutex` on acquire/release.
- `HealthServer` — `sync.Mutex` on check list and readiness flag.
- `Registry` — filesystem-level isolation (one file per daemon).
- Global singleton — `atomic.Pointer` for lock-free reads.

164
docs/development.md Normal file
View file

@ -0,0 +1,164 @@
---
title: Development
description: How to build, test, and contribute to go-process.
---
# Development
## Prerequisites
- **Go 1.26+** (uses Go workspaces)
- **Core CLI** (`core` binary) for running tests and quality checks
- Access to `forge.lthn.ai` (private module registry)
Ensure `GOPRIVATE` includes `forge.lthn.ai/*`:
```bash
go env -w GOPRIVATE=forge.lthn.ai/*
```
## Go Workspace
This module is part of the workspace defined at `~/Code/go.work`. After
cloning, run:
```bash
go work sync
```
## Running Tests
```bash
# All tests
core go test
# Single test
core go test --run TestService_Start
# With verbose output
core go test -v
```
Alternatively, using `go test` directly:
```bash
go test ./...
go test -run TestRunner_RunAll ./...
go test -v -count=1 ./exec/...
```
## Quality Assurance
```bash
# Format, vet, lint, test
core go qa
# Full suite (includes race detector, vulnerability scan, security audit)
core go qa full
```
Individual commands:
```bash
core go fmt # Format code
core go vet # Go vet
core go lint # Lint
core go cov # Generate coverage report
core go cov --open # Open coverage in browser
```
## Test Naming Convention
Tests follow the `_Good`, `_Bad`, `_Ugly` suffix pattern used across the Core
ecosystem:
- **`_Good`** — happy path, expected success.
- **`_Bad`** — expected error conditions, graceful handling.
- **`_Ugly`** — panics, edge cases, degenerate inputs.
Where this pattern does not fit naturally, descriptive sub-test names are used
instead (e.g. `TestService_Start/echo_command`, `TestService_Start/context_cancellation`).
## Project Structure
```
go-process/
.core/
build.yaml # Build configuration
release.yaml # Release configuration
exec/
exec.go # Fluent command wrapper
exec_test.go # exec tests
logger.go # Logger interface and NopLogger
actions.go # IPC action message types
buffer.go # RingBuffer implementation
buffer_test.go # RingBuffer tests
daemon.go # Daemon lifecycle manager
daemon_test.go # Daemon tests
go.mod # Module definition
health.go # HTTP health check server
health_test.go # Health server tests
pidfile.go # PID file single-instance lock
pidfile_test.go # PID file tests
process.go # Process type and methods
process_global.go # Global singleton and convenience API
process_test.go # Process tests
global_test.go # Global API tests (concurrency)
registry.go # Daemon registry (JSON file store)
registry_test.go # Registry tests
runner.go # Pipeline runner (sequential, parallel, DAG)
runner_test.go # Runner tests
service.go # Core service (DI integration, lifecycle)
service_test.go # Service tests
types.go # Shared types (Status, Stream, RunOptions, Info)
```
## Adding a New Feature
1. Write the implementation in the appropriate file (or create a new one if
the feature is clearly distinct).
2. Add tests following the naming conventions above.
3. If the feature introduces new IPC events, add the message types to
`actions.go`.
4. Run `core go qa` to verify formatting, linting, and tests pass.
5. Commit using conventional commits: `feat(process): add XYZ support`.
## Coding Standards
- **UK English** in documentation and comments (colour, organisation, centre).
- **`declare(strict_types=1)`-equivalent**: all functions have explicit
parameter and return types.
- **Error handling**: return errors rather than panicking. Use sentinel errors
(`ErrProcessNotFound`, `ErrProcessNotRunning`, `ErrStdinNotAvailable`) for
well-known conditions.
- **Thread safety**: all public types must be safe for concurrent use. Use
`sync.RWMutex` for read-heavy workloads, `sync.Mutex` where writes dominate.
- **Formatting**: `gofmt` / `goimports` via `core go fmt`.
## Error Types
| Error | Meaning |
|-------|---------|
| `ErrProcessNotFound` | No process with the given ID exists in the service |
| `ErrProcessNotRunning` | Operation requires a running process (e.g. SendInput, Signal) |
| `ErrStdinNotAvailable` | Stdin pipe is nil (already closed or never created) |
| `ErrServiceNotInitialized` | Global convenience function called before `process.Init()` |
| `ServiceError` | Wraps service-level errors with a message string |
## Build Configuration
The `.core/build.yaml` defines cross-compilation targets:
| OS | Architecture |
|----|-------------|
| linux | amd64 |
| linux | arm64 |
| darwin | arm64 |
| windows | amd64 |
Since this is a library (no binary), the build configuration is primarily
used for CI validation. The `binary` field is empty.
## Licence
EUPL-1.2. See the repository root for the full licence text.

124
docs/index.md Normal file
View file

@ -0,0 +1,124 @@
---
title: go-process
description: Process management with Core IPC integration for Go applications.
---
# go-process
`forge.lthn.ai/core/go-process` is a process management library that provides
spawning, monitoring, and controlling external processes with real-time output
streaming via the Core ACTION (IPC) system. It integrates directly with the
[Core DI framework](https://forge.lthn.ai/core/go) as a first-class service.
## Features
- Spawn and manage external processes with full lifecycle tracking
- Real-time stdout/stderr streaming via Core IPC actions
- Ring buffer output capture (default 1 MB, configurable)
- Process pipeline runner with dependency graphs, sequential, and parallel modes
- Daemon mode with PID file locking, health check HTTP server, and graceful shutdown
- Daemon registry for tracking running instances across the system
- Lightweight `exec` sub-package for one-shot command execution with logging
- Thread-safe throughout; designed for concurrent use
## Quick Start
### Register with Core
```go
import (
"context"
framework "forge.lthn.ai/core/go/pkg/core"
"forge.lthn.ai/core/go-process"
)
// Create a Core instance with the process service
c, err := framework.New(
framework.WithName("process", process.NewService(process.Options{})),
)
if err != nil {
log.Fatal(err)
}
// Retrieve the typed service
svc, err := framework.ServiceFor[*process.Service](c, "process")
if err != nil {
log.Fatal(err)
}
```
### Run a Command
```go
// Fire-and-forget (async)
proc, err := svc.Start(ctx, "go", "test", "./...")
if err != nil {
return err
}
<-proc.Done()
fmt.Println(proc.Output())
// Synchronous convenience
output, err := svc.Run(ctx, "echo", "hello world")
```
### Listen for Events
Process lifecycle events are broadcast through Core's ACTION system:
```go
c.RegisterAction(func(c *framework.Core, msg framework.Message) error {
switch m := msg.(type) {
case process.ActionProcessStarted:
fmt.Printf("Started: %s (PID %d)\n", m.Command, m.PID)
case process.ActionProcessOutput:
fmt.Print(m.Line)
case process.ActionProcessExited:
fmt.Printf("Exit code: %d (%s)\n", m.ExitCode, m.Duration)
case process.ActionProcessKilled:
fmt.Printf("Killed with %s\n", m.Signal)
}
return nil
})
```
### Global Convenience API
For applications that only need a single process service, a global singleton
is available:
```go
// Initialise once at startup
process.Init(coreInstance)
// Then use package-level functions anywhere
proc, _ := process.Start(ctx, "ls", "-la")
output, _ := process.Run(ctx, "date")
procs := process.List()
running := process.Running()
```
## Package Layout
| Path | Description |
|------|-------------|
| `*.go` (root) | Core process service, types, actions, runner, daemon, health, PID file, registry |
| `exec/` | Lightweight command wrapper with fluent API and structured logging |
## Module Information
| Field | Value |
|-------|-------|
| Module path | `forge.lthn.ai/core/go-process` |
| Go version | 1.26.0 |
| Licence | EUPL-1.2 |
## Dependencies
| Module | Purpose |
|--------|---------|
| `forge.lthn.ai/core/go` | Core DI framework (`ServiceRuntime`, `Core.ACTION`, lifecycle interfaces) |
| `github.com/stretchr/testify` | Test assertions (test-only) |
The package has no other runtime dependencies beyond the Go standard library
and the Core framework.