271 lines
9.8 KiB
Markdown
271 lines
9.8 KiB
Markdown
---
|
|
title: Architecture
|
|
description: Internal design of go-container -- types, data flow, hypervisor abstraction, state management, and template engine.
|
|
---
|
|
|
|
# Architecture
|
|
|
|
go-container is organised into three packages with clear responsibilities. The root `container` package owns the core abstractions. The `devenv` package composes those abstractions into a higher-level development environment. The `sources` package provides pluggable image download backends.
|
|
|
|
```
|
|
container (root)
|
|
|-- Manager interface + LinuxKitManager implementation
|
|
|-- Hypervisor interface (QEMU, Hyperkit)
|
|
|-- State (persistent container registry)
|
|
|-- Template engine (embedded + user templates)
|
|
|
|
|
+-- devenv/
|
|
| |-- DevOps orchestrator
|
|
| |-- ImageManager (download, manifest, update checks)
|
|
| |-- Shell, Serve, Test, Claude sessions
|
|
| +-- Config (from ~/.core/config.yaml)
|
|
|
|
|
+-- sources/
|
|
|-- ImageSource interface
|
|
|-- CDNSource
|
|
+-- GitHubSource
|
|
```
|
|
|
|
|
|
## Key types
|
|
|
|
### Container
|
|
|
|
The central data structure representing a running or stopped VM instance.
|
|
|
|
```go
|
|
type Container struct {
|
|
ID string `json:"id"`
|
|
Name string `json:"name,omitempty"`
|
|
Image string `json:"image"`
|
|
Status Status `json:"status"`
|
|
PID int `json:"pid"`
|
|
StartedAt time.Time `json:"started_at"`
|
|
Ports map[int]int `json:"ports,omitempty"`
|
|
Memory int `json:"memory,omitempty"`
|
|
CPUs int `json:"cpus,omitempty"`
|
|
}
|
|
```
|
|
|
|
Each container gets a unique 8-character hex ID generated from `crypto/rand`. Status transitions follow `running -> stopped | error`.
|
|
|
|
|
|
### Manager interface
|
|
|
|
The `Manager` interface defines the contract for container lifecycle management:
|
|
|
|
```go
|
|
type Manager interface {
|
|
Run(ctx context.Context, image string, opts RunOptions) (*Container, error)
|
|
Stop(ctx context.Context, id string) error
|
|
List(ctx context.Context) ([]*Container, error)
|
|
Logs(ctx context.Context, id string, follow bool) (io.ReadCloser, error)
|
|
Exec(ctx context.Context, id string, cmd []string) error
|
|
}
|
|
```
|
|
|
|
The only implementation is `LinuxKitManager`, which delegates VM execution to a `Hypervisor` and tracks state in a `State` store.
|
|
|
|
|
|
### Hypervisor interface
|
|
|
|
Abstracts the underlying virtualisation technology:
|
|
|
|
```go
|
|
type Hypervisor interface {
|
|
Name() string
|
|
Available() bool
|
|
BuildCommand(ctx context.Context, image string, opts *HypervisorOptions) (*exec.Cmd, error)
|
|
}
|
|
```
|
|
|
|
Two implementations exist:
|
|
|
|
| Implementation | Platform | Acceleration | Binary |
|
|
|----------------|----------|-------------|--------|
|
|
| `QemuHypervisor` | All | KVM (Linux), HVF (macOS) | `qemu-system-x86_64` |
|
|
| `HyperkitHypervisor` | macOS only | Native macOS hypervisor | `hyperkit` |
|
|
|
|
`DetectHypervisor()` auto-selects the best available hypervisor. On macOS it prefers Hyperkit, falling back to QEMU. On Linux it uses QEMU with KVM if `/dev/kvm` is present.
|
|
|
|
|
|
## Data flow: running a container
|
|
|
|
When `LinuxKitManager.Run()` is called, the following sequence occurs:
|
|
|
|
1. **Validate** -- Checks the image file exists via `io.Medium` and detects its format from the file extension (`.iso`, `.qcow2`, `.vmdk`, `.raw`, `.img`).
|
|
|
|
2. **Generate ID** -- Creates an 8-character hex identifier using `crypto/rand`.
|
|
|
|
3. **Apply defaults** -- Memory defaults to 1024 MB, CPUs to 1, SSH port to 2222.
|
|
|
|
4. **Build command** -- Delegates to the `Hypervisor.BuildCommand()` method, which constructs the full command line including:
|
|
- Memory and CPU allocation
|
|
- Disk image attachment (format-specific flags)
|
|
- Network with port forwarding (SSH + user-defined ports)
|
|
- 9p volume shares (QEMU only)
|
|
- Hardware acceleration flags
|
|
|
|
5. **Start process** -- In **detached** mode, stdout/stderr are redirected to a log file under `~/.core/logs/<id>.log`, and a background goroutine monitors the process for exit. In **foreground** mode, output is tee'd to both the log file and the terminal.
|
|
|
|
6. **Persist state** -- The container record is written to `~/.core/containers.json` via the `State` store.
|
|
|
|
7. **Monitor** -- For detached containers, `waitForExit()` runs in a goroutine, updating the container status to `stopped` or `error` when the process terminates.
|
|
|
|
|
|
## State management
|
|
|
|
The `State` struct provides a thread-safe, JSON-persisted container registry:
|
|
|
|
```go
|
|
type State struct {
|
|
Containers map[string]*Container `json:"containers"`
|
|
mu sync.RWMutex
|
|
filePath string
|
|
}
|
|
```
|
|
|
|
Key design decisions:
|
|
|
|
- **Copy-on-read**: `Get()` and `All()` return copies of container structs to prevent data races when callers modify the returned values.
|
|
- **Write-through**: Every mutation (`Add`, `Update`, `Remove`) immediately persists to disk via `SaveState()`.
|
|
- **Auto-create**: `LoadState()` returns an empty state if the file does not exist, and `SaveState()` creates parent directories as needed.
|
|
|
|
Default paths:
|
|
|
|
| Path | Purpose |
|
|
|------|---------|
|
|
| `~/.core/containers.json` | Container state file |
|
|
| `~/.core/logs/<id>.log` | Per-container log files |
|
|
|
|
|
|
## Stopping a container
|
|
|
|
`LinuxKitManager.Stop()` performs a graceful shutdown:
|
|
|
|
1. Sends `SIGTERM` to the hypervisor process.
|
|
2. Waits up to 10 seconds for the process to exit.
|
|
3. If the process does not exit in time, sends `SIGKILL`.
|
|
4. Respects context cancellation throughout -- if the context is cancelled, the process is killed immediately.
|
|
|
|
|
|
## Template engine
|
|
|
|
LinuxKit templates are YAML files that define a complete VM image configuration (kernel, init, services, files). The template engine adds variable substitution on top.
|
|
|
|
### Variable syntax
|
|
|
|
Two forms are supported:
|
|
|
|
- `${VAR}` -- Required variable. Produces an error if not provided.
|
|
- `${VAR:-default}` -- Optional variable with a default value.
|
|
|
|
### Resolution order
|
|
|
|
`GetTemplate(name)` searches for templates in this order:
|
|
|
|
1. **Embedded templates** -- Compiled into the binary via `//go:embed templates/*.yml`. These always take precedence.
|
|
2. **Workspace templates** -- `.core/linuxkit/` relative to the current working directory.
|
|
3. **User templates** -- `~/.core/linuxkit/` in the user's home directory.
|
|
|
|
User-defined templates that share a name with a built-in template are ignored (built-ins win).
|
|
|
|
### Variable extraction
|
|
|
|
`ExtractVariables(content)` parses a template and returns two collections:
|
|
|
|
- A sorted slice of required variable names (those using `${VAR}` syntax with no default).
|
|
- A map of optional variable names to their default values (those using `${VAR:-default}` syntax).
|
|
|
|
This powers the `core vm templates vars <name>` command.
|
|
|
|
|
|
## Development environment (devenv)
|
|
|
|
The `DevOps` struct in the `devenv` package composes the lower-level primitives into a complete development workflow.
|
|
|
|
### Boot sequence
|
|
|
|
1. Checks the dev image is installed (platform-specific qcow2 file: `core-devops-{os}-{arch}.qcow2`).
|
|
2. Launches the image via `LinuxKitManager.Run()` in detached mode with 4096 MB RAM, 2 CPUs, and SSH on port 2222.
|
|
3. Polls for up to 60 seconds until the VM's SSH host key can be scanned, then writes it to `~/.core/known_hosts`.
|
|
|
|
### Shell access
|
|
|
|
Two modes are available:
|
|
|
|
- **SSH** (default) -- Connects via `ssh -A -p 2222 root@localhost` with agent forwarding and strict host key checking against `~/.core/known_hosts`.
|
|
- **Serial console** -- Attaches to the QEMU serial console socket via `socat`.
|
|
|
|
### Project mounting
|
|
|
|
Projects are mounted into the VM at `/app` using a reverse SSHFS tunnel. The VM opens an SSH reverse tunnel back to the host (port 10000) and mounts the host directory via SSHFS.
|
|
|
|
### Auto-detection
|
|
|
|
Several operations auto-detect the project type by inspecting files on disk:
|
|
|
|
| File detected | Serve command | Test command |
|
|
|---------------|---------------|--------------|
|
|
| `artisan` | `php artisan octane:start` | -- |
|
|
| `package.json` with `dev` script | `npm run dev -- --host 0.0.0.0` | `npm test` |
|
|
| `composer.json` with `test` script | `frankenphp php-server` | `composer test` |
|
|
| `go.mod` | `go run .` | `go test ./...` |
|
|
| `manage.py` | `python manage.py runserver` | -- |
|
|
| `pytest.ini` or `pyproject.toml` | -- | `pytest` |
|
|
|
|
Auto-detection can be overridden with `.core/test.yaml` for tests or explicit `--command` flags.
|
|
|
|
### Claude integration
|
|
|
|
`DevOps.Claude()` starts a sandboxed Claude session inside the VM:
|
|
|
|
1. Auto-boots the dev environment if not running.
|
|
2. Mounts the project directory at `/app`.
|
|
3. Forwards authentication credentials (Anthropic API key, GitHub CLI config, SSH agent, git identity) based on configurable options.
|
|
4. Launches `claude` inside the VM via SSH with agent forwarding.
|
|
|
|
|
|
## Image sources (sources)
|
|
|
|
The `ImageSource` interface defines how dev environment images are downloaded:
|
|
|
|
```go
|
|
type ImageSource interface {
|
|
Name() string
|
|
Available() bool
|
|
LatestVersion(ctx context.Context) (string, error)
|
|
Download(ctx context.Context, m io.Medium, dest string, progress func(downloaded, total int64)) error
|
|
}
|
|
```
|
|
|
|
| Source | Backend | Availability check |
|
|
|--------|---------|--------------------|
|
|
| `CDNSource` | HTTP GET from a configured CDN URL | CDN URL is configured |
|
|
| `GitHubSource` | `gh release download` via the GitHub CLI | `gh` is installed and authenticated |
|
|
|
|
The `ImageManager` in `devenv` maintains a `manifest.json` in `~/.core/images/` that tracks installed image versions, SHA256 checksums, download timestamps, and source names. When the source is set to `"auto"` (the default), it tries GitHub first, then CDN.
|
|
|
|
|
|
## Configuration
|
|
|
|
The `devenv` package reads `~/.core/config.yaml` via the `config` library:
|
|
|
|
```yaml
|
|
version: 1
|
|
images:
|
|
source: auto # auto | github | cdn
|
|
github:
|
|
repo: host-uk/core-images
|
|
registry:
|
|
image: ghcr.io/host-uk/core-devops
|
|
cdn:
|
|
url: https://cdn.example.com/images
|
|
```
|
|
|
|
If the file does not exist, sensible defaults are used.
|
|
|
|
|
|
## Licence
|
|
|
|
EUPL-1.2
|