docs: add human-friendly documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Snider 2026-03-11 13:02:40 +00:00
parent badea4b31a
commit 3a9b766eaf
3 changed files with 688 additions and 0 deletions

258
docs/architecture.md Normal file
View file

@ -0,0 +1,258 @@
---
title: Architecture
description: Internal design of go-build -- types, data flow, and extension points.
---
# Architecture
go-build is organised into three independent subsystems that share common types: **build**, **release**, and **sdk**. The CLI layer in `cmd/` wires them together but the library packages under `pkg/` can be used programmatically without the CLI.
## Build Subsystem
### Project Discovery
`build.Discover()` scans a directory for marker files and returns detected project types in priority order (most specific first). For example, a Wails project contains both `wails.json` and `go.mod`, so `Discover` returns `[wails, go]`. `PrimaryType()` returns only the first match.
Detection order:
1. `wails.json` -- `ProjectTypeWails`
2. `go.mod` -- `ProjectTypeGo`
3. `package.json` -- `ProjectTypeNode`
4. `composer.json` -- `ProjectTypePHP`
Docker (`Dockerfile`), LinuxKit (`linuxkit.yml` or `.core/linuxkit/*.yml`), C++ (`CMakeLists.txt`), and Taskfile (`Taskfile.yml`) are detected by their respective builders' `Detect()` methods rather than the central discovery function.
### Builder Interface
Every builder implements:
```go
type Builder interface {
Name() string
Detect(fs io.Medium, dir string) (bool, error)
Build(ctx context.Context, cfg *Config, targets []Target) ([]Artifact, error)
}
```
The `Config` struct carries runtime parameters (filesystem medium, project directory, output directory, binary name, version, linker flags) plus Docker- and LinuxKit-specific fields.
`Build()` returns a slice of `Artifact` values, each recording the output path, target OS, and target architecture:
```go
type Artifact struct {
Path string
OS string
Arch string
Checksum string
}
```
### Builder Implementations
| Builder | Detection | Strategy |
|---|---|---|
| **GoBuilder** | `go.mod` or `wails.json` | Sets `GOOS`/`GOARCH`/`CGO_ENABLED=0`, runs `go build -trimpath` with ldflags. Output per target: `dist/{os}_{arch}/{binary}`. |
| **WailsBuilder** | `wails.json` | Checks `go.mod` for Wails v3 vs v2. V3 delegates to TaskfileBuilder; V2 runs `wails build -platform` then copies from `build/bin/` to `dist/`. |
| **DockerBuilder** | `Dockerfile` | Validates `docker` and `buildx`, builds multi-platform images with `docker buildx build --platform`. Supports `--push` or local load/OCI tarball. |
| **LinuxKitBuilder** | `linuxkit.yml` or `.core/linuxkit/*.yml` | Validates `linuxkit` CLI, runs `linuxkit build --format --name --dir --arch`. Outputs qcow2, iso, raw, vmdk, vhd, or cloud images. Linux-only targets. |
| **CPPBuilder** | `CMakeLists.txt` | Validates `make`, runs `make configure` then `make build` then `make package` for host builds. Cross-compilation uses Conan profile targets (e.g. `make gcc-linux-armv8`). Finds artifacts in `build/packages/` or `build/release/src/`. |
| **TaskfileBuilder** | `Taskfile.yml` / `Taskfile.yaml` / `Taskfile` | Validates `task` CLI, runs `task build` with `GOOS`, `GOARCH`, `OUTPUT_DIR`, `NAME`, `VERSION` as both env vars and task vars. Discovers artifacts by platform subdirectory or filename pattern. |
### Post-Build Pipeline
After building, the CLI orchestrates three optional steps:
1. **Signing** -- `signing.SignBinaries()` codesigns darwin artifacts with hardened runtime. `signing.NotarizeBinaries()` submits to Apple via `xcrun notarytool` and staples. `signing.SignChecksums()` creates GPG detached signatures (`.asc`).
2. **Archiving** -- `build.ArchiveAll()` (or `ArchiveAllXZ()`) wraps each artifact. Linux/macOS get `tar.gz` (or `tar.xz`); Windows gets `zip`. XZ compression uses the Borg library. Archive filenames follow the pattern `{binary}_{os}_{arch}.tar.gz`.
3. **Checksums** -- `build.ChecksumAll()` computes SHA-256 for each archive. `build.WriteChecksumFile()` writes a sorted `CHECKSUMS.txt` in the standard `sha256 filename` format.
### Signing Architecture
The `Signer` interface:
```go
type Signer interface {
Name() string
Available() bool
Sign(ctx context.Context, fs io.Medium, path string) error
}
```
Three implementations:
- **GPGSigner** -- `gpg --detach-sign --armor --local-user {key}`. Produces `.asc` files.
- **MacOSSigner** -- `codesign --sign {identity} --timestamp --options runtime --force`. Notarisation via `xcrun notarytool submit --wait` then `xcrun stapler staple`.
- **WindowsSigner** -- Placeholder (returns `Available() == false`).
Configuration supports `$ENV` expansion in all credential fields, so secrets can come from environment variables without being written to YAML.
### Configuration Loading
`build.LoadConfig(fs, dir)` reads `.core/build.yaml`. If the file is missing, `DefaultConfig()` provides:
- Version 1 format
- Main package: `.`
- Flags: `["-trimpath"]`
- LDFlags: `["-s", "-w"]`
- CGO: disabled
- Targets: `linux/amd64`, `linux/arm64`, `darwin/arm64`, `windows/amd64`
- Signing: enabled, credentials from environment
Fields present in the YAML override defaults; omitted fields inherit defaults via `applyDefaults()`.
### Filesystem Abstraction
All file operations go through `io.Medium` from `forge.lthn.ai/core/go-io`. Production code uses `io.Local` (real filesystem); tests can inject mock mediums. This makes builders unit-testable without touching the real filesystem for detection and configuration loading.
---
## Release Subsystem
### Version Resolution
`release.DetermineVersion(dir)` resolves the release version:
1. If HEAD has an exact git tag, use it.
2. If there is a previous tag, increment its patch number (e.g. `v1.2.3` becomes `v1.2.4`).
3. If no tags exist, default to `v0.0.1`.
Helper functions `IncrementMinor()` and `IncrementMajor()` are available for manual version bumps. `ParseVersion()` decomposes a semver string into major, minor, patch, pre-release, and build components. `CompareVersions()` returns -1, 0, or 1.
All versions are normalised to include a `v` prefix.
### Changelog Generation
`release.Generate(dir, fromRef, toRef)` parses git history between two refs and produces grouped Markdown.
Commits are parsed against the conventional-commit regex:
```
^(\w+)(?:\(([^)]+)\))?(!)?:\s*(.+)$
```
This matches patterns like `feat: add feature`, `fix(scope): fix bug`, or `feat!: breaking change`.
Parsed commits are grouped by type and rendered in a fixed order: breaking changes first, then features, bug fixes, performance, refactoring, and so on. Each entry includes the optional scope (bolded) and the short commit hash.
`GenerateWithConfig()` adds include/exclude filtering by commit type, driven by the `changelog` section in `.core/release.yaml`.
### Release Orchestration
Two entry points:
- **`release.Run()`** -- Full pipeline: determine version, generate changelog, build artifacts (via the build subsystem), archive, checksum, then publish to all configured targets.
- **`release.Publish()`** -- Publish-only: expects pre-built artifacts in `dist/`, generates changelog, then publishes. This supports the separated `core build` then `core ci` workflow.
Both accept a `dryRun` parameter. When true, publishers print what would happen without executing.
### Publisher Interface
```go
type Publisher interface {
Name() string
Publish(ctx context.Context, release *Release, pubCfg PublisherConfig, relCfg ReleaseConfig, dryRun bool) error
}
```
Eight publishers are implemented:
| Publisher | Mechanism |
|---|---|
| **GitHub** | `gh release create` via the GitHub CLI. Auto-detects repository from git remote. Uploads all artifacts as release assets. |
| **Docker** | `docker buildx build` with multi-platform support. Pushes to configured registry with version tags. |
| **npm** | `npm publish` with configurable access level and package name. |
| **Homebrew** | Generates a Ruby formula file. Optionally targets an official tap repository. |
| **Scoop** | Generates a JSON manifest for a Scoop bucket. |
| **AUR** | Generates a PKGBUILD file for the Arch User Repository. |
| **Chocolatey** | Generates a `.nuspec` and `chocolateyinstall.ps1`. Optionally pushes via `choco push`. |
| **LinuxKit** | Builds LinuxKit VM images in specified formats and uploads them as release assets. |
Publisher-specific configuration (registry, tap, bucket, image, etc.) is carried in `PublisherConfig` fields and mapped to an `Extended` map at runtime.
### SDK Release Integration
`release.RunSDK()` handles SDK-specific releases: it runs a breaking-change diff (if enabled), then generates SDKs via the SDK subsystem. This can be wired into a CI pipeline to auto-generate client libraries on each release.
---
## SDK Subsystem
### Spec Detection
`sdk.DetectSpec()` locates the OpenAPI specification:
1. If a path is configured in `.core/release.yaml` under `sdk.spec`, use it.
2. Check common paths: `api/openapi.yaml`, `api/openapi.json`, `openapi.yaml`, `openapi.json`, `docs/api.yaml`, `docs/api.json`, `swagger.yaml`, `swagger.json`.
3. Check for Laravel Scramble in `composer.json` (export not yet implemented).
### Breaking-Change Detection
`sdk.Diff(basePath, revisionPath)` loads two OpenAPI specs via `kin-openapi`, computes a structural diff via `oasdiff`, and runs the `oasdiff/checker` backward-compatibility checks at error level. Returns a `DiffResult` with a boolean `Breaking` flag, a list of change descriptions, and a human-readable summary.
`DiffExitCode()` maps results to CI exit codes: 0 (clean), 1 (breaking changes), 2 (error).
### Code Generation
The `generators.Generator` interface:
```go
type Generator interface {
Language() string
Generate(ctx context.Context, opts Options) error
Available() bool
Install() string
}
```
Generators are held in a `Registry` and looked up by language identifier. Each generator tries three strategies in order:
1. **Native tool** -- e.g. `oapi-codegen` for Go, `openapi-typescript-codegen` for TypeScript.
2. **npx** -- Falls back to `npx` invocation where applicable (TypeScript).
3. **Docker** -- Uses the `openapitools/openapi-generator-cli` image as a last resort.
| Language | Native Tool | Docker Generator |
|---|---|---|
| TypeScript | `openapi-typescript-codegen` or `npx` | `typescript-fetch` |
| Python | `openapi-python-client` | `python` |
| Go | `oapi-codegen` | `go` |
| PHP | `openapi-generator-cli` via Docker | `php` |
On Unix systems, Docker containers run with `--user {uid}:{gid}` to match host file ownership.
---
## Data Flow Summary
```
.core/build.yaml -----> LoadConfig() -----> BuildConfig
|
project directory ----> Discover() -----------> ProjectType
|
getBuilder()
|
Builder.Build()
|
[]Artifact (raw binaries)
|
+----------------------+---------------------+
| | |
SignBinaries() ArchiveAll() (optional)
| | NotarizeBinaries()
| []Artifact (archives)
| |
| ChecksumAll()
| |
| []Artifact (with checksums)
| |
| WriteChecksumFile()
| |
+----------+-----------+
|
SignChecksums() (GPG)
|
Publisher.Publish()
|
GitHub / Docker / npm / Homebrew / ...
```

222
docs/development.md Normal file
View file

@ -0,0 +1,222 @@
---
title: Development
description: Building, testing, and contributing to go-build.
---
# Development
## Prerequisites
- **Go 1.26+** (the module declares `go 1.26.0`)
- **Go workspace** -- this module is part of the workspace at `~/Code/go.work`. After cloning, run `go work sync` to ensure local replacements resolve correctly.
- `GOPRIVATE=forge.lthn.ai/*` must be set for private module fetching.
## Building
```bash
cd /Users/snider/Code/core/go-build
go build ./...
```
There is no standalone binary produced by this repository. The `cmd/` packages register CLI commands that are compiled into the `core` binary from `forge.lthn.ai/core/cli`.
To build the full CLI with these commands included:
```bash
cd /Users/snider/Code/core/cli
core build # or: go build -o bin/core ./cmd/core
```
## Running Tests
```bash
go test ./...
```
To run a single test by name:
```bash
go test ./pkg/build/... -run TestLoadConfig_Good
go test ./pkg/release/... -run TestIncrementVersion
go test ./pkg/sdk/... -run TestDiff
```
To run tests with race detection:
```bash
go test -race ./...
```
### Test Naming Convention
Tests follow the `_Good`, `_Bad`, `_Ugly` suffix pattern used across the Core ecosystem:
- `_Good` -- Happy-path tests. Valid inputs produce expected outputs.
- `_Bad` -- Expected error conditions. Invalid inputs are handled gracefully.
- `_Ugly` -- Edge cases, panics, and boundary conditions.
Example:
```go
func TestLoadConfig_Good(t *testing.T) {
// Valid .core/build.yaml is loaded correctly
}
func TestLoadConfig_Bad(t *testing.T) {
// Malformed YAML returns a parse error
}
func TestChecksum_Ugly(t *testing.T) {
// Empty artifact path returns an error
}
```
### Test Helpers
Tests use `t.TempDir()` for filesystem isolation and `io.Local` as the medium:
```go
func setupConfigTestDir(t *testing.T, configContent string) string {
t.Helper()
dir := t.TempDir()
if configContent != "" {
coreDir := filepath.Join(dir, ConfigDir)
err := os.MkdirAll(coreDir, 0755)
require.NoError(t, err)
err = os.WriteFile(
filepath.Join(coreDir, ConfigFileName),
[]byte(configContent), 0644,
)
require.NoError(t, err)
}
return dir
}
```
### Testing Libraries
- **testify** (`assert` and `require`) for assertions.
- `io.Local` from `forge.lthn.ai/core/go-io` as the filesystem medium.
## Code Style
- **UK English** in comments and user-facing strings (colour, organisation, centre, notarisation).
- **Strict types** -- all parameters and return types are explicitly typed.
- **Error format** -- use `fmt.Errorf("package.Function: descriptive message: %w", err)` for wrapped errors.
- **PSR-style** formatting via `gofmt` / `goimports`.
## Adding a New Builder
1. Create `pkg/build/builders/mybuilder.go` implementing `build.Builder`:
```go
type MyBuilder struct{}
func NewMyBuilder() *MyBuilder { return &MyBuilder{} }
func (b *MyBuilder) Name() string { return "mybuilder" }
func (b *MyBuilder) Detect(fs io.Medium, dir string) (bool, error) {
return fs.IsFile(filepath.Join(dir, "mymarker.toml")), nil
}
func (b *MyBuilder) Build(ctx context.Context, cfg *build.Config, targets []build.Target) ([]build.Artifact, error) {
// Build logic here
return artifacts, nil
}
var _ build.Builder = (*MyBuilder)(nil) // Compile-time check
```
2. Add the builder to the `getBuilder()` switch in both `cmd/build/cmd_project.go` and `pkg/release/release.go`.
3. Optionally add a `ProjectType` constant and marker to `pkg/build/build.go` and `pkg/build/discovery.go` if the new type should participate in auto-discovery.
4. Write tests in `pkg/build/builders/mybuilder_test.go` following the `_Good`/`_Bad`/`_Ugly` pattern.
## Adding a New Publisher
1. Create `pkg/release/publishers/mypub.go` implementing `publishers.Publisher`:
```go
type MyPublisher struct{}
func NewMyPublisher() *MyPublisher { return &MyPublisher{} }
func (p *MyPublisher) Name() string { return "mypub" }
func (p *MyPublisher) Publish(ctx context.Context, release *Release, pubCfg PublisherConfig, relCfg ReleaseConfig, dryRun bool) error {
if dryRun {
// Print what would happen
return nil
}
// Publish logic here
return nil
}
```
2. Add the publisher to the `getPublisher()` switch in `pkg/release/release.go`.
3. Add any publisher-specific fields to `PublisherConfig` in `pkg/release/config.go` and map them in `buildExtendedConfig()` in `pkg/release/release.go`.
4. Write tests in `pkg/release/publishers/mypub_test.go`.
## Adding a New SDK Generator
1. Create `pkg/sdk/generators/mylang.go` implementing `generators.Generator`:
```go
type MyLangGenerator struct{}
func NewMyLangGenerator() *MyLangGenerator { return &MyLangGenerator{} }
func (g *MyLangGenerator) Language() string { return "mylang" }
func (g *MyLangGenerator) Available() bool {
_, err := exec.LookPath("mylang-codegen")
return err == nil
}
func (g *MyLangGenerator) Install() string {
return "pip install mylang-codegen"
}
func (g *MyLangGenerator) Generate(ctx context.Context, opts Options) error {
// Try native, then Docker fallback
return nil
}
```
2. Register it in `pkg/sdk/sdk.go` inside `GenerateLanguage()`:
```go
registry.Register(generators.NewMyLangGenerator())
```
3. Write tests in `pkg/sdk/generators/mylang_test.go`.
## Directory Conventions
- **`pkg/`** -- Library code. Importable by other modules.
- **`cmd/`** -- CLI command registration. Each subdirectory registers commands via `cli.RegisterCommands()` in an `init()` function. These packages are imported by the CLI binary.
- **`.core/`** -- Per-project configuration directory (not part of this repository; created in consumer projects).
## Commit Guidelines
Follow conventional commits:
```
type(scope): description
```
Types: `feat`, `fix`, `perf`, `refactor`, `docs`, `style`, `test`, `build`, `ci`, `chore`.
Include the co-author trailer:
```
Co-Authored-By: Virgil <virgil@lethean.io>
```
## Licence
EUPL-1.2. See `LICENSE` in the repository root.

208
docs/index.md Normal file
View file

@ -0,0 +1,208 @@
---
title: go-build
description: Build system, release pipeline, and SDK generation for the Core ecosystem.
---
# go-build
`forge.lthn.ai/core/go-build` is the build, release, and SDK generation toolkit for Core projects. It provides:
- **Auto-detecting builders** for Go, Wails, Docker, LinuxKit, C++, and Taskfile projects
- **Cross-compilation** with per-target archiving (tar.gz, tar.xz, zip) and SHA-256 checksums
- **Code signing** -- macOS codesign with notarisation, GPG detached signatures, Windows signtool (placeholder)
- **Release automation** -- semantic versioning from git tags, conventional-commit changelogs, multi-target publishing
- **SDK generation** -- OpenAPI spec diffing for breaking-change detection, code generation for TypeScript, Python, Go, and PHP
- **CLI integration** -- registers `core build`, `core ci`, and `core sdk` commands via the Core CLI framework
## Module Path
```
forge.lthn.ai/core/go-build
```
Requires **Go 1.26+**.
## Quick Start
### Build a project
From any project directory containing a recognised marker file:
```bash
core build # Auto-detect type, build for configured targets
core build --targets linux/amd64 # Single target
core build --ci # JSON output for CI pipelines
core build --verbose # Detailed step-by-step output
```
The builder is chosen by marker-file priority:
| Marker file | Builder |
|-------------------|------------|
| `wails.json` | Wails |
| `go.mod` | Go |
| `package.json` | Node (stub)|
| `composer.json` | PHP (stub) |
| `CMakeLists.txt` | C++ |
| `Dockerfile` | Docker |
| `linuxkit.yml` | LinuxKit |
| `Taskfile.yml` | Taskfile |
### Release artifacts
```bash
core build release --we-are-go-for-launch # Build + archive + checksum + publish
core build release # Dry-run (default without the flag)
core build release --draft --prerelease # Mark as draft pre-release
```
### Publish pre-built artifacts
After `core build` has populated `dist/`:
```bash
core ci # Dry-run publish from dist/
core ci --we-are-go-for-launch # Actually publish
core ci --version v1.2.3 # Override version
```
### Generate changelogs
```bash
core ci changelog # From latest tag to HEAD
core ci changelog --from v0.1.0 --to v0.2.0
core ci version # Show determined next version
core ci init # Scaffold .core/release.yaml
```
### SDK operations
```bash
core build sdk # Generate SDKs for all configured languages
core build sdk --lang typescript # Single language
core sdk diff --base v1.0.0 --spec api/openapi.yaml # Breaking-change check
core sdk validate # Validate OpenAPI spec
```
## Package Layout
```
forge.lthn.ai/core/go-build/
|
|-- cmd/
| |-- build/ CLI commands for `core build` (build, from-path, pwa, sdk, release)
| |-- ci/ CLI commands for `core ci` (init, changelog, version, publish)
| +-- sdk/ CLI commands for `core sdk` (diff, validate)
|
+-- pkg/
|-- build/ Core build types, config loading, discovery, archiving, checksums
| |-- builders/ Builder implementations (Go, Wails, Docker, LinuxKit, C++, Taskfile)
| +-- signing/ Code-signing implementations (macOS codesign, GPG, Windows stub)
|
|-- release/ Release orchestration, versioning, changelog, config
| +-- publishers/ Publisher implementations (GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit)
|
+-- sdk/ OpenAPI SDK generation and breaking-change diffing
+-- generators/ Language generators (TypeScript, Python, Go, PHP)
```
## Configuration Files
Build and release behaviour is driven by two YAML files in the `.core/` directory.
### `.core/build.yaml`
Controls compilation targets, flags, and signing:
```yaml
version: 1
project:
name: myapp
description: My application
main: ./cmd/myapp
binary: myapp
build:
cgo: false
flags: ["-trimpath"]
ldflags: ["-s", "-w"]
env: []
targets:
- os: linux
arch: amd64
- os: linux
arch: arm64
- os: darwin
arch: arm64
- os: windows
arch: amd64
sign:
enabled: true
gpg:
key: $GPG_KEY_ID
macos:
identity: $CODESIGN_IDENTITY
notarize: false
apple_id: $APPLE_ID
team_id: $APPLE_TEAM_ID
app_password: $APPLE_APP_PASSWORD
```
When no `.core/build.yaml` exists, sensible defaults apply (CGO off, `-trimpath -s -w`, four standard targets).
### `.core/release.yaml`
Controls versioning, changelog filtering, publishers, and SDK generation:
```yaml
version: 1
project:
name: myapp
repository: owner/repo
build:
targets:
- os: linux
arch: amd64
- os: darwin
arch: arm64
publishers:
- type: github
draft: false
prerelease: false
- type: homebrew
tap: owner/homebrew-tap
- type: docker
registry: ghcr.io
image: owner/myapp
tags: ["latest", "{{.Version}}"]
changelog:
include: [feat, fix, perf, refactor]
exclude: [chore, docs, style, test, ci]
sdk:
spec: api/openapi.yaml
languages: [typescript, python, go, php]
output: sdk
diff:
enabled: true
fail_on_breaking: false
```
## Dependencies
| Dependency | Purpose |
|---|---|
| `forge.lthn.ai/core/cli` | CLI command registration and TUI styling |
| `forge.lthn.ai/core/go-io` | Filesystem abstraction (`io.Medium`, `io.Local`) |
| `forge.lthn.ai/core/go-i18n` | Internationalised CLI labels |
| `forge.lthn.ai/core/go-log` | Structured error logging |
| `github.com/Snider/Borg` | XZ compression for tar.xz archives |
| `github.com/getkin/kin-openapi` | OpenAPI spec loading and validation |
| `github.com/oasdiff/oasdiff` | OpenAPI diff and breaking-change detection |
| `gopkg.in/yaml.v3` | YAML config parsing |
| `github.com/leaanthony/debme` | Embedded filesystem anchoring (PWA templates) |
| `github.com/leaanthony/gosod` | Template extraction for PWA builds |
| `golang.org/x/net` | HTML parsing for PWA manifest detection |
| `golang.org/x/text` | Changelog section title casing |
## Licence
EUPL-1.2