From cbdb8028dc41bc22d244285ab79cb643fd009ad4 Mon Sep 17 00:00:00 2001 From: Snider Date: Tue, 14 Apr 2026 21:24:17 +0100 Subject: [PATCH] feat(build): action-style upload artifact naming + doc parity MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - release.yml: generated workflow computes upload artifact names in action shape {build-name}_{os}_{arch}_{tag|shortsha} with repo-name fallback - workflow_test + cmd_workflow_test: artifact-name derivation coverage - README + docs/index + docs/architecture + CLAUDE.md: refreshed to describe current module path, builders/publishers, generated workflow/action surface, Apple pipeline — replaces pre-spec documentation Verified: go test ./... passes Co-Authored-By: Virgil --- CLAUDE.md | 101 ++++------- README.md | 39 ++++- cmd/build/cmd_workflow_test.go | 7 + docs/architecture.md | 299 +++++++++----------------------- docs/index.md | 279 +++++++++-------------------- pkg/build/templates/release.yml | 34 +++- pkg/build/workflow_test.go | 7 + 7 files changed, 275 insertions(+), 491 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 0832bf4..4ad506e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,84 +1,55 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. +Agent guidance for `core/go-build`. ## Project Overview -`core/go-build` is the build system, release pipeline, and SDK generation tool for the Core ecosystem. Three subsystems under `pkg/` — **build**, **release**, **sdk** — can be used as libraries or wired together via CLI commands in `cmd/`. This repo produces no standalone binary; `cmd/` packages register commands via `cli.RegisterCommands()` in `init()` functions, compiled into the `core` binary from `forge.lthn.ai/core/cli`. Module path: `dappco.re/go/core/build`. +`dappco.re/go/core/build` is a command-registration and library module for: -## Build & Test +- `core build` +- `core build apple` +- `core build workflow` +- `core build sdk` +- `core ci` +- `core sdk` + +It also carries the reusable release workflow template that mirrors the public `dAppCore/build@v3` action surface. + +## Build and Test ```bash -go build ./... # compile all packages -go test ./... # run all tests -go test ./pkg/build/... -run TestLoadConfig_Good # single test by name -go test -race ./... # with race detection +go build ./... +go test ./... +go test ./pkg/build/... -run TestWorkflow_WriteReleaseWorkflow_Good +go test ./pkg/build/... -run TestApple_ ``` -**Go workspace**: this module is part of `~/Code/go.work`. Run `go work sync` after cloning. Set `GOPRIVATE=dappco.re/*,forge.lthn.ai/*` for private module fetching. +## Main Packages -## Architecture +- `pkg/build/`: discovery, config loading, caches, checksums, archives, workflow generation, Apple implementation +- `pkg/build/builders/`: Go, Wails, Node, PHP, Python, Rust, Docs, Docker, LinuxKit, C++, Taskfile +- `pkg/build/apple/`: RFC-facing Apple wrapper that exposes `core.Result` +- `pkg/build/signing/`: GPG, macOS codesign/notarisation, Windows signtool +- `pkg/release/`: versioning, changelogs, orchestration +- `pkg/release/publishers/`: GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit +- `pkg/sdk/`: OpenAPI detection, diffing, generation -The three subsystems share common types but are independent: +## Important Behaviour -- **`pkg/build/`** — Config loading (`.core/build.yaml`), project discovery via marker files, `Builder` interface, archiving (tar.gz/zip), checksums (SHA-256) -- **`pkg/build/builders/`** — Go, Wails, Docker, LinuxKit, C++, Taskfile builder implementations -- **`pkg/build/signing/`** — `Signer` interface with GPG, macOS codesign/notarisation, Windows (placeholder). Credentials support `$ENV` expansion -- **`pkg/release/`** — Version resolution from git tags, conventional-commit changelog generation, release orchestration. Two entry points: `Run()` (full pipeline) and `Publish()` (pre-built artifacts from `dist/`) -- **`pkg/release/publishers/`** — `Publisher` interface: GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit -- **`pkg/sdk/`** — OpenAPI spec detection, breaking-change diff via oasdiff, SDK code generation -- **`pkg/sdk/generators/`** — `Generator` interface with registry. TypeScript, Python, Go, PHP generators (native tool -> npx -> Docker fallback) -- **`cmd/build/`** — `core build` commands (build, from-path, pwa, sdk, release) -- **`cmd/ci/`** — `core ci` commands (publish, init, changelog, version) -- **`cmd/sdk/`** — `core sdk` commands (diff, validate) - -### Key Data Flow - -``` -.core/build.yaml -> LoadConfig() -> BuildConfig -project dir -> Discover() -> ProjectType -> getBuilder() -> Builder.Build() -> []Artifact - -> SignBinaries() -> ArchiveAll() -> ChecksumAll() -> Publisher.Publish() -``` - -### Key Interfaces - -- `build.Builder` — `Name()`, `Detect(fs, dir)`, `Build(ctx, cfg, targets)` -- `publishers.Publisher` — `Name()`, `Publish(ctx, release, pubCfg, relCfg, dryRun)` -- `signing.Signer` — `Name()`, `Available()`, `Sign(ctx, fs, path)` -- `generators.Generator` — `Language()`, `Generate(ctx, opts)`, `Available()`, `Install()` - -### Filesystem Abstraction - -All file operations use `io.Medium` from `dappco.re/go/core/io`. Production uses `io.Local`; tests inject mocks for isolation. - -### Configuration Files - -- `.core/build.yaml` — Build config (targets, flags, signing) -- `.core/release.yaml` — Release config (publishers, changelog, SDK settings) +- Discovery is richer than simple marker lookup: it handles subtree frontends, MkDocs roots, distro-aware Linux package hints, and action-facing stack suggestions +- The generated release workflow must stay aligned with the action-style inputs: `build-name`, `build-platform`, `build-tags`, `build-obfuscate`, `nsis`, `deno-build`, `wails-build-webview2`, and `build-cache` +- Workflow artifact naming is expected to follow `{build-name}_{os}_{arch}_{tag|shortsha}` +- Apple support includes universal builds, notarisation, DMG creation, Xcode Cloud script generation, TestFlight, and App Store submission ## Coding Standards -- **UK English** in comments and strings (colour, organisation, notarisation) -- **Strict types** — all parameters and return types explicitly typed -- **Error wrapping** — `coreerr.E("package.Function", "message", err)` via `coreerr "dappco.re/go/core/log"` -- **testify** (`assert`/`require`) for assertions -- **Test naming** — `_Good` (happy path), `_Bad` (expected errors), `_Ugly` (edge cases) -- **Conventional commits** — `type(scope): description` -- **Licence** — EUPL-1.2 +- Use `coreerr.E("package.Function", "message", err)` for wrapped errors +- Prefer UK English in user-facing strings and comments +- Keep tests in `testify` style with `_Good`, `_Bad`, and `_Ugly` naming +- Preserve env expansion support in config models and signing/apple credentials ## Extension Points -**New builder**: implement `build.Builder` in `pkg/build/builders/`, add to `getBuilder()` in `cmd/build/cmd_project.go` and `pkg/release/release.go`, optionally add `ProjectType` to `pkg/build/build.go` and marker to `pkg/build/discovery.go`. - -**New publisher**: implement `publishers.Publisher` in `pkg/release/publishers/`, add to `getPublisher()` in `pkg/release/release.go`, add config fields to `PublisherConfig` in `pkg/release/config.go` and `buildExtendedConfig()`. - -**New SDK generator**: implement `generators.Generator` in `pkg/sdk/generators/`, register in `pkg/sdk/sdk.go` `GenerateLanguage()`. - -## Dependencies - -- `forge.lthn.ai/core/cli` — Command registration (`cli.RegisterCommands`, `cli.Command`) *(not yet migrated)* -- `dappco.re/go/core/io` — Filesystem abstraction (`io.Medium`, `io.Local`) -- `dappco.re/go/core/i18n` — Internationalisation (`i18n.T()`, `i18n.Label()`) -- `dappco.re/go/core/log` — Structured logging -- `github.com/Snider/Borg` — XZ compression for archives -- `github.com/getkin/kin-openapi` + `github.com/oasdiff/oasdiff` — OpenAPI parsing and diff +- New builder: add the implementation in `pkg/build/builders/`, register the project type in discovery/resolution, and add coverage in command and release paths +- New workflow input: update the template, workflow tests, and any CLI alias plumbing together +- New Apple capability: update both `pkg/build/apple.go` and the RFC-facing wrapper in `pkg/build/apple/` diff --git a/README.md b/README.md index 6e70136..5c190bb 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,40 @@ # go-build -Build system, release publishers, and SDK generation \ No newline at end of file +`dappco.re/go/core/build` is the build, release, SDK, Apple packaging, and workflow toolkit behind `core build`, `core release`, `core sdk`, `core ci`, and the public `dAppCore/build@v3` GitHub Action surface. + +## What It Covers + +- Project discovery across Go, Wails, Node, Deno, PHP, Python, Rust, Docs, Docker, LinuxKit, C++, and Taskfile projects +- Cross-platform artifact builds with archive and checksum generation +- Eight release publishers: GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, and LinuxKit +- macOS Apple pipeline: `core build apple`, codesign, notarisation, DMG packaging, TestFlight/App Store submission, Info.plist and entitlements generation, Xcode Cloud scripts +- OpenAPI SDK generation for TypeScript, Python, Go, and PHP with `oasdiff` breaking-change checks +- Reusable release workflow generation via `core build workflow` + +## Action/Workflow Parity + +The generated reusable workflow mirrors the `dAppCore/build@v3` action architecture: + +- Auto-detects stack markers including subtree frontend manifests and MkDocs projects +- Installs Go, Node, Python, Conan, MkDocs, Deno, and Wails when required +- Applies distro-aware Linux WebKit dependencies for Wails builds +- Supports obfuscation, NSIS packaging, WebView2 modes, Deno frontend overrides, and build cache restore/save +- Uploads workflow artifacts using the action-style naming shape: `{build-name}_{os}_{arch}_{tag|shortsha}` + +## Commands + +```bash +core build +core build apple +core build workflow +core build sdk +core ci +``` + +## Module + +```go +import "dappco.re/go/core/build/pkg/build" +``` + +The repository is a library/command-registration module. It does not ship its own standalone binary. diff --git a/cmd/build/cmd_workflow_test.go b/cmd/build/cmd_workflow_test.go index cd09164..7002f29 100644 --- a/cmd/build/cmd_workflow_test.go +++ b/cmd/build/cmd_workflow_test.go @@ -187,6 +187,11 @@ func TestBuildCmd_RunReleaseWorkflow_Good(t *testing.T) { assert.Contains(t, content, "--build-cache=false") assert.Contains(t, content, "--wails-build-webview2") assert.Contains(t, content, "--archive-format") + assert.Contains(t, content, "Compute artifact upload name") + assert.Contains(t, content, "build_name=\"${{ inputs.build-name }}\"") + assert.Contains(t, content, "build_name=\"${GITHUB_REPOSITORY##*/}\"") + assert.Contains(t, content, "suffix=\"${GITHUB_SHA::7}\"") + assert.Contains(t, content, "name: ${{ steps.artifact-name.outputs.value }}") assert.Contains(t, content, "if: ${{ inputs.package }}") assert.Contains(t, content, "if: ${{ inputs.build && inputs.package }}") assert.Contains(t, content, "actions/download-artifact@v4") @@ -238,6 +243,8 @@ func TestBuildCmd_RunReleaseWorkflow_Good(t *testing.T) { assert.Contains(t, content, "--build-name") assert.Contains(t, content, "--build-tags") assert.Contains(t, content, "--archive-format") + assert.Contains(t, content, "Compute artifact upload name") + assert.Contains(t, content, "name: ${{ steps.artifact-name.outputs.value }}") assert.Contains(t, content, "actions/download-artifact@v4") assert.Contains(t, content, "command: ci") }) diff --git a/docs/architecture.md b/docs/architecture.md index 3473767..6bc9bd0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,29 +1,38 @@ --- title: Architecture -description: Internal design of go-build -- types, data flow, and extension points. +description: Internal design of the build, action/workflow, Apple, release, and SDK layers. --- # Architecture -go-build is organised into three independent subsystems that share common types: **build**, **release**, and **sdk**. The CLI layer in `cmd/` wires them together but the library packages under `pkg/` can be used programmatically without the CLI. +go-build has four major surfaces that share types but can be used independently: -## Build Subsystem +1. `pkg/build` for discovery, builders, artifacts, caches, workflow generation, and Apple packaging +2. `pkg/release` for versioning, changelogs, and publishers +3. `pkg/sdk` for OpenAPI diffing and SDK generation +4. `cmd/` for registering `core build`, `core ci`, and `core sdk` -### Project Discovery +## Discovery and Stack Suggestion -`build.Discover()` scans a directory for marker files and returns detected project types in priority order (most specific first). For example, a Wails project contains both `wails.json` and `go.mod`, so `Discover` returns `[wails, go]`. `PrimaryType()` returns only the first match. +`build.Discover()` and `build.DiscoverFull()` implement the action-style discovery pass. -Detection order: +They record: -1. `wails.json` -- `ProjectTypeWails` -2. `go.mod` -- `ProjectTypeGo` -3. `package.json` -- `ProjectTypeNode` -4. `composer.json` -- `ProjectTypePHP` -5. `mkdocs.yml` -- `ProjectTypeDocs` +- detected project types in priority order +- raw marker presence +- whether a frontend exists at the root, in `frontend/`, or in a subtree up to depth 2 +- distro-aware Linux package requirements +- an action-facing stack suggestion (`wails2`, `cpp`, `docs`, `node`, `go`) -Docker (`Dockerfile`), LinuxKit (`linuxkit.yml` or `.core/linuxkit/*.yml`), C++ (`CMakeLists.txt`), and Taskfile (`Taskfile.yml`) are detected by their respective builders' `Detect()` methods rather than the central discovery function. +Important detection behaviour: -### Builder Interface +- Wails detection accepts `wails.json` and also Go roots that contain frontend manifests +- Docs detection accepts `mkdocs.yml` and `mkdocs.yaml` in the root or `docs/` +- Docker detection accepts `Dockerfile` and `Containerfile` variants +- LinuxKit detection accepts root manifests and `.core/linuxkit/*.yml` +- Taskfile detection accepts common case variants + +## Builder Layer Every builder implements: @@ -35,230 +44,80 @@ type Builder interface { } ``` -The `Config` struct carries runtime parameters (filesystem medium, project directory, output directory, binary name, version, linker flags) plus Docker- and LinuxKit-specific fields. +Current implementations: -`Build()` returns a slice of `Artifact` values, each recording the output path, target OS, and target architecture: - -```go -type Artifact struct { - Path string - OS string - Arch string - Checksum string -} -``` - -### Builder Implementations - -| Builder | Detection | Strategy | -|---|---|---| -| **GoBuilder** | `go.mod` or `wails.json` | Sets `GOOS`/`GOARCH`/`CGO_ENABLED=0`, runs `go build -trimpath` with ldflags. Output per target: `dist/{os}_{arch}/{binary}`. | -| **WailsBuilder** | `wails.json` | Checks `go.mod` for Wails v3 vs v2. V3 delegates to TaskfileBuilder; V2 runs `wails build -platform` then copies from `build/bin/` to `dist/`. | -| **NodeBuilder** | `package.json` | Detects the active package manager from lockfiles, runs the build script once per target, and collects artifacts from `dist/{os}_{arch}/`. | -| **PHPBuilder** | `composer.json` | Runs `composer install`, then `composer run-script build` when present. Falls back to a deterministic zip bundle in `dist/{os}_{arch}/`. | -| **PythonBuilder** | `pyproject.toml` or `requirements.txt` | Packages the project tree into a deterministic zip bundle in `dist/{os}_{arch}/`. | -| **RustBuilder** | `Cargo.toml` | Runs `cargo build --release --target` per platform and collects executables from `target/{triple}/release/`. | -| **DocsBuilder** | `mkdocs.yml` | Runs `mkdocs build --clean --site-dir` and packages the generated `site/` tree into a zip bundle per target. | -| **DockerBuilder** | `Dockerfile` | Validates `docker` and `buildx`, builds multi-platform images with `docker buildx build --platform`. Supports `--push` or local load/OCI tarball. | -| **LinuxKitBuilder** | `linuxkit.yml` or `.core/linuxkit/*.yml` | Validates `linuxkit` CLI, runs `linuxkit build --format --name --dir --arch`. Outputs qcow2, iso, raw, vmdk, vhd, or cloud images. Linux-only targets. | -| **CPPBuilder** | `CMakeLists.txt` | Validates `make`, runs `make configure` then `make build` then `make package` for host builds. Cross-compilation uses Conan profile targets (e.g. `make gcc-linux-armv8`). Finds artifacts in `build/packages/` or `build/release/src/`. | -| **TaskfileBuilder** | `Taskfile.yml` / `Taskfile.yaml` / `Taskfile` | Validates `task` CLI, runs `task build` with `GOOS`, `GOARCH`, `OUTPUT_DIR`, `NAME`, `VERSION` as both env vars and task vars. Discovers artifacts by platform subdirectory or filename pattern. | - -### Post-Build Pipeline - -After building, the CLI orchestrates three optional steps: - -1. **Signing** -- `signing.SignBinaries()` codesigns darwin artifacts with hardened runtime. `signing.NotarizeBinaries()` submits to Apple via `xcrun notarytool` and staples. `signing.SignChecksums()` creates GPG detached signatures (`.asc`). - -2. **Archiving** -- `build.ArchiveAll()` (or `ArchiveAllXZ()`) wraps each artifact. Linux/macOS get `tar.gz` (or `tar.xz`); Windows gets `zip`. XZ compression uses the Borg library. Archive filenames follow the pattern `{binary}_{os}_{arch}.tar.gz`. - -3. **Checksums** -- `build.ChecksumAll()` computes SHA-256 for each archive. `build.WriteChecksumFile()` writes a sorted `CHECKSUMS.txt` in the standard `sha256 filename` format. - -### Signing Architecture - -The `Signer` interface: - -```go -type Signer interface { - Name() string - Available() bool - Sign(ctx context.Context, fs io.Medium, path string) error -} -``` - -Three implementations: - -- **GPGSigner** -- `gpg --detach-sign --armor --local-user {key}`. Produces `.asc` files. -- **MacOSSigner** -- `codesign --sign {identity} --timestamp --options runtime --force`. Notarisation via `xcrun notarytool submit --wait` then `xcrun stapler staple`. -- **WindowsSigner** -- Uses `signtool` on Windows when a certificate is configured. - -Configuration supports `$ENV` expansion in all credential fields, so secrets can come from environment variables without being written to YAML. - -### Configuration Loading - -`build.LoadConfig(fs, dir)` reads `.core/build.yaml`. If the file is missing, `DefaultConfig()` provides: - -- Version 1 format -- Main package: `.` -- Flags: `["-trimpath"]` -- LDFlags: `["-s", "-w"]` -- CGO: disabled -- Targets: `linux/amd64`, `linux/arm64`, `darwin/arm64`, `windows/amd64` -- Signing: enabled, credentials from environment - -Fields present in the YAML override defaults; omitted fields inherit defaults via `applyDefaults()`. - -### Filesystem Abstraction - -All file operations go through `io.Medium` from `forge.lthn.ai/core/go-io`. Production code uses `io.Local` (real filesystem); tests can inject mock mediums. This makes builders unit-testable without touching the real filesystem for detection and configuration loading. - ---- - -## Release Subsystem - -### Version Resolution - -`release.DetermineVersion(dir)` resolves the release version: - -1. If HEAD has an exact git tag, use it. -2. If there is a previous tag, increment its patch number (e.g. `v1.2.3` becomes `v1.2.4`). -3. If no tags exist, default to `v0.0.1`. - -Helper functions `IncrementMinor()` and `IncrementMajor()` are available for manual version bumps. `ParseVersion()` decomposes a semver string into major, minor, patch, pre-release, and build components. `CompareVersions()` returns -1, 0, or 1. - -All versions are normalised to include a `v` prefix. - -### Changelog Generation - -`release.Generate(dir, fromRef, toRef)` parses git history between two refs and produces grouped Markdown. - -Commits are parsed against the conventional-commit regex: - -``` -^(\w+)(?:\(([^)]+)\))?(!)?:\s*(.+)$ -``` - -This matches patterns like `feat: add feature`, `fix(scope): fix bug`, or `feat!: breaking change`. - -Parsed commits are grouped by type and rendered in a fixed order: breaking changes first, then features, bug fixes, performance, refactoring, and so on. Each entry includes the optional scope (bolded) and the short commit hash. - -`GenerateWithConfig()` adds include/exclude filtering by commit type, driven by the `changelog` section in `.core/release.yaml`. - -### Release Orchestration - -Two entry points: - -- **`release.Run()`** -- Full pipeline: determine version, generate changelog, build artifacts (via the build subsystem), archive, checksum, then publish to all configured targets. -- **`release.Publish()`** -- Publish-only: expects pre-built artifacts in `dist/`, generates changelog, then publishes. This supports the separated `core build` then `core ci` workflow. - -Both accept a `dryRun` parameter. When true, publishers print what would happen without executing. - -### Publisher Interface - -```go -type Publisher interface { - Name() string - Publish(ctx context.Context, release *Release, pubCfg PublisherConfig, relCfg ReleaseConfig, dryRun bool) error -} -``` - -Eight publishers are implemented: - -| Publisher | Mechanism | +| Builder | Notes | |---|---| -| **GitHub** | `gh release create` via the GitHub CLI. Auto-detects repository from git remote. Uploads all artifacts as release assets. | -| **Docker** | `docker buildx build` with multi-platform support. Pushes to configured registry with version tags. | -| **npm** | `npm publish` with configurable access level and package name. | -| **Homebrew** | Generates a Ruby formula file. Optionally targets an official tap repository. | -| **Scoop** | Generates a JSON manifest for a Scoop bucket. | -| **AUR** | Generates a PKGBUILD file for the Arch User Repository. | -| **Chocolatey** | Generates a `.nuspec` and `chocolateyinstall.ps1`. Optionally pushes via `choco push`. | -| **LinuxKit** | Builds LinuxKit VM images in specified formats and uploads them as release assets. | +| Go | Cross-compiles binaries and supports garble obfuscation plus cache env wiring | +| Wails | Handles Wails v2 directly and Wails v3 through Taskfile or CLI fallback; supports NSIS, WebView2, Deno, subtree frontends, and obfuscation | +| Node | Detects package manager, supports Deno manifests, and builds nested frontend projects | +| PHP | Composer-backed builds with deterministic zip fallback | +| Python | Deterministic source bundle packaging | +| Rust | Cargo release builds by target triple | +| Docs | MkDocs build plus zipped site output | +| Docker | Buildx-backed image builds with push/load/archive modes | +| LinuxKit | LinuxKit image generation in configured formats | +| C++ | Make + Conan orchestration with profile-based cross-builds | +| Taskfile | Generic task-backed build pipeline used heavily by Wails v3 projects | -Publisher-specific configuration (registry, tap, bucket, image, etc.) is carried in `PublisherConfig` fields and mapped to an `Extended` map at runtime. +## Generated GitHub Workflow -### SDK Release Integration +`core build workflow` writes `.github/workflows/release.yml`. The generated workflow mirrors the modular `dAppCore/build@v3` action pipeline: -`release.RunSDK()` handles SDK-specific releases: it runs a breaking-change diff (if enabled), then generates SDKs via the SDK subsystem. This can be wired into a CI pipeline to auto-generate client libraries on each release. +1. Checkout +2. Discovery by file markers through `hashFiles(...)` +3. Toolchain setup for Go, Node, Python, Conan, MkDocs, Deno, and Wails +4. Linux distro-aware WebKit dependency setup for Wails +5. Cache restore under `.core/cache` and `cache/` +6. `core build --archive --checksum` +7. Artifact upload with action-style names: `{build-name}_{os}_{arch}_{tag|shortsha}` +8. Release publishing through `core ci` ---- +The workflow keeps the action inputs exposed at the CLI layer: -## SDK Subsystem +- `build-name` +- `build-platform` +- `build-tags` +- `build-obfuscate` +- `nsis` +- `deno-build` +- `wails-build-webview2` +- `build-cache` -### Spec Detection +## Apple Pipeline -`sdk.DetectSpec()` locates the OpenAPI specification: +The Apple implementation lives in `pkg/build/apple.go`, with an RFC-facing wrapper in `pkg/build/apple/`. -1. If a path is configured in `.core/release.yaml` under `sdk.spec`, use it. -2. Check common paths: `api/openapi.yaml`, `api/openapi.json`, `openapi.yaml`, `openapi.json`, `docs/api.yaml`, `docs/api.json`, `swagger.yaml`, `swagger.json`. -3. Check for Laravel Scramble in `composer.json` (export not yet implemented). +Key pieces: -### Breaking-Change Detection +- `AppleOptions` for the runtime pipeline +- `BuildApple()` for the end-to-end macOS build flow +- `BuildWailsApp()` and `CreateUniversal()` for architecture-specific and universal app bundles +- `Sign()`, `Notarise()`, `CreateDMG()`, `UploadTestFlight()`, and `SubmitAppStore()` for post-build delivery +- generated `Info.plist` and entitlements +- Xcode Cloud script generation from `.core/build.yaml` -`sdk.Diff(basePath, revisionPath)` loads two OpenAPI specs via `kin-openapi`, computes a structural diff via `oasdiff`, and runs the `oasdiff/checker` backward-compatibility checks at error level. Returns a `DiffResult` with a boolean `Breaking` flag, a list of change descriptions, and a human-readable summary. +`cmd/build/cmd_apple.go` wires this into `core build apple`. -`DiffExitCode()` maps results to CI exit codes: 0 (clean), 1 (breaking changes), 2 (error). +## Release Layer -### Code Generation +`pkg/release` owns: -The `generators.Generator` interface: +- semver version resolution from git tags +- changelog generation from conventional commits +- building or reusing `dist/` artifacts +- checksum and artifact metadata handling +- publisher orchestration -```go -type Generator interface { - Language() string - Generate(ctx context.Context, opts Options) error - Available() bool - Install() string -} -``` +Publishers currently cover GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, and LinuxKit. -Generators are held in a `Registry` and looked up by language identifier. Each generator tries three strategies in order: +## SDK Layer -1. **Native tool** -- e.g. `oapi-codegen` for Go, `openapi-typescript-codegen` for TypeScript. -2. **npx** -- Falls back to `npx` invocation where applicable (TypeScript). -3. **Docker** -- Uses the `openapitools/openapi-generator-cli` image as a last resort. +`pkg/sdk` detects an OpenAPI spec, validates it, compares revisions with `oasdiff`, and generates SDKs for: -| Language | Native Tool | Docker Generator | -|---|---|---| -| TypeScript | `openapi-typescript-codegen` or `npx` | `typescript-fetch` | -| Python | `openapi-python-client` | `python` | -| Go | `oapi-codegen` | `go` | -| PHP | `openapi-generator-cli` via Docker | `php` | +- TypeScript +- Python +- Go +- PHP -On Unix systems, Docker containers run with `--user {uid}:{gid}` to match host file ownership. - ---- - -## Data Flow Summary - -``` -.core/build.yaml -----> LoadConfig() -----> BuildConfig - | -project directory ----> Discover() -----------> ProjectType - | - getBuilder() - | - Builder.Build() - | - []Artifact (raw binaries) - | - +----------------------+---------------------+ - | | | - SignBinaries() ArchiveAll() (optional) - | | NotarizeBinaries() - | []Artifact (archives) - | | - | ChecksumAll() - | | - | []Artifact (with checksums) - | | - | WriteChecksumFile() - | | - +----------+-----------+ - | - SignChecksums() (GPG) - | - Publisher.Publish() - | - GitHub / Docker / npm / Homebrew / ... -``` +Generators prefer native tooling first and fall back to `npx` or Docker where appropriate. diff --git a/docs/index.md b/docs/index.md index fbc1270..00127db 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,211 +1,92 @@ --- title: go-build -description: Build system, release pipeline, and SDK generation for the Core ecosystem. +description: Build, release, Apple packaging, SDK generation, and GitHub workflow tooling for Core projects. --- # go-build -`forge.lthn.ai/core/go-build` is the build, release, and SDK generation toolkit for Core projects. It provides: +`dappco.re/go/core/build` is the build system and release engine used by the Core CLI and the public `dAppCore/build@v3` GitHub Action. -- **Auto-detecting builders** for Go, Wails, Node, PHP, Python, Rust, Docs, Docker, LinuxKit, C++, and Taskfile projects -- **Cross-compilation** with per-target archiving (tar.gz, tar.xz, zip) and SHA-256 checksums -- **Code signing** -- macOS codesign with notarisation, GPG detached signatures, Windows signtool -- **Release automation** -- semantic versioning from git tags, conventional-commit changelogs, multi-target publishing -- **SDK generation** -- OpenAPI spec diffing for breaking-change detection, code generation for TypeScript, Python, Go, and PHP -- **CLI integration** -- registers `core build`, `core ci`, and `core sdk` commands via the Core CLI framework +## Highlights + +- Auto-detecting builders for Go, Wails, Node, PHP, Python, Rust, Docs, Docker, LinuxKit, C++, and Taskfile projects +- Action-oriented discovery hints for `wails2`, `cpp`, `docs`, `node`, and `go` +- Generated reusable GitHub release workflow with Go/Node/Python/Deno setup, Conan/MkDocs hooks, distro-aware WebKit packages, cache restore/save, and canonical artifact naming +- macOS Apple pipeline with `core build apple`, DMG packaging, notarisation, Xcode Cloud script generation, TestFlight, and App Store submission +- Release orchestration with eight publishers +- OpenAPI SDK generation with breaking-change detection + +## Commands + +```bash +core build +core build apple +core build workflow +core build sdk +core ci +core sdk +``` + +## Build Surfaces + +| Surface | Purpose | +|---|---| +| `pkg/build/` | Discovery, config, caches, archives, checksums, workflow generation, Apple pipeline | +| `pkg/build/builders/` | Builder implementations for all supported stacks | +| `pkg/build/apple/` | RFC-facing Apple wrapper that exposes `core.Result` contracts | +| `pkg/build/signing/` | GPG, macOS codesign/notarisation, Windows signtool | +| `pkg/release/` | Versioning, changelog generation, publishing orchestration | +| `pkg/release/publishers/` | GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit | +| `pkg/sdk/` | Spec detection, diffing, and SDK generation | +| `cmd/build/` | `core build`, `core build apple`, `core build workflow`, `core build sdk`, `core build release` | +| `cmd/ci/` | `core ci` publish/version/changelog commands | +| `cmd/sdk/` | `core sdk diff` and `core sdk validate` | + +## Builder Detection + +Discovery checks the project root and selected nested paths: + +| Marker | Result | +|---|---| +| `.core/build.yaml` | Config-driven override | +| `wails.json` or `go.mod`/`go.work` plus frontend manifests | Wails | +| `go.mod` or `go.work` | Go | +| `package.json`, `deno.json`, `deno.jsonc` | Node/Deno | +| `mkdocs.yml`, `mkdocs.yaml`, `docs/mkdocs.yml`, `docs/mkdocs.yaml` | Docs | +| `CMakeLists.txt` | C++ | +| `Dockerfile`, `Containerfile` variants | Docker | +| `linuxkit.yml`, `linuxkit.yaml`, `.core/linuxkit/*.yml` | LinuxKit | +| `Taskfile.yml`, `Taskfile.yaml`, `Taskfile` variants | Taskfile | +| `composer.json`, `pyproject.toml`, `requirements.txt`, `Cargo.toml` | PHP, Python, Rust | + +Monorepo frontend discovery scans subtree manifests to depth 2 and ignores `node_modules` and hidden directories. + +## GitHub Workflow Generation + +`core build workflow` writes a reusable release workflow that: + +1. Detects the required toolchains from the repository contents. +2. Installs Go, Node, Python, Conan, MkDocs, Deno, and Wails only when needed. +3. Restores build caches under `.core/cache` and `cache/`. +4. Applies Ubuntu 24.04 WebKit 4.1 handling for Wails Linux builds. +5. Runs `core build --archive --checksum`. +6. Uploads artifacts with action-style names and publishes with `core ci`. + +## Apple Pipeline + +The Apple surface is available both through `pkg/build/apple/` and `core build apple`. It supports: + +- universal, arm64, and amd64 app builds +- codesign and notarisation +- DMG creation +- TestFlight and App Store submission +- generated `Info.plist` and entitlements +- Xcode Cloud helper scripts checked into the project ## Module Path -``` -forge.lthn.ai/core/go-build +```go +import "dappco.re/go/core/build/pkg/build" ``` -Requires **Go 1.26+**. - -## Quick Start - -### Build a project - -From any project directory containing a recognised marker file: - -```bash -core build # Auto-detect type, build for configured targets -core build --targets linux/amd64 # Single target -core build --ci # JSON output for CI pipelines -core build --verbose # Detailed step-by-step output -``` - -The builder is chosen by marker-file priority: - -| Marker file | Builder | -|-------------------|------------| -| `wails.json` | Wails | -| `go.mod` | Go | -| `package.json` | Node | -| `composer.json` | PHP | -| `pyproject.toml` | Python | -| `Cargo.toml` | Rust | -| `mkdocs.yml` | Docs | -| `CMakeLists.txt` | C++ | -| `Dockerfile` | Docker | -| `linuxkit.yml` | LinuxKit | -| `Taskfile.yml` | Taskfile | - -### Release artifacts - -```bash -core build release --we-are-go-for-launch # Build + archive + checksum + publish -core build release # Dry-run (default without the flag) -core build release --draft --prerelease # Mark as draft pre-release -``` - -### Publish pre-built artifacts - -After `core build` has populated `dist/`: - -```bash -core ci # Dry-run publish from dist/ -core ci --we-are-go-for-launch # Actually publish -core ci --version v1.2.3 # Override version -``` - -### Generate changelogs - -```bash -core ci changelog # From latest tag to HEAD -core ci changelog --from v0.1.0 --to v0.2.0 -core ci version # Show determined next version -core ci init # Scaffold .core/release.yaml -``` - -### SDK operations - -```bash -core build sdk # Generate SDKs for all configured languages -core build sdk --lang typescript # Single language -core sdk diff --base v1.0.0 --spec api/openapi.yaml # Breaking-change check -core sdk validate # Validate OpenAPI spec -``` - -## Package Layout - -``` -forge.lthn.ai/core/go-build/ -| -|-- cmd/ -| |-- build/ CLI commands for `core build` (build, from-path, pwa, sdk, release) -| |-- ci/ CLI commands for `core ci` (init, changelog, version, publish) -| +-- sdk/ CLI commands for `core sdk` (diff, validate) -| -+-- pkg/ - |-- build/ Core build types, config loading, discovery, archiving, checksums -| |-- builders/ Builder implementations (Go, Wails, Node, PHP, Python, Docs, Docker, LinuxKit, C++, Taskfile) - | +-- signing/ Code-signing implementations (macOS codesign, GPG, Windows stub) - | - |-- release/ Release orchestration, versioning, changelog, config - | +-- publishers/ Publisher implementations (GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit) - | - +-- sdk/ OpenAPI SDK generation and breaking-change diffing - +-- generators/ Language generators (TypeScript, Python, Go, PHP) -``` - -## Configuration Files - -Build and release behaviour is driven by two YAML files in the `.core/` directory. - -### `.core/build.yaml` - -Controls compilation targets, flags, and signing: - -```yaml -version: 1 -project: - name: myapp - description: My application - main: ./cmd/myapp - binary: myapp -build: - cgo: false - flags: ["-trimpath"] - ldflags: ["-s", "-w"] - env: [] -targets: - - os: linux - arch: amd64 - - os: linux - arch: arm64 - - os: darwin - arch: arm64 - - os: windows - arch: amd64 -sign: - enabled: true - gpg: - key: $GPG_KEY_ID - macos: - identity: $CODESIGN_IDENTITY - notarize: false - apple_id: $APPLE_ID - team_id: $APPLE_TEAM_ID - app_password: $APPLE_APP_PASSWORD -``` - -When no `.core/build.yaml` exists, sensible defaults apply (CGO off, `-trimpath -s -w`, four standard targets). - -### `.core/release.yaml` - -Controls versioning, changelog filtering, publishers, and SDK generation: - -```yaml -version: 1 -project: - name: myapp - repository: owner/repo -build: - targets: - - os: linux - arch: amd64 - - os: darwin - arch: arm64 -publishers: - - type: github - draft: false - prerelease: false - - type: homebrew - tap: owner/homebrew-tap - - type: docker - registry: ghcr.io - image: owner/myapp - tags: ["latest", "{{.Version}}"] -changelog: - include: [feat, fix, perf, refactor] - exclude: [chore, docs, style, test, ci] -sdk: - spec: api/openapi.yaml - languages: [typescript, python, go, php] - output: sdk - diff: - enabled: true - fail_on_breaking: false -``` - -## Dependencies - -| Dependency | Purpose | -|---|---| -| `forge.lthn.ai/core/cli` | CLI command registration and TUI styling | -| `forge.lthn.ai/core/go-io` | Filesystem abstraction (`io.Medium`, `io.Local`) | -| `forge.lthn.ai/core/go-i18n` | Internationalised CLI labels | -| `forge.lthn.ai/core/go-log` | Structured error logging | -| `github.com/Snider/Borg` | XZ compression for tar.xz archives | -| `github.com/getkin/kin-openapi` | OpenAPI spec loading and validation | -| `github.com/oasdiff/oasdiff` | OpenAPI diff and breaking-change detection | -| `gopkg.in/yaml.v3` | YAML config parsing | -| `github.com/leaanthony/debme` | Embedded filesystem anchoring (PWA templates) | -| `github.com/leaanthony/gosod` | Template extraction for PWA builds | -| `golang.org/x/net` | HTML parsing for PWA manifest detection | -| `golang.org/x/text` | Changelog section title casing | - -## Licence - -EUPL-1.2 +Requires Go 1.26+. diff --git a/pkg/build/templates/release.yml b/pkg/build/templates/release.yml index 4a36ba6..af16ba3 100644 --- a/pkg/build/templates/release.yml +++ b/pkg/build/templates/release.yml @@ -219,19 +219,14 @@ jobs: matrix: include: - target: linux/amd64 - artifact_name: linux-amd64 runner: ubuntu-latest - target: linux/arm64 - artifact_name: linux-arm64 runner: ubuntu-latest - target: darwin/amd64 - artifact_name: darwin-amd64 runner: macos-13 - target: darwin/arm64 - artifact_name: darwin-arm64 runner: macos-14 - target: windows/amd64 - artifact_name: windows-amd64 runner: windows-latest steps: - name: Checkout repository @@ -445,11 +440,38 @@ jobs: "${args[@]}" + - name: Compute artifact upload name + id: artifact-name + shell: bash + run: | + set -euo pipefail + + build_name="${{ inputs.build-name }}" + if [ -z "$build_name" ]; then + build_name="${GITHUB_REPOSITORY##*/}" + fi + + target="${{ matrix.target }}" + target_os="${target%%/*}" + target_arch="${target#*/}" + + suffix="${GITHUB_SHA::7}" + if [ "${GITHUB_REF:-}" != "${GITHUB_REF#refs/tags/}" ] && [ -n "${GITHUB_REF_NAME:-}" ]; then + suffix="${GITHUB_REF_NAME}" + fi + + artifact_name="${build_name}_${target_os}_${target_arch}" + if [ -n "$suffix" ]; then + artifact_name="${artifact_name}_${suffix}" + fi + + echo "value=release-${artifact_name}" >> "${GITHUB_OUTPUT}" + - name: Upload artefacts if: ${{ inputs.package }} uses: actions/upload-artifact@v4 with: - name: release-${{ matrix.artifact_name }} + name: ${{ steps.artifact-name.outputs.value }} path: ${{ inputs.working-directory }}/dist/** if-no-files-found: error diff --git a/pkg/build/workflow_test.go b/pkg/build/workflow_test.go index ff6b2ec..6ae3d15 100644 --- a/pkg/build/workflow_test.go +++ b/pkg/build/workflow_test.go @@ -72,6 +72,13 @@ func TestWorkflow_WriteReleaseWorkflow_Good(t *testing.T) { assert.Contains(t, content, "--build-cache=false") assert.Contains(t, content, "--wails-build-webview2") assert.Contains(t, content, "--archive-format") + assert.Contains(t, content, "Compute artifact upload name") + assert.Contains(t, content, "build_name=\"${{ inputs.build-name }}\"") + assert.Contains(t, content, "build_name=\"${GITHUB_REPOSITORY##*/}\"") + assert.Contains(t, content, "suffix=\"${GITHUB_SHA::7}\"") + assert.Contains(t, content, "GITHUB_REF#refs/tags/") + assert.Contains(t, content, "name: ${{ steps.artifact-name.outputs.value }}") + assert.Contains(t, content, "echo \"value=release-${artifact_name}\" >> \"${GITHUB_OUTPUT}\"") assert.Contains(t, content, "if: ${{ inputs.package }}") assert.Contains(t, content, "if: ${{ inputs.build && inputs.package }}") assert.Contains(t, content, "actions/download-artifact@v4")