diff --git a/.core/reference/RFC-025-AGENT-EXPERIENCE.md b/.core/reference/RFC-025-AGENT-EXPERIENCE.md new file mode 100644 index 0000000..a18e6bb --- /dev/null +++ b/.core/reference/RFC-025-AGENT-EXPERIENCE.md @@ -0,0 +1,588 @@ +# RFC-025: Agent Experience (AX) Design Principles + +- **Status:** Active +- **Authors:** Snider, Cladius +- **Date:** 2026-03-25 +- **Applies to:** All Core ecosystem packages (CoreGO, CorePHP, CoreTS, core-agent) + +## Abstract + +Agent Experience (AX) is a design paradigm for software systems where the primary code consumer is an AI agent, not a human developer. AX sits alongside User Experience (UX) and Developer Experience (DX) as the third era of interface design. + +This RFC establishes AX as a formal design principle for the Core ecosystem and defines the conventions that follow from it. + +## Motivation + +As of early 2026, AI agents write, review, and maintain the majority of code in the Core ecosystem. The original author has not manually edited code (outside of Core struct design) since October 2025. Code is processed semantically — agents reason about intent, not characters. + +Design patterns inherited from the human-developer era optimise for the wrong consumer: + +- **Short names** save keystrokes but increase semantic ambiguity +- **Functional option chains** are fluent for humans but opaque for agents tracing configuration +- **Error-at-every-call-site** produces 50% boilerplate that obscures intent +- **Generic type parameters** force agents to carry type context that the runtime already has +- **Panic-hiding conventions** (`Must*`) create implicit control flow that agents must special-case +- **Raw exec.Command** bypasses Core primitives — untestable, no entitlement check, path traversal risk + +AX acknowledges this shift and provides principles for designing code, APIs, file structures, and conventions that serve AI agents as first-class consumers. + +## The Three Eras + +| Era | Primary Consumer | Optimises For | Key Metric | +|-----|-----------------|---------------|------------| +| UX | End users | Discoverability, forgiveness, visual clarity | Task completion time | +| DX | Developers | Typing speed, IDE support, convention familiarity | Time to first commit | +| AX | AI agents | Predictability, composability, semantic navigation | Correct-on-first-pass rate | + +AX does not replace UX or DX. End users still need good UX. Developers still need good DX. But when the primary code author and maintainer is an AI agent, the codebase should be designed for that consumer first. + +## Principles + +### 1. Predictable Names Over Short Names + +Names are tokens that agents pattern-match across languages and contexts. Abbreviations introduce mapping overhead. + +``` +Config not Cfg +Service not Srv +Embed not Emb +Error not Err (as a subsystem name; err for local variables is fine) +Options not Opts +``` + +**Rule:** If a name would require a comment to explain, it is too short. + +**Exception:** Industry-standard abbreviations that are universally understood (`HTTP`, `URL`, `ID`, `IPC`, `I18n`) are acceptable. The test: would an agent trained on any mainstream language recognise it without context? + +### 2. Comments as Usage Examples + +The function signature tells WHAT. The comment shows HOW with real values. + +```go +// Entitled checks if an action is permitted. +// +// e := c.Entitled("process.run") +// e := c.Entitled("social.accounts", 3) +// if e.Allowed { proceed() } + +// WriteAtomic writes via temp file then rename (safe for concurrent readers). +// +// r := fs.WriteAtomic("/status.json", data) + +// Action registers or invokes a named callable. +// +// c.Action("git.log", handler) // register +// c.Action("git.log").Run(ctx, opts) // invoke +``` + +**Rule:** If a comment restates what the type signature already says, delete it. If a comment shows a concrete usage with realistic values, keep it. + +**Rationale:** Agents learn from examples more effectively than from descriptions. A comment like "Run executes the setup process" adds zero information. A comment like `setup.Run(setup.Options{Path: ".", Template: "auto"})` teaches an agent exactly how to call the function. + +### 3. Path Is Documentation + +File and directory paths should be self-describing. An agent navigating the filesystem should understand what it is looking at without reading a README. + +``` +pkg/agentic/dispatch.go — agent dispatch logic +pkg/agentic/handlers.go — IPC event handlers +pkg/lib/task/bug-fix.yaml — bug fix plan template +pkg/lib/persona/engineering/ — engineering personas +flow/deploy/to/homelab.yaml — deploy TO the homelab +template/dir/workspace/default/ — default workspace scaffold +docs/RFC.md — authoritative API contract +``` + +**Rule:** If an agent needs to read a file to understand what a directory contains, the directory naming has failed. + +**Corollary:** The unified path convention (folder structure = HTTP route = CLI command = test path) is AX-native. One path, every surface. + +### 4. Templates Over Freeform + +When an agent generates code from a template, the output is constrained to known-good shapes. When an agent writes freeform, the output varies. + +```go +// Template-driven — consistent output +lib.ExtractWorkspace("default", targetDir, &lib.WorkspaceData{ + Repo: "go-io", Branch: "dev", Task: "fix tests", Agent: "codex", +}) + +// Freeform — variance in output +"write a workspace setup script that..." +``` + +**Rule:** For any code pattern that recurs, provide a template. Templates are guardrails for agents. + +**Scope:** Templates apply to file generation, workspace scaffolding, config generation, and commit messages. They do NOT apply to novel logic — agents should write business logic freeform with the domain knowledge available. + +### 5. Declarative Over Imperative + +Agents reason better about declarations of intent than sequences of operations. + +```yaml +# Declarative — agent sees what should happen +steps: + - name: build + flow: tools/docker-build + with: + context: "{{ .app_dir }}" + image_name: "{{ .image_name }}" + + - name: deploy + flow: deploy/with/docker + with: + host: "{{ .host }}" +``` + +```go +// Imperative — agent must trace execution +cmd := exec.Command("docker", "build", "--platform", "linux/amd64", "-t", imageName, ".") +cmd.Dir = appDir +if err := cmd.Run(); err != nil { + return core.E("build", "docker build failed", err) +} +``` + +**Rule:** Orchestration, configuration, and pipeline logic should be declarative (YAML/JSON). Implementation logic should be imperative (Go/PHP/TS). The boundary is: if an agent needs to compose or modify the logic, make it declarative. + +Core's `Task` is the Go-native declarative equivalent — a sequence of named Action steps: + +```go +c.Task("deploy", core.Task{ + Steps: []core.Step{ + {Action: "docker.build"}, + {Action: "docker.push"}, + {Action: "deploy.ansible", Async: true}, + }, +}) +``` + +### 6. Core Primitives — Universal Types and DI + +Every component in the ecosystem registers with Core and communicates through Core's primitives. An agent processing any level of the tree sees identical shapes. + +#### Creating Core + +```go +c := core.New( + core.WithOption("name", "core-agent"), + core.WithService(process.Register), + core.WithService(agentic.Register), + core.WithService(monitor.Register), + core.WithService(brain.Register), + core.WithService(mcp.Register), +) +c.Run() // or: if err := c.RunE(); err != nil { ... } +``` + +`core.New()` returns `*Core`. `WithService` registers a factory `func(*Core) Result`. Services auto-discover: name from package path, lifecycle from `Startable`/`Stoppable` (return `Result`). `HandleIPCEvents` is the one remaining magic method — auto-registered via reflection if the service implements it. + +#### Service Registration Pattern + +```go +// Service factory — receives Core, returns Result +func Register(c *core.Core) core.Result { + svc := &MyService{ + ServiceRuntime: core.NewServiceRuntime(c, MyOptions{}), + } + return core.Result{Value: svc, OK: true} +} +``` + +#### Core Subsystem Accessors + +| Accessor | Purpose | +|----------|---------| +| `c.Options()` | Input configuration | +| `c.App()` | Application metadata (name, version) | +| `c.Config()` | Runtime settings, feature flags | +| `c.Data()` | Embedded assets (Registry[*Embed]) | +| `c.Drive()` | Transport handles (Registry[*DriveHandle]) | +| `c.Fs()` | Filesystem I/O (sandboxable) | +| `c.Process()` | Managed execution (Action sugar) | +| `c.API()` | Remote streams (protocol handlers) | +| `c.Action(name)` | Named callable (register/invoke) | +| `c.Task(name)` | Composed Action sequence | +| `c.Entitled(name)` | Permission check | +| `c.RegistryOf(n)` | Cross-cutting registry queries | +| `c.Cli()` | CLI command framework | +| `c.IPC()` | Message bus (ACTION, QUERY) | +| `c.Log()` | Structured logging | +| `c.Error()` | Panic recovery | +| `c.I18n()` | Internationalisation | + +#### Primitive Types + +```go +// Option — the atom +core.Option{Key: "name", Value: "brain"} + +// Options — universal input +opts := core.NewOptions( + core.Option{Key: "name", Value: "myapp"}, + core.Option{Key: "port", Value: 8080}, +) +opts.String("name") // "myapp" +opts.Int("port") // 8080 + +// Result — universal output +core.Result{Value: svc, OK: true} +``` + +#### Named Actions — The Primary Communication Pattern + +Services register capabilities as named Actions. No direct function calls, no untyped dispatch — declare intent by name, invoke by name. + +```go +// Register a capability during OnStartup +c.Action("workspace.create", func(ctx context.Context, opts core.Options) core.Result { + name := opts.String("name") + path := core.JoinPath("/srv/workspaces", name) + return core.Result{Value: path, OK: true} +}) + +// Invoke by name — typed, inspectable, entitlement-checked +r := c.Action("workspace.create").Run(ctx, core.NewOptions( + core.Option{Key: "name", Value: "alpha"}, +)) + +// Check capability before calling +if c.Action("process.run").Exists() { /* go-process is registered */ } + +// List all capabilities +c.Actions() // ["workspace.create", "process.run", "brain.recall", ...] +``` + +#### Task Composition — Sequencing Actions + +```go +c.Task("agent.completion", core.Task{ + Steps: []core.Step{ + {Action: "agentic.qa"}, + {Action: "agentic.auto-pr"}, + {Action: "agentic.verify"}, + {Action: "agentic.poke", Async: true}, // doesn't block + }, +}) +``` + +#### Anonymous Broadcast — Legacy Layer + +`ACTION` and `QUERY` remain for backwards-compatible anonymous dispatch. New code should prefer named Actions. + +```go +// Broadcast — all handlers fire, type-switch to filter +c.ACTION(messages.DeployCompleted{Env: "production"}) + +// Query — first responder wins +r := c.QUERY(countQuery{}) +``` + +#### Process Execution — Use Core Primitives + +All external command execution MUST go through `c.Process()`, not raw `os/exec`. This makes process execution testable, gatable by entitlements, and managed by Core's lifecycle. + +```go +// AX-native: Core Process primitive +r := c.Process().RunIn(ctx, repoDir, "git", "log", "--oneline", "-20") +if r.OK { output := r.Value.(string) } + +// Not AX: raw exec.Command — untestable, no entitlement, no lifecycle +cmd := exec.Command("git", "log", "--oneline", "-20") +cmd.Dir = repoDir +out, err := cmd.Output() +``` + +**Rule:** If a package imports `os/exec`, it is bypassing Core's process primitive. The only package that should import `os/exec` is `go-process` itself. + +**Quality gate:** An agent reviewing a diff can mechanically check: does this import `os/exec`, `unsafe`, or `encoding/json` directly? If so, it bypassed a Core primitive. + +#### What This Replaces + +| Go Convention | Core AX | Why | +|--------------|---------|-----| +| `func With*(v) Option` | `core.WithOption(k, v)` | Named key-value is greppable; option chains require tracing | +| `func Must*(v) T` | `core.Result` | No hidden panics; errors flow through Result.OK | +| `func *For[T](c) T` | `c.Service("name")` | String lookup is greppable; generics require type context | +| `val, err :=` everywhere | Single return via `core.Result` | Intent not obscured by error handling | +| `exec.Command(...)` | `c.Process().Run(ctx, cmd, args...)` | Testable, gatable, lifecycle-managed | +| `map[string]*T + mutex` | `core.Registry[T]` | Thread-safe, ordered, lockable, queryable | +| untyped `any` dispatch | `c.Action("name").Run(ctx, opts)` | Named, typed, inspectable, entitlement-checked | + +### 7. Tests as Behavioural Specification + +Test names are structured data. An agent querying "what happens when dispatch fails?" should find the answer by scanning test names, not reading prose. + +``` +TestDispatch_DetectFinalStatus_Good — clean exit → completed +TestDispatch_DetectFinalStatus_Bad — non-zero exit → failed +TestDispatch_DetectFinalStatus_Ugly — BLOCKED.md overrides exit code +``` + +**Convention:** `Test{File}_{Function}_{Good|Bad|Ugly}` + +| Category | Purpose | +|----------|---------| +| `_Good` | Happy path — proves the contract works | +| `_Bad` | Expected errors — proves error handling works | +| `_Ugly` | Edge cases, panics, corruption — proves it doesn't blow up | + +**Rule:** Every testable function gets all three categories. Missing categories are gaps in the specification, detectable by scanning: + +```bash +# Find under-tested functions +for f in *.go; do + [[ "$f" == *_test.go ]] && continue + while IFS= read -r line; do + fn=$(echo "$line" | sed 's/func.*) //; s/(.*//; s/ .*//') + [[ -z "$fn" || "$fn" == register* ]] && continue + cap="${fn^}" + grep -q "_${cap}_Good\|_${fn}_Good" *_test.go || echo "$f: $fn missing Good" + grep -q "_${cap}_Bad\|_${fn}_Bad" *_test.go || echo "$f: $fn missing Bad" + grep -q "_${cap}_Ugly\|_${fn}_Ugly" *_test.go || echo "$f: $fn missing Ugly" + done < <(grep "^func " "$f") +done +``` + +**Rationale:** The test suite IS the behavioural spec. `grep _TrackFailureRate_ *_test.go` returns three concrete scenarios — no prose needed. The naming convention makes the entire test suite machine-queryable. An agent dispatched to fix a function can read its tests to understand the full contract before making changes. + +**What this replaces:** + +| Convention | AX Test Naming | Why | +|-----------|---------------|-----| +| `TestFoo_works` | `TestFile_Foo_Good` | File prefix enables cross-file search | +| Unnamed table tests | Explicit Good/Bad/Ugly | Categories are scannable without reading test body | +| Coverage % as metric | Missing categories as metric | 100% coverage with only Good tests is a false signal | + +### 7b. Example Tests as AX TDD + +Go `Example` functions serve triple duty: they run as tests (count toward coverage), show in godoc (usage documentation), and seed user guide generation. + +```go +// file: action_example_test.go + +func ExampleAction_Run() { + c := New() + c.Action("double", func(_ context.Context, opts Options) Result { + return Result{Value: opts.Int("n") * 2, OK: true} + }) + + r := c.Action("double").Run(context.Background(), NewOptions( + Option{Key: "n", Value: 21}, + )) + Println(r.Value) + // Output: 42 +} +``` + +**AX TDD pattern:** Write the Example first — it defines how the API should feel. If the Example is awkward, the API is wrong. The Example IS the test, the documentation, and the design feedback loop. + +**Convention:** One `{source}_example_test.go` per source file. Every exported function should have at least one Example. The Example output comment makes it a verified test. + +**Quality gate:** A source file without a corresponding example file is missing documentation that compiles. + +### Operational Principles + +Principles 1-7 govern code design. Principles 8-10 govern how agents and humans work with the codebase. + +### 8. RFC as Domain Load + +An agent's first action in a session should be loading the repo's RFC.md. The full spec in context produces zero-correction sessions — every decision aligns with the design because the design is loaded. + +**Validated:** Loading core/go's RFC.md (42k tokens from a 500k token discovery session) at session start eliminated all course corrections. The spec is compressed domain knowledge that survives context compaction. + +**Rule:** Every repo that has non-trivial architecture should have a `docs/RFC.md`. The RFC is not documentation for humans — it's a context document for agents. It should be loadable in one read and contain everything needed to make correct decisions. + +### 9. Primitives as Quality Gates + +Core primitives become mechanical code review rules. An agent reviewing a diff checks: + +| Import | Violation | Use Instead | +|--------|-----------|-------------| +| `os` | Bypasses Fs/Env primitives | `c.Fs()`, `core.Env()`, `core.DirFS()`, `Fs.TempDir()` | +| `os/exec` | Bypasses Process primitive | `c.Process().Run()` | +| `io` | Bypasses stream primitives | `core.ReadAll()`, `core.WriteAll()`, `core.CloseStream()` | +| `fmt` | Bypasses string/print primitives | `core.Println()`, `core.Sprintf()`, `core.Sprint()` | +| `errors` | Bypasses error primitive | `core.NewError()`, `core.E()`, `core.Is()`, `core.As()` | +| `log` | Bypasses logging | `core.Info()`, `core.Warn()`, `core.Error()`, `c.Log()` | +| `encoding/json` | Bypasses Core serialisation | `core.JSONMarshal()`, `core.JSONUnmarshal()` | +| `path/filepath` | Bypasses path security boundary | `core.Path()`, `core.JoinPath()`, `core.PathBase()` | +| `unsafe` | Bypasses Fs sandbox | `Fs.NewUnrestricted()` | +| `strings` | Bypasses string guardrails | `core.Contains()`, `core.Split()`, `core.Trim()`, etc. | + +**Rule:** If a diff introduces a disallowed import, it failed code review. The import list IS the quality gate. No subjective judgement needed — a weaker model can enforce this mechanically. + +### 10. Registration IS Capability, Entitlement IS Permission + +Two layers of permission, both declarative: + +``` +Registration = "this action EXISTS" → c.Action("process.run").Exists() +Entitlement = "this Core is ALLOWED" → c.Entitled("process.run").Allowed +``` + +A sandboxed Core has no `process.run` registered — the action doesn't exist. A SaaS Core has it registered but entitlement-gated — the action exists but the workspace may not be allowed to use it. + +**Rule:** Never check permissions with `if` statements in business logic. Register capabilities as Actions. Gate them with Entitlements. The framework enforces both — `Action.Run()` checks both before executing. + +## Applying AX to Existing Patterns + +### File Structure + +``` +# AX-native: path describes content +core/agent/ +├── cmd/core-agent/ # CLI entry point (minimal — just core.New + Run) +├── pkg/agentic/ # Agent orchestration (dispatch, prep, verify, scan) +├── pkg/brain/ # OpenBrain integration +├── pkg/lib/ # Embedded templates, personas, flows +├── pkg/messages/ # Typed IPC message definitions +├── pkg/monitor/ # Agent monitoring + notifications +├── pkg/setup/ # Workspace scaffolding + detection +└── claude/ # Claude Code plugin definitions + +# Not AX: generic names requiring README +src/ +├── lib/ +├── utils/ +└── helpers/ +``` + +### Error Handling + +```go +// AX-native: errors flow through Result, not call sites +func Register(c *core.Core) core.Result { + svc := &MyService{ServiceRuntime: core.NewServiceRuntime(c, MyOpts{})} + return core.Result{Value: svc, OK: true} +} + +// Not AX: errors dominate the code +func Register(c *core.Core) (*MyService, error) { + svc, err := NewMyService(c) + if err != nil { + return nil, fmt.Errorf("create service: %w", err) + } + return svc, nil +} +``` + +### Command Registration + +```go +// AX-native: extracted methods, testable without CLI +func (s *MyService) OnStartup(ctx context.Context) core.Result { + c := s.Core() + c.Command("issue/get", core.Command{Action: s.cmdIssueGet}) + c.Command("issue/list", core.Command{Action: s.cmdIssueList}) + c.Action("forge.issue.get", s.handleIssueGet) + return core.Result{OK: true} +} + +func (s *MyService) cmdIssueGet(opts core.Options) core.Result { + // testable business logic — no closure, no CLI dependency +} + +// Not AX: closures that can only be tested via CLI integration +c.Command("issue/get", core.Command{ + Action: func(opts core.Options) core.Result { + // 50 lines of untestable inline logic + }, +}) +``` + +### Process Execution + +```go +// AX-native: Core Process primitive, testable with mock handler +func (s *MyService) getGitLog(repoPath string) string { + r := s.Core().Process().RunIn(context.Background(), repoPath, "git", "log", "--oneline", "-20") + if !r.OK { return "" } + return core.Trim(r.Value.(string)) +} + +// Not AX: raw exec.Command — untestable, no entitlement check, path traversal risk +func (s *MyService) getGitLog(repoPath string) string { + cmd := exec.Command("git", "log", "--oneline", "-20") + cmd.Dir = repoPath // user-controlled path goes directly to OS + output, err := cmd.Output() + if err != nil { return "" } + return strings.TrimSpace(string(output)) +} +``` + +The AX-native version routes through `c.Process()` → named Action → entitlement check. The non-AX version passes user input directly to `os/exec` with no permission gate. + +### Permission Gating + +```go +// AX-native: entitlement checked by framework, not by business logic +c.Action("agentic.dispatch", func(ctx context.Context, opts core.Options) core.Result { + // Action.Run() already checked c.Entitled("agentic.dispatch") + // If we're here, we're allowed. Just do the work. + return dispatch(ctx, opts) +}) + +// Not AX: permission logic scattered through business code +func handleDispatch(ctx context.Context, opts core.Options) core.Result { + if !isAdmin(ctx) && !hasPlan(ctx, "pro") { + return core.Result{Value: core.E("dispatch", "upgrade required", nil), OK: false} + } + // duplicate permission check in every handler +} +``` + +## Compatibility + +AX conventions are valid, idiomatic Go/PHP/TS. They do not require language extensions, code generation, or non-standard tooling. An AX-designed codebase compiles, tests, and deploys with standard toolchains. + +The conventions diverge from community patterns (functional options, Must/For, etc.) but do not violate language specifications. This is a style choice, not a fork. + +## Adoption + +AX applies to all code in the Core ecosystem. core/go is fully migrated (v0.8.0). Consumer packages migrate via their RFCs. + +Priority for migrating a package: +1. **Lifecycle** — `OnStartup`/`OnShutdown` return `Result` +2. **Actions** — register capabilities as named Actions +3. **Imports** — replace all 10 disallowed imports (Principle 9) +4. **String ops** — `+` concat → `Concat()`, `path +` → `Path()` +5. **Test naming** — `TestFile_Function_{Good,Bad,Ugly}` +6. **Examples** — one `{source}_example_test.go` per source file +7. **Comments** — every exported function has usage example (Principle 2) + +## Verification + +An agent auditing AX compliance checks: + +```bash +# Disallowed imports (Principle 9) +grep -rn '"os"\|"os/exec"\|"io"\|"fmt"\|"errors"\|"log"\|"encoding/json"\|"path/filepath"\|"unsafe"\|"strings"' *.go \ + | grep -v _test.go + +# Test naming (Principle 7) +grep "^func Test" *_test.go | grep -v "Test[A-Z][a-z]*_.*_\(Good\|Bad\|Ugly\)" + +# String concat (should use Concat/Path) +grep -n '" + \| + "' *.go | grep -v _test.go | grep -v "//" + +# Untyped dispatch (should prefer named Actions) +grep "RegisterTask\|PERFORM\|type Task any" *.go +``` + +If any check produces output, the code needs migration. + +## References + +- `core/go/docs/RFC.md` — CoreGO API contract (21 sections, reference implementation) +- `core/go-process/docs/RFC.md` — Process consumer spec +- `core/agent/docs/RFC.md` — Agent consumer spec +- RFC-004 (Entitlements) — permission model ported to `c.Entitled()` +- RFC-021 (Core Platform Architecture) — 7-layer stack, provider model +- dAppServer unified path convention (2024) — path = route = command = test +- Go Proverbs, Rob Pike (2015) — AX provides an updated lens + +## Changelog + +- 2026-03-25: v0.8.0 alignment — all examples match implemented API. Added Principles 8 (RFC as Domain Load), 9 (Primitives as Quality Gates), 10 (Registration + Entitlement). Updated subsystem table (Process, API, Action, Task, Entitled, RegistryOf). Process examples use `c.Process()` not old `process.RunWithOptions`. Removed PERFORM references. +- 2026-03-19: Initial draft — 7 principles diff --git a/.core/reference/app.go b/.core/reference/app.go new file mode 100644 index 0000000..9fc1984 --- /dev/null +++ b/.core/reference/app.go @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Application identity for the Core framework. + +package core + +import ( + "os" + "path/filepath" +) + +// App holds the application identity and optional GUI runtime. +// +// app := core.App{}.New(core.NewOptions( +// core.Option{Key: "name", Value: "Core CLI"}, +// core.Option{Key: "version", Value: "1.0.0"}, +// )) +type App struct { + Name string + Version string + Description string + Filename string + Path string + Runtime any // GUI runtime (e.g., Wails App). Nil for CLI-only. +} + +// New creates an App from Options. +// +// app := core.App{}.New(core.NewOptions( +// core.Option{Key: "name", Value: "myapp"}, +// core.Option{Key: "version", Value: "1.0.0"}, +// )) +func (a App) New(opts Options) App { + if name := opts.String("name"); name != "" { + a.Name = name + } + if version := opts.String("version"); version != "" { + a.Version = version + } + if desc := opts.String("description"); desc != "" { + a.Description = desc + } + if filename := opts.String("filename"); filename != "" { + a.Filename = filename + } + return a +} + +// Find locates a program on PATH and returns a Result containing the App. +// Uses os.Stat to search PATH directories — no os/exec dependency. +// +// r := core.App{}.Find("node", "Node.js") +// if r.OK { app := r.Value.(*App) } +func (a App) Find(filename, name string) Result { + // If filename contains a separator, check it directly + if Contains(filename, string(os.PathSeparator)) { + abs, err := filepath.Abs(filename) + if err != nil { + return Result{err, false} + } + if isExecutable(abs) { + return Result{&App{Name: name, Filename: filename, Path: abs}, true} + } + return Result{E("app.Find", Concat(filename, " not found"), nil), false} + } + + // Search PATH + pathEnv := os.Getenv("PATH") + if pathEnv == "" { + return Result{E("app.Find", "PATH is empty", nil), false} + } + for _, dir := range Split(pathEnv, string(os.PathListSeparator)) { + candidate := filepath.Join(dir, filename) + if isExecutable(candidate) { + abs, err := filepath.Abs(candidate) + if err != nil { + continue + } + return Result{&App{Name: name, Filename: filename, Path: abs}, true} + } + } + return Result{E("app.Find", Concat(filename, " not found on PATH"), nil), false} +} + +// isExecutable checks if a path exists and is executable. +func isExecutable(path string) bool { + info, err := os.Stat(path) + if err != nil { + return false + } + // Regular file with at least one execute bit + return !info.IsDir() && info.Mode()&0111 != 0 +} diff --git a/.core/reference/array.go b/.core/reference/array.go new file mode 100644 index 0000000..ff085bb --- /dev/null +++ b/.core/reference/array.go @@ -0,0 +1,101 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Generic slice operations for the Core framework. +// Based on leaanthony/slicer, rewritten with Go 1.18+ generics. + +package core + +// Array is a typed slice with common operations. +type Array[T comparable] struct { + items []T +} + +// NewArray creates an empty Array. +func NewArray[T comparable](items ...T) *Array[T] { + return &Array[T]{items: items} +} + +// Add appends values. +func (s *Array[T]) Add(values ...T) { + s.items = append(s.items, values...) +} + +// AddUnique appends values only if not already present. +func (s *Array[T]) AddUnique(values ...T) { + for _, v := range values { + if !s.Contains(v) { + s.items = append(s.items, v) + } + } +} + +// Contains returns true if the value is in the slice. +func (s *Array[T]) Contains(val T) bool { + for _, v := range s.items { + if v == val { + return true + } + } + return false +} + +// Filter returns a new Array with elements matching the predicate. +func (s *Array[T]) Filter(fn func(T) bool) Result { + filtered := &Array[T]{} + for _, v := range s.items { + if fn(v) { + filtered.items = append(filtered.items, v) + } + } + return Result{filtered, true} +} + +// Each runs a function on every element. +func (s *Array[T]) Each(fn func(T)) { + for _, v := range s.items { + fn(v) + } +} + +// Remove removes the first occurrence of a value. +func (s *Array[T]) Remove(val T) { + for i, v := range s.items { + if v == val { + s.items = append(s.items[:i], s.items[i+1:]...) + return + } + } +} + +// Deduplicate removes duplicate values, preserving order. +func (s *Array[T]) Deduplicate() { + seen := make(map[T]struct{}) + result := make([]T, 0, len(s.items)) + for _, v := range s.items { + if _, exists := seen[v]; !exists { + seen[v] = struct{}{} + result = append(result, v) + } + } + s.items = result +} + +// Len returns the number of elements. +func (s *Array[T]) Len() int { + return len(s.items) +} + +// Clear removes all elements. +func (s *Array[T]) Clear() { + s.items = nil +} + +// AsSlice returns a copy of the underlying slice. +func (s *Array[T]) AsSlice() []T { + if s.items == nil { + return nil + } + out := make([]T, len(s.items)) + copy(out, s.items) + return out +} diff --git a/.core/reference/cli.go b/.core/reference/cli.go new file mode 100644 index 0000000..5e4b9f7 --- /dev/null +++ b/.core/reference/cli.go @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Cli is the CLI surface layer for the Core command tree. +// +// c := core.New(core.WithOption("name", "myapp")).Value.(*Core) +// c.Command("deploy", core.Command{Action: handler}) +// c.Cli().Run() +package core + +import ( + "io" + "os" +) + +// CliOptions holds configuration for the Cli service. +type CliOptions struct{} + +// Cli is the CLI surface for the Core command tree. +type Cli struct { + *ServiceRuntime[CliOptions] + output io.Writer + banner func(*Cli) string +} + +// Register creates a Cli service factory for core.WithService. +// +// core.New(core.WithService(core.CliRegister)) +func CliRegister(c *Core) Result { + cl := &Cli{output: os.Stdout} + cl.ServiceRuntime = NewServiceRuntime[CliOptions](c, CliOptions{}) + return c.RegisterService("cli", cl) +} + +// Print writes to the CLI output (defaults to os.Stdout). +// +// c.Cli().Print("hello %s", "world") +func (cl *Cli) Print(format string, args ...any) { + Print(cl.output, format, args...) +} + +// SetOutput sets the CLI output writer. +// +// c.Cli().SetOutput(os.Stderr) +func (cl *Cli) SetOutput(w io.Writer) { + cl.output = w +} + +// Run resolves os.Args to a command path and executes it. +// +// c.Cli().Run() +// c.Cli().Run("deploy", "to", "homelab") +func (cl *Cli) Run(args ...string) Result { + if len(args) == 0 { + args = os.Args[1:] + } + + clean := FilterArgs(args) + c := cl.Core() + + if c == nil || c.commands == nil { + if cl.banner != nil { + cl.Print(cl.banner(cl)) + } + return Result{} + } + + if c.commands.Len() == 0 { + if cl.banner != nil { + cl.Print(cl.banner(cl)) + } + return Result{} + } + + // Resolve command path from args + var cmd *Command + var remaining []string + + for i := len(clean); i > 0; i-- { + path := JoinPath(clean[:i]...) + if r := c.commands.Get(path); r.OK { + cmd = r.Value.(*Command) + remaining = clean[i:] + break + } + } + + if cmd == nil { + if cl.banner != nil { + cl.Print(cl.banner(cl)) + } + cl.PrintHelp() + return Result{} + } + + // Build options from remaining args + opts := NewOptions() + for _, arg := range remaining { + key, val, valid := ParseFlag(arg) + if valid { + if Contains(arg, "=") { + opts.Set(key, val) + } else { + opts.Set(key, true) + } + } else if !IsFlag(arg) { + opts.Set("_arg", arg) + } + } + + if cmd.Action != nil { + return cmd.Run(opts) + } + return Result{E("core.Cli.Run", Concat("command \"", cmd.Path, "\" is not executable"), nil), false} +} + +// PrintHelp prints available commands. +// +// c.Cli().PrintHelp() +func (cl *Cli) PrintHelp() { + c := cl.Core() + if c == nil || c.commands == nil { + return + } + + name := "" + if c.app != nil { + name = c.app.Name + } + if name != "" { + cl.Print("%s commands:", name) + } else { + cl.Print("Commands:") + } + + c.commands.Each(func(path string, cmd *Command) { + if cmd.Hidden || (cmd.Action == nil && !cmd.IsManaged()) { + return + } + tr := c.I18n().Translate(cmd.I18nKey()) + desc, _ := tr.Value.(string) + if desc == "" || desc == cmd.I18nKey() { + cl.Print(" %s", path) + } else { + cl.Print(" %-30s %s", path, desc) + } + }) +} + +// SetBanner sets the banner function. +// +// c.Cli().SetBanner(func(_ *core.Cli) string { return "My App v1.0" }) +func (cl *Cli) SetBanner(fn func(*Cli) string) { + cl.banner = fn +} + +// Banner returns the banner string. +func (cl *Cli) Banner() string { + if cl.banner != nil { + return cl.banner(cl) + } + c := cl.Core() + if c != nil && c.app != nil && c.app.Name != "" { + return c.app.Name + } + return "" +} diff --git a/.core/reference/command.go b/.core/reference/command.go new file mode 100644 index 0000000..660f866 --- /dev/null +++ b/.core/reference/command.go @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Command is a DTO representing an executable operation. +// Commands don't know if they're root, child, or nested — the tree +// structure comes from composition via path-based registration. +// +// Register a command: +// +// c.Command("deploy", func(opts core.Options) core.Result { +// return core.Result{"deployed", true} +// }) +// +// Register a nested command: +// +// c.Command("deploy/to/homelab", handler) +// +// Description is an i18n key — derived from path if omitted: +// +// "deploy" → "cmd.deploy.description" +// "deploy/to/homelab" → "cmd.deploy.to.homelab.description" +package core + + +// CommandAction is the function signature for command handlers. +// +// func(opts core.Options) core.Result +type CommandAction func(Options) Result + +// Command is the DTO for an executable operation. +// Commands are declarative — they carry enough information for multiple consumers: +// - core.Cli() runs the Action +// - core/cli adds rich help, completion, man pages +// - go-process wraps Managed commands with lifecycle (PID, health, signals) +// +// c.Command("serve", core.Command{ +// Action: handler, +// Managed: "process.daemon", // go-process provides start/stop/restart +// }) +type Command struct { + Name string + Description string // i18n key — derived from path if empty + Path string // "deploy/to/homelab" + Action CommandAction // business logic + Managed string // "" = one-shot, "process.daemon" = managed lifecycle + Flags Options // declared flags + Hidden bool + commands map[string]*Command // child commands (internal) +} + +// I18nKey returns the i18n key for this command's description. +// +// cmd with path "deploy/to/homelab" → "cmd.deploy.to.homelab.description" +func (cmd *Command) I18nKey() string { + if cmd.Description != "" { + return cmd.Description + } + path := cmd.Path + if path == "" { + path = cmd.Name + } + return Concat("cmd.", Replace(path, "/", "."), ".description") +} + +// Run executes the command's action with the given options. +// +// result := cmd.Run(core.NewOptions(core.Option{Key: "target", Value: "homelab"})) +func (cmd *Command) Run(opts Options) Result { + if cmd.Action == nil { + return Result{E("core.Command.Run", Concat("command \"", cmd.Path, "\" is not executable"), nil), false} + } + return cmd.Action(opts) +} + +// IsManaged returns true if this command has a managed lifecycle. +// +// if cmd.IsManaged() { /* go-process handles start/stop */ } +func (cmd *Command) IsManaged() bool { + return cmd.Managed != "" +} + +// --- Command Registry (on Core) --- + +// CommandRegistry holds the command tree. Embeds Registry[*Command] +// for thread-safe named storage with insertion order. +type CommandRegistry struct { + *Registry[*Command] +} + +// Command gets or registers a command by path. +// +// c.Command("deploy", Command{Action: handler}) +// r := c.Command("deploy") +func (c *Core) Command(path string, command ...Command) Result { + if len(command) == 0 { + return c.commands.Get(path) + } + + if path == "" || HasPrefix(path, "/") || HasSuffix(path, "/") || Contains(path, "//") { + return Result{E("core.Command", Concat("invalid command path: \"", path, "\""), nil), false} + } + + // Check for duplicate executable command + if r := c.commands.Get(path); r.OK { + existing := r.Value.(*Command) + if existing.Action != nil || existing.IsManaged() { + return Result{E("core.Command", Concat("command \"", path, "\" already registered"), nil), false} + } + } + + cmd := &command[0] + cmd.Name = pathName(path) + cmd.Path = path + if cmd.commands == nil { + cmd.commands = make(map[string]*Command) + } + + // Preserve existing subtree when overwriting a placeholder parent + if r := c.commands.Get(path); r.OK { + existing := r.Value.(*Command) + for k, v := range existing.commands { + if _, has := cmd.commands[k]; !has { + cmd.commands[k] = v + } + } + } + + c.commands.Set(path, cmd) + + // Build parent chain — "deploy/to/homelab" creates "deploy" and "deploy/to" if missing + parts := Split(path, "/") + for i := len(parts) - 1; i > 0; i-- { + parentPath := JoinPath(parts[:i]...) + if !c.commands.Has(parentPath) { + c.commands.Set(parentPath, &Command{ + Name: parts[i-1], + Path: parentPath, + commands: make(map[string]*Command), + }) + } + parent := c.commands.Get(parentPath).Value.(*Command) + parent.commands[parts[i]] = cmd + cmd = parent + } + + return Result{OK: true} +} + +// Commands returns all registered command paths in registration order. +// +// paths := c.Commands() +func (c *Core) Commands() []string { + if c.commands == nil { + return nil + } + return c.commands.Names() +} + +// pathName extracts the last segment of a path. +// "deploy/to/homelab" → "homelab" +func pathName(path string) string { + parts := Split(path, "/") + return parts[len(parts)-1] +} diff --git a/.core/reference/config.go b/.core/reference/config.go new file mode 100644 index 0000000..2f45f13 --- /dev/null +++ b/.core/reference/config.go @@ -0,0 +1,186 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Settings, feature flags, and typed configuration for the Core framework. + +package core + +import ( + "sync" +) + +// ConfigVar is a variable that can be set, unset, and queried for its state. +type ConfigVar[T any] struct { + val T + set bool +} + +// Get returns the current value. +// +// val := v.Get() +func (v *ConfigVar[T]) Get() T { return v.val } + +// Set sets the value and marks it as explicitly set. +// +// v.Set(true) +func (v *ConfigVar[T]) Set(val T) { v.val = val; v.set = true } + +// IsSet returns true if the value was explicitly set (distinguishes "set to false" from "never set"). +// +// if v.IsSet() { /* explicitly configured */ } +func (v *ConfigVar[T]) IsSet() bool { return v.set } + +// Unset resets to zero value and marks as not set. +// +// v.Unset() +// v.IsSet() // false +func (v *ConfigVar[T]) Unset() { + v.set = false + var zero T + v.val = zero +} + +// NewConfigVar creates a ConfigVar with an initial value marked as set. +// +// debug := core.NewConfigVar(true) +func NewConfigVar[T any](val T) ConfigVar[T] { + return ConfigVar[T]{val: val, set: true} +} + +// ConfigOptions holds configuration data. +type ConfigOptions struct { + Settings map[string]any + Features map[string]bool +} + +func (o *ConfigOptions) init() { + if o.Settings == nil { + o.Settings = make(map[string]any) + } + if o.Features == nil { + o.Features = make(map[string]bool) + } +} + +// Config holds configuration settings and feature flags. +type Config struct { + *ConfigOptions + mu sync.RWMutex +} + +// New initialises a Config with empty settings and features. +// +// cfg := (&core.Config{}).New() +func (e *Config) New() *Config { + e.ConfigOptions = &ConfigOptions{} + e.ConfigOptions.init() + return e +} + +// Set stores a configuration value by key. +func (e *Config) Set(key string, val any) { + e.mu.Lock() + if e.ConfigOptions == nil { + e.ConfigOptions = &ConfigOptions{} + } + e.ConfigOptions.init() + e.Settings[key] = val + e.mu.Unlock() +} + +// Get retrieves a configuration value by key. +func (e *Config) Get(key string) Result { + e.mu.RLock() + defer e.mu.RUnlock() + if e.ConfigOptions == nil || e.Settings == nil { + return Result{} + } + val, ok := e.Settings[key] + if !ok { + return Result{} + } + return Result{val, true} +} + +// String retrieves a string config value (empty string if missing). +// +// host := c.Config().String("database.host") +func (e *Config) String(key string) string { return ConfigGet[string](e, key) } + +// Int retrieves an int config value (0 if missing). +// +// port := c.Config().Int("database.port") +func (e *Config) Int(key string) int { return ConfigGet[int](e, key) } + +// Bool retrieves a bool config value (false if missing). +// +// debug := c.Config().Bool("debug") +func (e *Config) Bool(key string) bool { return ConfigGet[bool](e, key) } + +// ConfigGet retrieves a typed configuration value. +func ConfigGet[T any](e *Config, key string) T { + r := e.Get(key) + if !r.OK { + var zero T + return zero + } + typed, _ := r.Value.(T) + return typed +} + +// --- Feature Flags --- + +// Enable activates a feature flag. +// +// c.Config().Enable("dark-mode") +func (e *Config) Enable(feature string) { + e.mu.Lock() + if e.ConfigOptions == nil { + e.ConfigOptions = &ConfigOptions{} + } + e.ConfigOptions.init() + e.Features[feature] = true + e.mu.Unlock() +} + +// Disable deactivates a feature flag. +// +// c.Config().Disable("dark-mode") +func (e *Config) Disable(feature string) { + e.mu.Lock() + if e.ConfigOptions == nil { + e.ConfigOptions = &ConfigOptions{} + } + e.ConfigOptions.init() + e.Features[feature] = false + e.mu.Unlock() +} + +// Enabled returns true if a feature flag is active. +// +// if c.Config().Enabled("dark-mode") { ... } +func (e *Config) Enabled(feature string) bool { + e.mu.RLock() + defer e.mu.RUnlock() + if e.ConfigOptions == nil || e.Features == nil { + return false + } + return e.Features[feature] +} + +// EnabledFeatures returns all active feature flag names. +// +// features := c.Config().EnabledFeatures() +func (e *Config) EnabledFeatures() []string { + e.mu.RLock() + defer e.mu.RUnlock() + if e.ConfigOptions == nil || e.Features == nil { + return nil + } + var result []string + for k, v := range e.Features { + if v { + result = append(result, k) + } + } + return result +} diff --git a/.core/reference/contract.go b/.core/reference/contract.go new file mode 100644 index 0000000..8718a90 --- /dev/null +++ b/.core/reference/contract.go @@ -0,0 +1,226 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Contracts, options, and type definitions for the Core framework. + +package core + +import ( + "context" + "reflect" + "sync" +) + +// Message is the type for IPC broadcasts (fire-and-forget). +type Message any + +// Query is the type for read-only IPC requests. +type Query any + +// QueryHandler handles Query requests. Returns Result{Value, OK}. +type QueryHandler func(*Core, Query) Result + +// Startable is implemented by services that need startup initialisation. +// +// func (s *MyService) OnStartup(ctx context.Context) core.Result { +// return core.Result{OK: true} +// } +type Startable interface { + OnStartup(ctx context.Context) Result +} + +// Stoppable is implemented by services that need shutdown cleanup. +// +// func (s *MyService) OnShutdown(ctx context.Context) core.Result { +// return core.Result{OK: true} +// } +type Stoppable interface { + OnShutdown(ctx context.Context) Result +} + +// --- Action Messages --- + +type ActionServiceStartup struct{} +type ActionServiceShutdown struct{} + +type ActionTaskStarted struct { + TaskIdentifier string + Action string + Options Options +} + +type ActionTaskProgress struct { + TaskIdentifier string + Action string + Progress float64 + Message string +} + +type ActionTaskCompleted struct { + TaskIdentifier string + Action string + Result Result +} + +// --- Constructor --- + +// CoreOption is a functional option applied during Core construction. +// Returns Result — if !OK, New() stops and returns the error. +// +// core.New( +// core.WithService(agentic.Register), +// core.WithService(monitor.Register), +// core.WithServiceLock(), +// ) +type CoreOption func(*Core) Result + +// New initialises a Core instance by applying options in order. +// Services registered here form the application conclave — they share +// IPC access and participate in the lifecycle (ServiceStartup/ServiceShutdown). +// +// c := core.New( +// core.WithOption("name", "myapp"), +// core.WithService(auth.Register), +// core.WithServiceLock(), +// ) +// c.Run() +func New(opts ...CoreOption) *Core { + c := &Core{ + app: &App{}, + data: &Data{Registry: NewRegistry[*Embed]()}, + drive: &Drive{Registry: NewRegistry[*DriveHandle]()}, + fs: (&Fs{}).New("/"), + config: (&Config{}).New(), + error: &ErrorPanic{}, + log: &ErrorLog{}, + lock: &Lock{locks: NewRegistry[*sync.RWMutex]()}, + ipc: &Ipc{actions: NewRegistry[*Action](), tasks: NewRegistry[*Task]()}, + info: systemInfo, + i18n: &I18n{}, + api: &API{protocols: NewRegistry[StreamFactory]()}, + services: &ServiceRegistry{Registry: NewRegistry[*Service]()}, + commands: &CommandRegistry{Registry: NewRegistry[*Command]()}, + entitlementChecker: defaultChecker, + } + c.context, c.cancel = context.WithCancel(context.Background()) + c.api.core = c + + // Core services + CliRegister(c) + + for _, opt := range opts { + if r := opt(c); !r.OK { + Error("core.New failed", "err", r.Value) + break + } + } + + // Apply service lock after all opts — v0.3.3 parity + c.LockApply() + + return c +} + +// WithOptions applies key-value configuration to Core. +// +// core.WithOptions(core.NewOptions(core.Option{Key: "name", Value: "myapp"})) +func WithOptions(opts Options) CoreOption { + return func(c *Core) Result { + c.options = &opts + if name := opts.String("name"); name != "" { + c.app.Name = name + } + return Result{OK: true} + } +} + +// WithService registers a service via its factory function. +// If the factory returns a non-nil Value, WithService auto-discovers the +// service name from the factory's package path (last path segment, lowercase, +// with any "_test" suffix stripped) and calls RegisterService on the instance. +// IPC handler auto-registration is handled by RegisterService. +// +// If the factory returns nil Value (it registered itself), WithService +// returns success without a second registration. +// +// core.WithService(agentic.Register) +// core.WithService(display.Register(nil)) +func WithService(factory func(*Core) Result) CoreOption { + return func(c *Core) Result { + r := factory(c) + if !r.OK { + return r + } + if r.Value == nil { + // Factory self-registered — nothing more to do. + return Result{OK: true} + } + // Auto-discover the service name from the instance's package path. + instance := r.Value + typeOf := reflect.TypeOf(instance) + if typeOf.Kind() == reflect.Ptr { + typeOf = typeOf.Elem() + } + pkgPath := typeOf.PkgPath() + parts := Split(pkgPath, "/") + name := Lower(parts[len(parts)-1]) + if name == "" { + return Result{E("core.WithService", Sprintf("service name could not be discovered for type %T", instance), nil), false} + } + + // RegisterService handles Startable/Stoppable/HandleIPCEvents discovery + return c.RegisterService(name, instance) + } +} + +// WithName registers a service with an explicit name (no reflect discovery). +// +// core.WithName("ws", func(c *Core) Result { +// return Result{Value: hub, OK: true} +// }) +func WithName(name string, factory func(*Core) Result) CoreOption { + return func(c *Core) Result { + r := factory(c) + if !r.OK { + return r + } + if r.Value == nil { + return Result{E("core.WithName", Sprintf("failed to create service %q", name), nil), false} + } + return c.RegisterService(name, r.Value) + } +} + +// WithOption is a convenience for setting a single key-value option. +// +// core.New( +// core.WithOption("name", "myapp"), +// core.WithOption("port", 8080), +// ) +func WithOption(key string, value any) CoreOption { + return func(c *Core) Result { + if c.options == nil { + opts := NewOptions() + c.options = &opts + } + c.options.Set(key, value) + if key == "name" { + if s, ok := value.(string); ok { + c.app.Name = s + } + } + return Result{OK: true} + } +} + +// WithServiceLock prevents further service registration after construction. +// +// core.New( +// core.WithService(auth.Register), +// core.WithServiceLock(), +// ) +func WithServiceLock() CoreOption { + return func(c *Core) Result { + c.LockEnable() + return Result{OK: true} + } +} diff --git a/.core/reference/core.go b/.core/reference/core.go new file mode 100644 index 0000000..21f13c1 --- /dev/null +++ b/.core/reference/core.go @@ -0,0 +1,239 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Package core is a dependency injection and service lifecycle framework for Go. +// This file defines the Core struct, accessors, and IPC/error wrappers. + +package core + +import ( + "context" + "os" + "sync" + "sync/atomic" +) + +// --- Core Struct --- + +// Core is the central application object that manages services, assets, and communication. +type Core struct { + options *Options // c.Options() — Input configuration used to create this Core + app *App // c.App() — Application identity + optional GUI runtime + data *Data // c.Data() — Embedded/stored content from packages + drive *Drive // c.Drive() — Resource handle registry (transports) + fs *Fs // c.Fs() — Local filesystem I/O (sandboxable) + config *Config // c.Config() — Configuration, settings, feature flags + error *ErrorPanic // c.Error() — Panic recovery and crash reporting + log *ErrorLog // c.Log() — Structured logging + error wrapping + // cli accessed via ServiceFor[*Cli](c, "cli") + commands *CommandRegistry // c.Command("path") — Command tree + services *ServiceRegistry // c.Service("name") — Service registry + lock *Lock // c.Lock("name") — Named mutexes + ipc *Ipc // c.IPC() — Message bus for IPC + api *API // c.API() — Remote streams + info *SysInfo // c.Env("key") — Read-only system/environment information + i18n *I18n // c.I18n() — Internationalisation and locale collection + + entitlementChecker EntitlementChecker // default: everything permitted + usageRecorder UsageRecorder // default: nil (no-op) + + context context.Context + cancel context.CancelFunc + taskIDCounter atomic.Uint64 + waitGroup sync.WaitGroup + shutdown atomic.Bool +} + +// --- Accessors --- + +// Options returns the input configuration passed to core.New(). +// +// opts := c.Options() +// name := opts.String("name") +func (c *Core) Options() *Options { return c.options } + +// App returns application identity metadata. +// +// c.App().Name // "my-app" +// c.App().Version // "1.0.0" +func (c *Core) App() *App { return c.app } + +// Data returns the embedded asset registry (Registry[*Embed]). +// +// r := c.Data().ReadString("prompts/coding.md") +func (c *Core) Data() *Data { return c.data } + +// Drive returns the transport handle registry (Registry[*DriveHandle]). +// +// r := c.Drive().Get("forge") +func (c *Core) Drive() *Drive { return c.drive } + +// Fs returns the sandboxed filesystem. +// +// r := c.Fs().Read("/path/to/file") +// c.Fs().WriteAtomic("/status.json", data) +func (c *Core) Fs() *Fs { return c.fs } + +// Config returns runtime settings and feature flags. +// +// host := c.Config().String("database.host") +// c.Config().Enable("dark-mode") +func (c *Core) Config() *Config { return c.config } + +// Error returns the panic recovery subsystem. +// +// c.Error().Recover() +func (c *Core) Error() *ErrorPanic { return c.error } + +// Log returns the structured logging subsystem. +// +// c.Log().Info("started", "port", 8080) +func (c *Core) Log() *ErrorLog { return c.log } + +// Cli returns the CLI command framework (registered as service "cli"). +// +// c.Cli().Run("deploy", "to", "homelab") +func (c *Core) Cli() *Cli { + cl, _ := ServiceFor[*Cli](c, "cli") + return cl +} + +// IPC returns the message bus internals. +// +// c.IPC() +func (c *Core) IPC() *Ipc { return c.ipc } + +// I18n returns the internationalisation subsystem. +// +// tr := c.I18n().Translate("cmd.deploy.description") +func (c *Core) I18n() *I18n { return c.i18n } + +// Env returns an environment variable by key (cached at init, falls back to os.Getenv). +// +// home := c.Env("DIR_HOME") +// token := c.Env("FORGE_TOKEN") +func (c *Core) Env(key string) string { return Env(key) } + +// Context returns Core's lifecycle context (cancelled on shutdown). +// +// ctx := c.Context() +func (c *Core) Context() context.Context { return c.context } + +// Core returns self — satisfies the ServiceRuntime interface. +// +// c := s.Core() +func (c *Core) Core() *Core { return c } + +// --- Lifecycle --- + +// RunE starts all services, runs the CLI, then shuts down. +// Returns an error instead of calling os.Exit — let main() handle the exit. +// ServiceShutdown is always called via defer, even on startup failure or panic. +// +// if err := c.RunE(); err != nil { +// os.Exit(1) +// } +func (c *Core) RunE() error { + defer c.ServiceShutdown(context.Background()) + + r := c.ServiceStartup(c.context, nil) + if !r.OK { + if err, ok := r.Value.(error); ok { + return err + } + return E("core.Run", "startup failed", nil) + } + + if cli := c.Cli(); cli != nil { + r = cli.Run() + } + + if !r.OK { + if err, ok := r.Value.(error); ok { + return err + } + } + return nil +} + +// Run starts all services, runs the CLI, then shuts down. +// Calls os.Exit(1) on failure. For error handling use RunE(). +// +// c := core.New(core.WithService(myService.Register)) +// c.Run() +func (c *Core) Run() { + if err := c.RunE(); err != nil { + Error(err.Error()) + os.Exit(1) + } +} + +// --- IPC (uppercase aliases) --- + +// ACTION broadcasts a message to all registered handlers (fire-and-forget). +// Each handler is wrapped in panic recovery. All handlers fire regardless. +// +// c.ACTION(messages.AgentCompleted{Agent: "codex", Status: "completed"}) +func (c *Core) ACTION(msg Message) Result { return c.broadcast(msg) } + +// QUERY sends a request — first handler to return OK wins. +// +// r := c.QUERY(MyQuery{Name: "brain"}) +func (c *Core) QUERY(q Query) Result { return c.Query(q) } + +// QUERYALL sends a request — collects all OK responses. +// +// r := c.QUERYALL(countQuery{}) +// results := r.Value.([]any) +func (c *Core) QUERYALL(q Query) Result { return c.QueryAll(q) } + +// --- Error+Log --- + +// LogError logs an error and returns the Result from ErrorLog. +func (c *Core) LogError(err error, op, msg string) Result { + return c.log.Error(err, op, msg) +} + +// LogWarn logs a warning and returns the Result from ErrorLog. +func (c *Core) LogWarn(err error, op, msg string) Result { + return c.log.Warn(err, op, msg) +} + +// Must logs and panics if err is not nil. +func (c *Core) Must(err error, op, msg string) { + c.log.Must(err, op, msg) +} + +// --- Registry Accessor --- + +// RegistryOf returns a named registry for cross-cutting queries. +// Known registries: "services", "commands", "actions". +// +// c.RegistryOf("services").Names() // all service names +// c.RegistryOf("actions").List("process.*") // process capabilities +// c.RegistryOf("commands").Len() // command count +func (c *Core) RegistryOf(name string) *Registry[any] { + // Bridge typed registries to untyped access for cross-cutting queries. + // Each registry is wrapped in a read-only proxy. + switch name { + case "services": + return registryProxy(c.services.Registry) + case "commands": + return registryProxy(c.commands.Registry) + case "actions": + return registryProxy(c.ipc.actions) + default: + return NewRegistry[any]() // empty registry for unknown names + } +} + +// registryProxy creates a read-only any-typed view of a typed registry. +// Copies current state — not a live view (avoids type parameter leaking). +func registryProxy[T any](src *Registry[T]) *Registry[any] { + proxy := NewRegistry[any]() + src.Each(func(name string, item T) { + proxy.Set(name, item) + }) + return proxy +} + +// --- Global Instance --- diff --git a/.core/reference/data.go b/.core/reference/data.go new file mode 100644 index 0000000..460277c --- /dev/null +++ b/.core/reference/data.go @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: EUPL-1.2 + +// Data is the embedded/stored content system for core packages. +// Packages mount their embedded content here and other packages +// read from it by path. +// +// Mount a package's assets: +// +// c.Data().New(core.NewOptions( +// core.Option{Key: "name", Value: "brain"}, +// core.Option{Key: "source", Value: brainFS}, +// core.Option{Key: "path", Value: "prompts"}, +// )) +// +// Read from any mounted path: +// +// content := c.Data().ReadString("brain/coding.md") +// entries := c.Data().List("agent/flow") +// +// Extract a template directory: +// +// c.Data().Extract("agent/workspace/default", "/tmp/ws", data) +package core + +import ( + "io/fs" + "path/filepath" +) + +// Data manages mounted embedded filesystems from core packages. +// Embeds Registry[*Embed] for thread-safe named storage. +type Data struct { + *Registry[*Embed] +} + +// New registers an embedded filesystem under a named prefix. +// +// c.Data().New(core.NewOptions( +// core.Option{Key: "name", Value: "brain"}, +// core.Option{Key: "source", Value: brainFS}, +// core.Option{Key: "path", Value: "prompts"}, +// )) +func (d *Data) New(opts Options) Result { + name := opts.String("name") + if name == "" { + return Result{} + } + + r := opts.Get("source") + if !r.OK { + return r + } + + fsys, ok := r.Value.(fs.FS) + if !ok { + return Result{E("data.New", "source is not fs.FS", nil), false} + } + + path := opts.String("path") + if path == "" { + path = "." + } + + mr := Mount(fsys, path) + if !mr.OK { + return mr + } + + emb := mr.Value.(*Embed) + d.Set(name, emb) + return Result{emb, true} +} + +// resolve splits a path like "brain/coding.md" into mount name + relative path. +func (d *Data) resolve(path string) (*Embed, string) { + parts := SplitN(path, "/", 2) + if len(parts) < 2 { + return nil, "" + } + r := d.Get(parts[0]) + if !r.OK { + return nil, "" + } + return r.Value.(*Embed), parts[1] +} + +// ReadFile reads a file by full path. +// +// r := c.Data().ReadFile("brain/prompts/coding.md") +// if r.OK { data := r.Value.([]byte) } +func (d *Data) ReadFile(path string) Result { + emb, rel := d.resolve(path) + if emb == nil { + return Result{} + } + return emb.ReadFile(rel) +} + +// ReadString reads a file as a string. +// +// r := c.Data().ReadString("agent/flow/deploy/to/homelab.yaml") +// if r.OK { content := r.Value.(string) } +func (d *Data) ReadString(path string) Result { + r := d.ReadFile(path) + if !r.OK { + return r + } + return Result{string(r.Value.([]byte)), true} +} + +// List returns directory entries at a path. +// +// r := c.Data().List("agent/persona/code") +// if r.OK { entries := r.Value.([]fs.DirEntry) } +func (d *Data) List(path string) Result { + emb, rel := d.resolve(path) + if emb == nil { + return Result{} + } + r := emb.ReadDir(rel) + if !r.OK { + return r + } + return Result{r.Value, true} +} + +// ListNames returns filenames (without extensions) at a path. +// +// r := c.Data().ListNames("agent/flow") +// if r.OK { names := r.Value.([]string) } +func (d *Data) ListNames(path string) Result { + r := d.List(path) + if !r.OK { + return r + } + entries := r.Value.([]fs.DirEntry) + var names []string + for _, e := range entries { + name := e.Name() + if !e.IsDir() { + name = TrimSuffix(name, filepath.Ext(name)) + } + names = append(names, name) + } + return Result{names, true} +} + +// Extract copies a template directory to targetDir. +// +// r := c.Data().Extract("agent/workspace/default", "/tmp/ws", templateData) +func (d *Data) Extract(path, targetDir string, templateData any) Result { + emb, rel := d.resolve(path) + if emb == nil { + return Result{} + } + r := emb.Sub(rel) + if !r.OK { + return r + } + return Extract(r.Value.(*Embed).FS(), targetDir, templateData) +} + +// Mounts returns the names of all mounted content in registration order. +// +// names := c.Data().Mounts() +func (d *Data) Mounts() []string { + return d.Names() +} diff --git a/.core/reference/docs/RFC.md b/.core/reference/docs/RFC.md new file mode 100644 index 0000000..457ef22 --- /dev/null +++ b/.core/reference/docs/RFC.md @@ -0,0 +1,439 @@ +# core/agent API Contract — RFC Specification + +> `dappco.re/go/core/agent` — Agentic dispatch, orchestration, and pipeline management. +> An agent should be able to understand core/agent's architecture from this document alone. + +**Status:** v0.8.0+alpha.1 +**Module:** `dappco.re/go/core/agent` +**Depends on:** core/go v0.8.0, go-process v0.8.0 + +--- + +## 1. Purpose + +core/agent dispatches AI agents (Claude, Codex, Gemini) to work on tasks in sandboxed git worktrees, monitors their progress, verifies output, and manages the merge pipeline. + +core/go provides the primitives. core/agent composes them. + +### File Layout + +``` +cmd/core-agent/main.go — entry point: core.New + Run +pkg/agentic/ — orchestration (dispatch, prep, verify, scan, commands) +pkg/agentic/actions.go — named Action handlers (ctx, Options) → Result +pkg/agentic/proc.go — process helpers via s.Core().Process() +pkg/agentic/handlers.go — IPC completion pipeline handlers +pkg/agentic/status.go — workspace status (WriteAtomic + JSONMarshalString) +pkg/agentic/paths.go — paths, fs (NewUnrestricted), helpers +pkg/brain/ — OpenBrain (recall, remember, search) +pkg/lib/ — embedded templates, personas, flows, plans +pkg/messages/ — typed message structs for IPC broadcast +pkg/monitor/ — agent monitoring via IPC (ServiceRuntime) +pkg/setup/ — workspace detection + scaffolding (Service) +claude/ — Claude Code plugin definitions +docs/ — RFC, plans, architecture +``` + +--- + +## 2. Service Registration + +All services use `ServiceRuntime[T]` — no raw `core *core.Core` fields. + +```go +func Register(c *core.Core) core.Result { + prep := NewPrep() + prep.ServiceRuntime = core.NewServiceRuntime(c, AgentOptions{}) + + cfg := prep.loadAgentsConfig() + c.Config().Set("agents.concurrency", cfg.Concurrency) + c.Config().Set("agents.rates", cfg.Rates) + + RegisterHandlers(c, prep) + return core.Result{Value: prep, OK: true} +} + +// In main: +c := core.New( + core.WithService(process.Register), + core.WithService(agentic.Register), + core.WithService(brain.Register), + core.WithService(monitor.Register), + core.WithService(mcp.Register), +) +c.Run() +``` + +--- + +## 3. Named Actions — The Capability Map + +All capabilities registered as named Actions during OnStartup. Inspectable, composable, gatable by Entitlements. + +```go +func (s *PrepSubsystem) OnStartup(ctx context.Context) core.Result { + c := s.Core() + + // Dispatch & workspace + c.Action("agentic.dispatch", s.handleDispatch) + c.Action("agentic.prep", s.handlePrep) + c.Action("agentic.status", s.handleStatus) + c.Action("agentic.resume", s.handleResume) + c.Action("agentic.scan", s.handleScan) + c.Action("agentic.watch", s.handleWatch) + + // Pipeline + c.Action("agentic.qa", s.handleQA) + c.Action("agentic.auto-pr", s.handleAutoPR) + c.Action("agentic.verify", s.handleVerify) + c.Action("agentic.ingest", s.handleIngest) + c.Action("agentic.poke", s.handlePoke) + c.Action("agentic.mirror", s.handleMirror) + + // Forge + c.Action("agentic.issue.get", s.handleIssueGet) + c.Action("agentic.issue.list", s.handleIssueList) + c.Action("agentic.issue.create", s.handleIssueCreate) + c.Action("agentic.pr.get", s.handlePRGet) + c.Action("agentic.pr.list", s.handlePRList) + c.Action("agentic.pr.merge", s.handlePRMerge) + + // Review & Epic + c.Action("agentic.review-queue", s.handleReviewQueue) + c.Action("agentic.epic", s.handleEpic) + + // Completion pipeline — Task composition + c.Task("agent.completion", core.Task{ + Description: "QA → PR → Verify → Merge", + Steps: []core.Step{ + {Action: "agentic.qa"}, + {Action: "agentic.auto-pr"}, + {Action: "agentic.verify"}, + {Action: "agentic.ingest", Async: true}, + {Action: "agentic.poke", Async: true}, + }, + }) + + s.StartRunner() + s.registerCommands(ctx) + s.registerWorkspaceCommands() + s.registerForgeCommands() + return core.Result{OK: true} +} +``` + +--- + +## 4. Completion Pipeline + +When an agent completes, the IPC handler chain fires. Registered in `RegisterHandlers()`: + +``` +AgentCompleted → QA handler → QAResult +QAResult{Passed} → PR handler → PRCreated +PRCreated → Verify handler → PRMerged | PRNeedsReview +AgentCompleted → Ingest handler (findings → issues) +AgentCompleted → Poke handler (drain queue) +``` + +All handlers use `c.ACTION(messages.X{})` — no ChannelNotifier, no callbacks. + +--- + +## 5. Process Execution + +All commands via `s.Core().Process()`. Returns `core.Result` — Value is always a string. + +```go +func (s *PrepSubsystem) runCmd(ctx context.Context, dir, command string, args ...string) core.Result { + return s.Core().Process().RunIn(ctx, dir, command, args...) +} + +func (s *PrepSubsystem) runCmdOK(ctx context.Context, dir, command string, args ...string) bool { + return s.runCmd(ctx, dir, command, args...).OK +} + +func (s *PrepSubsystem) gitCmd(ctx context.Context, dir string, args ...string) core.Result { + return s.runCmd(ctx, dir, "git", args...) +} + +func (s *PrepSubsystem) gitOutput(ctx context.Context, dir string, args ...string) string { + r := s.gitCmd(ctx, dir, args...) + if !r.OK { return "" } + return core.Trim(r.Value.(string)) +} +``` + +go-process is fully Result-native. `Start`, `Run`, `StartWithOptions`, `RunWithOptions` all return `core.Result`. Value is `*Process` for Start, `string` for Run. OK=true guarantees the type. + +--- + +## 6. Status Management + +Workspace status uses `WriteAtomic` + `JSONMarshalString` for safe concurrent access: + +```go +func writeStatus(wsDir string, status *WorkspaceStatus) error { + status.UpdatedAt = time.Now() + statusPath := core.JoinPath(wsDir, "status.json") + if r := fs.WriteAtomic(statusPath, core.JSONMarshalString(status)); !r.OK { + err, _ := r.Value.(error) + return core.E("writeStatus", "failed to write status", err) + } + return nil +} +``` + +--- + +## 7. Filesystem + +No `unsafe.Pointer`. Package-level unrestricted Fs via Core primitive: + +```go +var fs = (&core.Fs{}).NewUnrestricted() +``` + +--- + +## 8. IPC Messages + +All inter-service communication via typed messages in `pkg/messages/`: + +```go +// Agent lifecycle +messages.AgentStarted{Agent, Repo, Workspace} +messages.AgentCompleted{Agent, Repo, Workspace, Status} + +// Pipeline +messages.QAResult{Workspace, Repo, Passed} +messages.PRCreated{Repo, Branch, PRURL, PRNum} +messages.PRMerged{Repo, PRURL, PRNum} +messages.PRNeedsReview{Repo, PRURL, PRNum, Reason} + +// Queue +messages.QueueDrained{Completed} +messages.PokeQueue{} + +// Monitor +messages.HarvestComplete{Repo, Branch, Files} +messages.HarvestRejected{Repo, Branch, Reason} +messages.InboxMessage{New, Total} +``` + +--- + +## 9. Monitor + +Embeds `*core.ServiceRuntime[MonitorOptions]`. All notifications via `m.Core().ACTION(messages.X{})` — no ChannelNotifier interface. Git operations via `m.Core().Process()`. + +```go +func Register(c *core.Core) core.Result { + mon := New() + mon.ServiceRuntime = core.NewServiceRuntime(c, MonitorOptions{}) + + c.RegisterAction(func(c *core.Core, msg core.Message) core.Result { + switch ev := msg.(type) { + case messages.AgentCompleted: + mon.handleAgentCompleted(ev) + case messages.AgentStarted: + mon.handleAgentStarted(ev) + } + return core.Result{OK: true} + }) + + return core.Result{Value: mon, OK: true} +} +``` + +--- + +## 10. Setup + +Service with `*core.ServiceRuntime[SetupOptions]`. Detects project type, generates configs, scaffolds workspaces. + +```go +func Register(c *core.Core) core.Result { + svc := &Service{ + ServiceRuntime: core.NewServiceRuntime(c, SetupOptions{}), + } + return core.Result{Value: svc, OK: true} +} +``` + +--- + +## 11. Entitlements + +Actions are gated by `c.Entitled()` — checked automatically in `Action.Run()`. + +```go +func (s *PrepSubsystem) handleDispatch(ctx context.Context, opts core.Options) core.Result { + e := s.Core().Entitled("agentic.concurrency", 1) + if !e.Allowed { + return core.Result{Value: core.E("dispatch", e.Reason, nil), OK: false} + } + // ... dispatch agent ... + s.Core().RecordUsage("agentic.dispatch") + return core.Result{OK: true} +} +``` + +--- + +## 12. MCP — Action Aggregator + +MCP auto-exposes all registered Actions as tools via `c.Actions()`. Register an Action → it appears as an MCP tool. The API stream primitive (`c.API()`) handles transport. + +--- + +## 13. Remote Dispatch + +Transparent local/remote via `host:action` syntax: + +```go +r := c.RemoteAction("agentic.status", ctx, opts) // local +r := c.RemoteAction("charon:agentic.dispatch", ctx, opts) // remote +r := c.RemoteAction("snider.lthn:brain.recall", ctx, opts) // web3 +``` + +--- + +## 14. Quality Gates + +```bash +# No disallowed imports (source files only) +grep -rn '"os"\|"os/exec"\|"io"\|"fmt"\|"errors"\|"log"\|"encoding/json"\|"path/filepath"\|"unsafe"\|"strings"' *.go **/*.go \ + | grep -v _test.go + +# Test naming: TestFile_Function_{Good,Bad,Ugly} +grep -rn "^func Test" *_test.go **/*_test.go \ + | grep -v "Test[A-Z][a-z]*_.*_\(Good\|Bad\|Ugly\)" +``` + +--- + +## 15. Validation and IDs + +```go +if r := core.ValidateName(input.Repo); !r.OK { return r } +safe := core.SanitisePath(userInput) +id := core.ID() // "id-42-a3f2b1" +``` + +--- + +## 16. JSON Serialisation + +All JSON via Core primitives. No `encoding/json` import. + +```go +data := core.JSONMarshalString(status) +core.JSONUnmarshalString(jsonStr, &result) +``` + +--- + +## 17. Configuration + +```go +c.Config().Set("agents.concurrency", 5) +c.Config().String("workspace.root") +c.Config().Int("agents.concurrency") +c.Config().Enable("auto-merge") +if c.Config().Enabled("auto-merge") { ... } +``` + +--- + +## 18. Registry + +Use `Registry[T]` for any named collection. No `map[string]*T + sync.Mutex`. + +```go +workspaces := core.NewRegistry[*WorkspaceStatus]() +workspaces.Set(wsDir, status) +workspaces.Get(wsDir) +workspaces.Each(func(dir string, st *WorkspaceStatus) { ... }) +workspaces.Names() // insertion order +c.RegistryOf("actions").List("agentic.*") +``` + +--- + +## 19. String Operations + +No `fmt`, no `strings`, no `+` concat. Core provides everything: + +```go +core.Println(value) // not fmt.Println +core.Sprintf("port: %d", port) // not fmt.Sprintf +core.Concat("hello ", name) // not "hello " + name +core.Path(dir, "status.json") // not dir + "/status.json" +core.Contains(s, "prefix") // not strings.Contains +core.Split(s, "/") // not strings.Split +core.Trim(s) // not strings.TrimSpace +``` + +--- + +## 20. Error Handling and Logging + +All errors via `core.E()`. All logging via Core. No `fmt`, `errors`, or `log` imports. + +```go +return core.E("dispatch.prep", "workspace not found", nil) +return core.E("dispatch.prep", core.Concat("repo ", repo, " invalid"), cause) +core.Info("agent dispatched", "repo", repo, "agent", agent) +core.Error("dispatch failed", "err", err) +core.Security("entitlement.denied", "action", action, "reason", reason) +``` + +--- + +## 21. Stream Helpers and Data + +```go +r := c.Data().ReadString("prompts/coding.md") +c.Data().List("templates/") +c.Drive().New(core.NewOptions( + core.Option{Key: "name", Value: "charon"}, + core.Option{Key: "transport", Value: "http://10.69.69.165:9101"}, +)) +``` + +--- + +## 22. Comments (AX Principle 2) + +Every exported function MUST have a usage-example comment: + +```go +// gitCmd runs a git command in a directory. +// +// r := s.gitCmd(ctx, "/repo", "log", "--oneline") +func (s *PrepSubsystem) gitCmd(ctx context.Context, dir string, args ...string) core.Result { +``` + +--- + +## 23. Test Strategy (AX Principle 7) + +`TestFile_Function_{Good,Bad,Ugly}` — 100% naming compliance target. + +--- + +## Consumer RFCs + +| Package | RFC | Role | +|---------|-----|------| +| core/go | `core/go/docs/RFC.md` | Primitives — all 21 sections | +| go-process | `core/go-process/docs/RFC.md` | Process Action handlers (Result-native) | + +--- + +## Changelog + +- 2026-03-26: WIP — net/http consolidated to transport.go (ONE file). net/url + io/fs eliminated. RFC-025 updated with 3 new quality gates (net/http, net/url, io/fs). 1:1 test + example test coverage. Array[T].Deduplicate replaces custom helpers. Remaining: remove dead `client` field from test literals, brain/provider.go Gin handler. +- 2026-03-25: Quality gates pass. Zero disallowed imports (all 10). encoding/json→Core JSON. path/filepath→Core Path. os→Core Env/Fs. io→Core ReadAll/WriteAll. go-process fully Result-native. ServiceRuntime on all subsystems. 22 named Actions + Task pipeline. ChannelNotifier→IPC. Reference docs synced. +- 2026-03-25: Initial spec — written with full core/go v0.8.0 domain context. diff --git a/.core/reference/docs/commands.md b/.core/reference/docs/commands.md new file mode 100644 index 0000000..46e2022 --- /dev/null +++ b/.core/reference/docs/commands.md @@ -0,0 +1,177 @@ +--- +title: Commands +description: Path-based command registration and CLI execution. +--- + +# Commands + +Commands are one of the most AX-native parts of CoreGO. The path is the identity. + +## Register a Command + +```go +c.Command("deploy/to/homelab", core.Command{ + Action: func(opts core.Options) core.Result { + target := opts.String("target") + return core.Result{Value: "deploying to " + target, OK: true} + }, +}) +``` + +## Command Paths + +Paths must be clean: + +- no empty path +- no leading slash +- no trailing slash +- no double slash + +These paths are valid: + +```text +deploy +deploy/to/homelab +workspace/create +``` + +These are rejected: + +```text +/deploy +deploy/ +deploy//to +``` + +## Parent Commands Are Auto-Created + +When you register `deploy/to/homelab`, CoreGO also creates placeholder parents if they do not already exist: + +- `deploy` +- `deploy/to` + +This makes the path tree navigable without extra setup. + +## Read a Command Back + +```go +r := c.Command("deploy/to/homelab") +if r.OK { + cmd := r.Value.(*core.Command) + _ = cmd +} +``` + +## Run a Command Directly + +```go +cmd := c.Command("deploy/to/homelab").Value.(*core.Command) + +r := cmd.Run(core.Options{ + {Key: "target", Value: "uk-prod"}, +}) +``` + +If `Action` is nil, `Run` returns `Result{OK:false}` with a structured error. + +## Run Through the CLI Surface + +```go +r := c.Cli().Run("deploy", "to", "homelab", "--target=uk-prod", "--debug") +``` + +`Cli.Run` resolves the longest matching command path from the arguments, then converts the remaining args into `core.Options`. + +## Flag Parsing Rules + +### Double Dash + +```text +--target=uk-prod -> key "target", value "uk-prod" +--debug -> key "debug", value true +``` + +### Single Dash + +```text +-v -> key "v", value true +-n=4 -> key "n", value "4" +``` + +### Positional Arguments + +Non-flag arguments after the command path are stored as repeated `_arg` options. + +```go +r := c.Cli().Run("workspace", "open", "alpha") +``` + +That produces an option like: + +```go +core.Option{Key: "_arg", Value: "alpha"} +``` + +### Important Details + +- flag values stay as strings +- `opts.Int("port")` only works if some code stored an actual `int` +- invalid flags such as `-verbose` and `--v` are ignored + +## Help Output + +`Cli.PrintHelp()` prints executable commands: + +```go +c.Cli().PrintHelp() +``` + +It skips: + +- hidden commands +- placeholder parents with no `Action` and no `Lifecycle` + +Descriptions are resolved through `cmd.I18nKey()`. + +## I18n Description Keys + +If `Description` is empty, CoreGO derives a key from the path. + +```text +deploy -> cmd.deploy.description +deploy/to/homelab -> cmd.deploy.to.homelab.description +workspace/create -> cmd.workspace.create.description +``` + +If `Description` is already set, CoreGO uses it as-is. + +## Lifecycle Commands + +Commands can also delegate to a lifecycle implementation. + +```go +type daemonCommand struct{} + +func (d *daemonCommand) Start(opts core.Options) core.Result { return core.Result{OK: true} } +func (d *daemonCommand) Stop() core.Result { return core.Result{OK: true} } +func (d *daemonCommand) Restart() core.Result { return core.Result{OK: true} } +func (d *daemonCommand) Reload() core.Result { return core.Result{OK: true} } +func (d *daemonCommand) Signal(sig string) core.Result { return core.Result{Value: sig, OK: true} } + +c.Command("agent/serve", core.Command{ + Lifecycle: &daemonCommand{}, +}) +``` + +Important behavior: + +- `Start` falls back to `Run` when `Lifecycle` is nil +- `Stop`, `Restart`, `Reload`, and `Signal` return an empty `Result` when `Lifecycle` is nil + +## List Command Paths + +```go +paths := c.Commands() +``` + +Like the service registry, the command registry is map-backed, so iteration order is not guaranteed. diff --git a/.core/reference/docs/configuration.md b/.core/reference/docs/configuration.md new file mode 100644 index 0000000..0a0cf11 --- /dev/null +++ b/.core/reference/docs/configuration.md @@ -0,0 +1,96 @@ +--- +title: Configuration +description: Constructor options, runtime settings, and feature flags. +--- + +# Configuration + +CoreGO uses two different configuration layers: + +- constructor-time `core.Options` +- runtime `c.Config()` + +## Constructor-Time Options + +```go +c := core.New(core.Options{ + {Key: "name", Value: "agent-workbench"}, +}) +``` + +### Current Behavior + +- `New` accepts `opts ...Options` +- the current implementation copies only the first `Options` slice +- the `name` key is applied to `c.App().Name` + +If you need more constructor data, put it in the first `core.Options` slice. + +## Runtime Settings with `Config` + +Use `c.Config()` for mutable process settings. + +```go +c.Config().Set("workspace.root", "/srv/workspaces") +c.Config().Set("max_agents", 8) +c.Config().Set("debug", true) +``` + +Read them back with: + +```go +root := c.Config().String("workspace.root") +maxAgents := c.Config().Int("max_agents") +debug := c.Config().Bool("debug") +raw := c.Config().Get("workspace.root") +``` + +### Important Details + +- missing keys return zero values +- typed accessors do not coerce strings into ints or bools +- `Get` returns `core.Result` + +## Feature Flags + +`Config` also tracks named feature flags. + +```go +c.Config().Enable("workspace.templates") +c.Config().Enable("agent.review") +c.Config().Disable("agent.review") +``` + +Read them with: + +```go +enabled := c.Config().Enabled("workspace.templates") +features := c.Config().EnabledFeatures() +``` + +Feature names are case-sensitive. + +## `ConfigVar[T]` + +Use `ConfigVar[T]` when you need a typed value that can also represent “set versus unset”. + +```go +theme := core.NewConfigVar("amber") + +if theme.IsSet() { + fmt.Println(theme.Get()) +} + +theme.Unset() +``` + +This is useful for package-local state where zero values are not enough to describe configuration presence. + +## Recommended Pattern + +Use the two layers for different jobs: + +- put startup identity such as `name` into `core.Options` +- put mutable runtime values and feature switches into `c.Config()` + +That keeps constructor intent separate from live process state. diff --git a/.core/reference/docs/errors.md b/.core/reference/docs/errors.md new file mode 100644 index 0000000..9b7d3f3 --- /dev/null +++ b/.core/reference/docs/errors.md @@ -0,0 +1,120 @@ +--- +title: Errors +description: Structured errors, logging helpers, and panic recovery. +--- + +# Errors + +CoreGO treats failures as structured operational data. + +Repository convention: use `E()` instead of `fmt.Errorf` for framework and service errors. + +## `Err` + +The structured error type is: + +```go +type Err struct { + Operation string + Message string + Cause error + Code string +} +``` + +## Create Errors + +### `E` + +```go +err := core.E("workspace.Load", "failed to read workspace manifest", cause) +``` + +### `Wrap` + +```go +err := core.Wrap(cause, "workspace.Load", "manifest parse failed") +``` + +### `WrapCode` + +```go +err := core.WrapCode(cause, "WORKSPACE_INVALID", "workspace.Load", "manifest parse failed") +``` + +### `NewCode` + +```go +err := core.NewCode("NOT_FOUND", "workspace not found") +``` + +## Inspect Errors + +```go +op := core.Operation(err) +code := core.ErrorCode(err) +msg := core.ErrorMessage(err) +root := core.Root(err) +stack := core.StackTrace(err) +pretty := core.FormatStackTrace(err) +``` + +These helpers keep the operational chain visible without extra type assertions. + +## Join and Standard Wrappers + +```go +combined := core.ErrorJoin(err1, err2) +same := core.Is(combined, err1) +``` + +`core.As` and `core.NewError` mirror the standard library for convenience. + +## Log-and-Return Helpers + +`Core` exposes two convenience wrappers: + +```go +r1 := c.LogError(err, "workspace.Load", "workspace load failed") +r2 := c.LogWarn(err, "workspace.Load", "workspace load degraded") +``` + +These log through the default logger and return `core.Result`. + +You can also use the underlying `ErrorLog` directly: + +```go +r := c.Log().Error(err, "workspace.Load", "workspace load failed") +``` + +`Must` logs and then panics when the error is non-nil: + +```go +c.Must(err, "workspace.Load", "workspace load failed") +``` + +## Panic Recovery + +`ErrorPanic` handles process-safe panic capture. + +```go +defer c.Error().Recover() +``` + +Run background work with recovery: + +```go +c.Error().SafeGo(func() { + panic("captured") +}) +``` + +If `ErrorPanic` has a configured crash file path, it appends JSON crash reports and `Reports(n)` reads them back. + +That crash file path is currently internal state on `ErrorPanic`, not a public constructor option on `Core.New()`. + +## Logging and Error Context + +The logging subsystem automatically extracts `op` and logical stack information from structured errors when those values are present in the key-value list. + +That makes errors created with `E`, `Wrap`, or `WrapCode` much easier to follow in logs. diff --git a/.core/reference/docs/getting-started.md b/.core/reference/docs/getting-started.md new file mode 100644 index 0000000..d2d8166 --- /dev/null +++ b/.core/reference/docs/getting-started.md @@ -0,0 +1,208 @@ +--- +title: Getting Started +description: Build a first CoreGO application with the current API. +--- + +# Getting Started + +This page shows the shortest path to a useful CoreGO application using the API that exists in this repository today. + +## Install + +```bash +go get dappco.re/go/core +``` + +## Create a Core + +`New` takes zero or more `core.Options` slices, but the current implementation only reads the first one. In practice, treat the constructor as `core.New(core.Options{...})`. + +```go +package main + +import "dappco.re/go/core" + +func main() { + c := core.New(core.Options{ + {Key: "name", Value: "agent-workbench"}, + }) + + _ = c +} +``` + +The `name` option is copied into `c.App().Name`. + +## Register a Service + +Services are registered explicitly with a name and a `core.Service` DTO. + +```go +c.Service("audit", core.Service{ + OnStart: func() core.Result { + core.Info("audit service started", "app", c.App().Name) + return core.Result{OK: true} + }, + OnStop: func() core.Result { + core.Info("audit service stopped", "app", c.App().Name) + return core.Result{OK: true} + }, +}) +``` + +This registry stores `core.Service` values. It is a lifecycle registry, not a typed object container. + +## Register a Query, Task, and Command + +```go +type workspaceCountQuery struct{} + +type createWorkspaceTask struct { + Name string +} + +c.RegisterQuery(func(_ *core.Core, q core.Query) core.Result { + switch q.(type) { + case workspaceCountQuery: + return core.Result{Value: 1, OK: true} + } + return core.Result{} +}) + +c.RegisterTask(func(_ *core.Core, t core.Task) core.Result { + switch task := t.(type) { + case createWorkspaceTask: + path := "/tmp/agent-workbench/" + task.Name + return core.Result{Value: path, OK: true} + } + return core.Result{} +}) + +c.Command("workspace/create", core.Command{ + Action: func(opts core.Options) core.Result { + return c.PERFORM(createWorkspaceTask{ + Name: opts.String("name"), + }) + }, +}) +``` + +## Start the Runtime + +```go +if !c.ServiceStartup(context.Background(), nil).OK { + panic("startup failed") +} +``` + +`ServiceStartup` returns `core.Result`, not `error`. + +## Run Through the CLI Surface + +```go +r := c.Cli().Run("workspace", "create", "--name=alpha") +if r.OK { + fmt.Println("created:", r.Value) +} +``` + +For flags with values, the CLI stores the value as a string. `--name=alpha` becomes `opts.String("name") == "alpha"`. + +## Query the System + +```go +count := c.QUERY(workspaceCountQuery{}) +if count.OK { + fmt.Println("workspace count:", count.Value) +} +``` + +## Shut Down Cleanly + +```go +_ = c.ServiceShutdown(context.Background()) +``` + +Shutdown cancels `c.Context()`, broadcasts `ActionServiceShutdown{}`, waits for background tasks to finish, and then runs service stop hooks. + +## Full Example + +```go +package main + +import ( + "context" + "fmt" + + "dappco.re/go/core" +) + +type workspaceCountQuery struct{} + +type createWorkspaceTask struct { + Name string +} + +func main() { + c := core.New(core.Options{ + {Key: "name", Value: "agent-workbench"}, + }) + + c.Config().Set("workspace.root", "/tmp/agent-workbench") + c.Config().Enable("workspace.templates") + + c.Service("audit", core.Service{ + OnStart: func() core.Result { + core.Info("service started", "service", "audit") + return core.Result{OK: true} + }, + OnStop: func() core.Result { + core.Info("service stopped", "service", "audit") + return core.Result{OK: true} + }, + }) + + c.RegisterQuery(func(_ *core.Core, q core.Query) core.Result { + switch q.(type) { + case workspaceCountQuery: + return core.Result{Value: 1, OK: true} + } + return core.Result{} + }) + + c.RegisterTask(func(_ *core.Core, t core.Task) core.Result { + switch task := t.(type) { + case createWorkspaceTask: + path := c.Config().String("workspace.root") + "/" + task.Name + return core.Result{Value: path, OK: true} + } + return core.Result{} + }) + + c.Command("workspace/create", core.Command{ + Action: func(opts core.Options) core.Result { + return c.PERFORM(createWorkspaceTask{ + Name: opts.String("name"), + }) + }, + }) + + if !c.ServiceStartup(context.Background(), nil).OK { + panic("startup failed") + } + + created := c.Cli().Run("workspace", "create", "--name=alpha") + fmt.Println("created:", created.Value) + + count := c.QUERY(workspaceCountQuery{}) + fmt.Println("workspace count:", count.Value) + + _ = c.ServiceShutdown(context.Background()) +} +``` + +## Next Steps + +- Read [primitives.md](primitives.md) next so the repeated shapes are clear. +- Read [commands.md](commands.md) if you are building a CLI-first system. +- Read [messaging.md](messaging.md) if services need to collaborate without direct imports. diff --git a/.core/reference/docs/index.md b/.core/reference/docs/index.md new file mode 100644 index 0000000..ca1ef99 --- /dev/null +++ b/.core/reference/docs/index.md @@ -0,0 +1,114 @@ +--- +title: CoreGO +description: AX-first documentation for the CoreGO framework. +--- + +# CoreGO + +CoreGO is the foundation layer for the Core ecosystem. It gives you one container, one command tree, one message bus, and a small set of shared primitives that repeat across the whole framework. + +The current module path is `dappco.re/go/core`. + +Start with [RFC.md](RFC.md) for the full API contract and [../RFC-025-AGENT-EXPERIENCE.md](../RFC-025-AGENT-EXPERIENCE.md) for the AX design rules that shape it. + +## AX View + +CoreGO already follows the main AX ideas from RFC-025: + +- predictable names such as `Core`, `Service`, `Command`, `Options`, `Result`, `Message` +- path-shaped command registration such as `deploy/to/homelab` +- one repeated input shape (`Options`) and one repeated return shape (`Result`) +- comments and examples that show real usage instead of restating the type signature + +## What CoreGO Owns + +| Surface | Purpose | +|---------|---------| +| `Core` | Central container and access point | +| `Service` | Managed lifecycle component | +| `Command` | Path-based command tree node | +| `ACTION`, `QUERY`, `PERFORM` | Decoupled communication between components | +| `Data`, `Drive`, `Fs`, `Config`, `I18n`, `Cli` | Built-in subsystems for common runtime work | +| `E`, `Wrap`, `ErrorLog`, `ErrorPanic` | Structured failures and panic recovery | + +## Quick Example + +```go +package main + +import ( + "context" + "fmt" + + "dappco.re/go/core" +) + +type flushCacheTask struct { + Name string +} + +func main() { + c := core.New(core.Options{ + {Key: "name", Value: "agent-workbench"}, + }) + + c.Service("cache", core.Service{ + OnStart: func() core.Result { + core.Info("cache ready", "app", c.App().Name) + return core.Result{OK: true} + }, + OnStop: func() core.Result { + core.Info("cache stopped", "app", c.App().Name) + return core.Result{OK: true} + }, + }) + + c.RegisterTask(func(_ *core.Core, task core.Task) core.Result { + switch task.(type) { + case flushCacheTask: + return core.Result{Value: "cache flushed", OK: true} + } + return core.Result{} + }) + + c.Command("cache/flush", core.Command{ + Action: func(opts core.Options) core.Result { + return c.PERFORM(flushCacheTask{Name: opts.String("name")}) + }, + }) + + if !c.ServiceStartup(context.Background(), nil).OK { + panic("startup failed") + } + + r := c.Cli().Run("cache", "flush", "--name=session-store") + fmt.Println(r.Value) + + _ = c.ServiceShutdown(context.Background()) +} +``` + +## Documentation Paths + +| Path | Covers | +|------|--------| +| [getting-started.md](getting-started.md) | First runnable CoreGO app | +| [primitives.md](primitives.md) | `Options`, `Result`, `Service`, `Message`, `Query`, `Task` | +| [services.md](services.md) | Service registry, service locks, runtime helpers | +| [commands.md](commands.md) | Path-based commands and CLI execution | +| [messaging.md](messaging.md) | `ACTION`, `QUERY`, `QUERYALL`, `PERFORM`, `PerformAsync` | +| [lifecycle.md](lifecycle.md) | Startup, shutdown, context, background task draining | +| [configuration.md](configuration.md) | Constructor options, config state, feature flags | +| [subsystems.md](subsystems.md) | `App`, `Data`, `Drive`, `Fs`, `I18n`, `Cli` | +| [errors.md](errors.md) | Structured errors, logging helpers, panic recovery | +| [testing.md](testing.md) | Test naming and framework-level testing patterns | +| [pkg/core.md](pkg/core.md) | Package-level reference summary | +| [pkg/log.md](pkg/log.md) | Logging reference for the root package | +| [pkg/PACKAGE_STANDARDS.md](pkg/PACKAGE_STANDARDS.md) | AX package-authoring guidance | + +## Good Reading Order + +1. Start with [getting-started.md](getting-started.md). +2. Learn the repeated shapes in [primitives.md](primitives.md). +3. Pick the integration path you need next: [services.md](services.md), [commands.md](commands.md), or [messaging.md](messaging.md). +4. Use [subsystems.md](subsystems.md), [errors.md](errors.md), and [testing.md](testing.md) as reference pages while building. diff --git a/.core/reference/docs/lifecycle.md b/.core/reference/docs/lifecycle.md new file mode 100644 index 0000000..59ba644 --- /dev/null +++ b/.core/reference/docs/lifecycle.md @@ -0,0 +1,111 @@ +--- +title: Lifecycle +description: Startup, shutdown, context ownership, and background task draining. +--- + +# Lifecycle + +CoreGO manages lifecycle through `core.Service` callbacks, not through reflection or implicit interfaces. + +## Service Hooks + +```go +c.Service("cache", core.Service{ + OnStart: func() core.Result { + return core.Result{OK: true} + }, + OnStop: func() core.Result { + return core.Result{OK: true} + }, +}) +``` + +Only services with `OnStart` appear in `Startables()`. Only services with `OnStop` appear in `Stoppables()`. + +## `ServiceStartup` + +```go +r := c.ServiceStartup(context.Background(), nil) +``` + +### What It Does + +1. clears the shutdown flag +2. stores a new cancellable context on `c.Context()` +3. runs each `OnStart` +4. broadcasts `ActionServiceStartup{}` + +### Failure Behavior + +- if the input context is already cancelled, startup returns that error +- if any `OnStart` returns `OK:false`, startup stops immediately and returns that result + +## `ServiceShutdown` + +```go +r := c.ServiceShutdown(context.Background()) +``` + +### What It Does + +1. sets the shutdown flag +2. cancels `c.Context()` +3. broadcasts `ActionServiceShutdown{}` +4. waits for background tasks created by `PerformAsync` +5. runs each `OnStop` + +### Failure Behavior + +- if draining background tasks hits the shutdown context deadline, shutdown returns that context error +- when service stop hooks fail, CoreGO returns the first error it sees + +## Ordering + +The current implementation builds `Startables()` and `Stoppables()` by iterating over a map-backed registry. + +That means lifecycle order is not guaranteed today. + +If your application needs strict startup or shutdown ordering, orchestrate it explicitly inside a smaller number of service callbacks instead of relying on registry order. + +## `c.Context()` + +`ServiceStartup` creates the context returned by `c.Context()`. + +Use it for background work that should stop when the application shuts down: + +```go +c.Service("watcher", core.Service{ + OnStart: func() core.Result { + go func(ctx context.Context) { + <-ctx.Done() + }(c.Context()) + return core.Result{OK: true} + }, +}) +``` + +## Built-In Lifecycle Actions + +You can listen for lifecycle state changes through the action bus. + +```go +c.RegisterAction(func(_ *core.Core, msg core.Message) core.Result { + switch msg.(type) { + case core.ActionServiceStartup: + core.Info("core startup completed") + case core.ActionServiceShutdown: + core.Info("core shutdown started") + } + return core.Result{OK: true} +}) +``` + +## Background Task Draining + +`ServiceShutdown` waits for the internal task waitgroup to finish before calling stop hooks. + +This is what makes `PerformAsync` safe for long-running work that should complete before teardown. + +## `OnReload` + +`Service` includes an `OnReload` callback field, but CoreGO does not currently expose a top-level lifecycle runner for reload operations. diff --git a/.core/reference/docs/messaging.md b/.core/reference/docs/messaging.md new file mode 100644 index 0000000..688893a --- /dev/null +++ b/.core/reference/docs/messaging.md @@ -0,0 +1,171 @@ +--- +title: Messaging +description: ACTION, QUERY, QUERYALL, PERFORM, and async task flow. +--- + +# Messaging + +CoreGO uses one message bus for broadcasts, lookups, and work dispatch. + +## Message Types + +```go +type Message any +type Query any +type Task any +``` + +Your own structs define the protocol. + +```go +type repositoryIndexed struct { + Name string +} + +type repositoryCountQuery struct{} + +type syncRepositoryTask struct { + Name string +} +``` + +## `ACTION` + +`ACTION` is a broadcast. + +```go +c.RegisterAction(func(_ *core.Core, msg core.Message) core.Result { + switch m := msg.(type) { + case repositoryIndexed: + core.Info("repository indexed", "name", m.Name) + return core.Result{OK: true} + } + return core.Result{OK: true} +}) + +r := c.ACTION(repositoryIndexed{Name: "core-go"}) +``` + +### Behavior + +- all registered action handlers are called in their current registration order +- if a handler returns `OK:false`, dispatch stops and that `Result` is returned +- if no handler fails, `ACTION` returns `Result{OK:true}` + +## `QUERY` + +`QUERY` is first-match request-response. + +```go +c.RegisterQuery(func(_ *core.Core, q core.Query) core.Result { + switch q.(type) { + case repositoryCountQuery: + return core.Result{Value: 42, OK: true} + } + return core.Result{} +}) + +r := c.QUERY(repositoryCountQuery{}) +``` + +### Behavior + +- handlers run until one returns `OK:true` +- the first successful result wins +- if nothing handles the query, CoreGO returns an empty `Result` + +## `QUERYALL` + +`QUERYALL` collects every successful non-nil response. + +```go +r := c.QUERYALL(repositoryCountQuery{}) +results := r.Value.([]any) +``` + +### Behavior + +- every query handler is called +- only `OK:true` results with non-nil `Value` are collected +- the call itself returns `OK:true` even when the result list is empty + +## `PERFORM` + +`PERFORM` dispatches a task to the first handler that accepts it. + +```go +c.RegisterTask(func(_ *core.Core, t core.Task) core.Result { + switch task := t.(type) { + case syncRepositoryTask: + return core.Result{Value: "synced " + task.Name, OK: true} + } + return core.Result{} +}) + +r := c.PERFORM(syncRepositoryTask{Name: "core-go"}) +``` + +### Behavior + +- handlers run until one returns `OK:true` +- the first successful result wins +- if nothing handles the task, CoreGO returns an empty `Result` + +## `PerformAsync` + +`PerformAsync` runs a task in a background goroutine and returns a generated task identifier. + +```go +r := c.PerformAsync(syncRepositoryTask{Name: "core-go"}) +taskID := r.Value.(string) +``` + +### Generated Events + +Async execution emits three action messages: + +| Message | When | +|---------|------| +| `ActionTaskStarted` | just before background execution begins | +| `ActionTaskProgress` | whenever `Progress` is called | +| `ActionTaskCompleted` | after the task finishes or panics | + +Example listener: + +```go +c.RegisterAction(func(_ *core.Core, msg core.Message) core.Result { + switch m := msg.(type) { + case core.ActionTaskCompleted: + core.Info("task completed", "task", m.TaskIdentifier, "err", m.Error) + } + return core.Result{OK: true} +}) +``` + +## Progress Updates + +```go +c.Progress(taskID, 0.5, "indexing commits", syncRepositoryTask{Name: "core-go"}) +``` + +That broadcasts `ActionTaskProgress`. + +## `TaskWithIdentifier` + +Tasks that implement `TaskWithIdentifier` receive the generated ID before dispatch. + +```go +type trackedTask struct { + ID string + Name string +} + +func (t *trackedTask) SetTaskIdentifier(id string) { t.ID = id } +func (t *trackedTask) GetTaskIdentifier() string { return t.ID } +``` + +## Shutdown Interaction + +When shutdown has started, `PerformAsync` returns an empty `Result` instead of scheduling more work. + +This is why `ServiceShutdown` can safely drain the outstanding background tasks before stopping services. diff --git a/.core/reference/docs/pkg/PACKAGE_STANDARDS.md b/.core/reference/docs/pkg/PACKAGE_STANDARDS.md new file mode 100644 index 0000000..398bbf6 --- /dev/null +++ b/.core/reference/docs/pkg/PACKAGE_STANDARDS.md @@ -0,0 +1,138 @@ +# AX Package Standards + +This page describes how to build packages on top of CoreGO in the style described by RFC-025. + +## 1. Prefer Predictable Names + +Use names that tell an agent what the thing is without translation. + +Good: + +- `RepositoryService` +- `RepositoryServiceOptions` +- `WorkspaceCountQuery` +- `SyncRepositoryTask` + +Avoid shortening names unless the abbreviation is already universal. + +## 2. Put Real Usage in Comments + +Write comments that show a real call with realistic values. + +Good: + +```go +// Sync a repository into the local workspace cache. +// svc.SyncRepository("core-go", "/srv/repos/core-go") +``` + +Avoid comments that only repeat the signature. + +## 3. Keep Paths Semantic + +If a command or template lives at a path, let the path explain the intent. + +Good: + +```text +deploy/to/homelab +workspace/create +template/workspace/go +``` + +That keeps the CLI, tests, docs, and message vocabulary aligned. + +## 4. Reuse CoreGO Primitives + +At Core boundaries, prefer the shared shapes: + +- `core.Options` for lightweight input +- `core.Result` for output +- `core.Service` for lifecycle registration +- `core.Message`, `core.Query`, `core.Task` for bus protocols + +Inside your package, typed structs are still good. Use `ServiceRuntime[T]` when you want typed package options plus a `Core` reference. + +```go +type repositoryServiceOptions struct { + BaseDirectory string +} + +type repositoryService struct { + *core.ServiceRuntime[repositoryServiceOptions] +} +``` + +## 5. Prefer Explicit Registration + +Register services and commands with names and paths that stay readable in grep results. + +```go +c.Service("repository", core.Service{...}) +c.Command("repository/sync", core.Command{...}) +``` + +## 6. Use the Bus for Decoupling + +When one package needs another package’s behavior, prefer queries and tasks over tight package coupling. + +```go +type repositoryCountQuery struct{} +type syncRepositoryTask struct { + Name string +} +``` + +That keeps the protocol visible in code and easy for agents to follow. + +## 7. Use Structured Errors + +Use `core.E`, `core.Wrap`, and `core.WrapCode`. + +```go +return core.Result{ + Value: core.E("repository.Sync", "git fetch failed", err), + OK: false, +} +``` + +Do not introduce free-form `fmt.Errorf` chains in framework code. + +## 8. Keep Testing Names Predictable + +Follow the repository pattern: + +- `_Good` +- `_Bad` +- `_Ugly` + +Example: + +```go +func TestRepositorySync_Good(t *testing.T) {} +func TestRepositorySync_Bad(t *testing.T) {} +func TestRepositorySync_Ugly(t *testing.T) {} +``` + +## 9. Prefer Stable Shapes Over Clever APIs + +For package APIs, avoid patterns that force an agent to infer too much hidden control flow. + +Prefer: + +- clear structs +- explicit names +- path-based commands +- visible message types + +Avoid: + +- implicit global state unless it is truly a default service +- panic-hiding constructors +- dense option chains when a small explicit struct would do + +## 10. Document the Current Reality + +If the implementation is in transition, document what the code does now, not the API shape you plan to have later. + +That keeps agents correct on first pass, which is the real AX metric. diff --git a/.core/reference/docs/pkg/core.md b/.core/reference/docs/pkg/core.md new file mode 100644 index 0000000..88bd18b --- /dev/null +++ b/.core/reference/docs/pkg/core.md @@ -0,0 +1,81 @@ +# Package Reference: `core` + +Import path: + +```go +import "dappco.re/go/core" +``` + +This repository exposes one root package. The main areas are: + +## Constructors and Accessors + +| Name | Purpose | +|------|---------| +| `New` | Create a `*Core` | +| `NewRuntime` | Create an empty runtime wrapper | +| `NewWithFactories` | Create a runtime wrapper from named service factories | +| `Options`, `App`, `Data`, `Drive`, `Fs`, `Config`, `Error`, `Log`, `Cli`, `IPC`, `I18n`, `Context` | Access the built-in subsystems | + +## Core Primitives + +| Name | Purpose | +|------|---------| +| `Option`, `Options` | Input configuration and metadata | +| `Result` | Shared output shape | +| `Service` | Lifecycle DTO | +| `Command` | Command tree node | +| `Message`, `Query`, `Task` | Message bus payload types | + +## Service and Runtime APIs + +| Name | Purpose | +|------|---------| +| `Service` | Register or read a named service | +| `Services` | List registered service names | +| `Startables`, `Stoppables` | Snapshot lifecycle-capable services | +| `LockEnable`, `LockApply` | Activate the service registry lock | +| `ServiceRuntime[T]` | Helper for package authors | + +## Command and CLI APIs + +| Name | Purpose | +|------|---------| +| `Command` | Register or read a command by path | +| `Commands` | List command paths | +| `Cli().Run` | Resolve arguments to a command and execute it | +| `Cli().PrintHelp` | Show executable commands | + +## Messaging APIs + +| Name | Purpose | +|------|---------| +| `ACTION`, `Action` | Broadcast a message | +| `QUERY`, `Query` | Return the first successful query result | +| `QUERYALL`, `QueryAll` | Collect all successful query results | +| `PERFORM`, `Perform` | Run the first task handler that accepts the task | +| `PerformAsync` | Run a task in the background | +| `Progress` | Broadcast async task progress | +| `RegisterAction`, `RegisterActions`, `RegisterQuery`, `RegisterTask` | Register bus handlers | + +## Subsystems + +| Name | Purpose | +|------|---------| +| `Config` | Runtime settings and feature flags | +| `Data` | Embedded filesystem mounts | +| `Drive` | Named transport handles | +| `Fs` | Local filesystem operations | +| `I18n` | Locale collection and translation delegation | +| `App`, `Find` | Application identity and executable lookup | + +## Errors and Logging + +| Name | Purpose | +|------|---------| +| `E`, `Wrap`, `WrapCode`, `NewCode` | Structured error creation | +| `Operation`, `ErrorCode`, `ErrorMessage`, `Root`, `StackTrace`, `FormatStackTrace` | Error inspection | +| `NewLog`, `Default`, `SetDefault`, `SetLevel`, `SetRedactKeys` | Logger creation and defaults | +| `LogErr`, `LogPanic`, `ErrorLog`, `ErrorPanic` | Error-aware logging and panic recovery | + +Use the top-level docs in `docs/` for task-oriented guidance, then use this page as a compact reference. diff --git a/.core/reference/docs/pkg/log.md b/.core/reference/docs/pkg/log.md new file mode 100644 index 0000000..15e9db1 --- /dev/null +++ b/.core/reference/docs/pkg/log.md @@ -0,0 +1,83 @@ +# Logging Reference + +Logging lives in the root `core` package in this repository. There is no separate `pkg/log` import path here. + +## Create a Logger + +```go +logger := core.NewLog(core.LogOptions{ + Level: core.LevelInfo, +}) +``` + +## Levels + +| Level | Meaning | +|-------|---------| +| `LevelQuiet` | no output | +| `LevelError` | errors and security events | +| `LevelWarn` | warnings, errors, security events | +| `LevelInfo` | informational, warnings, errors, security events | +| `LevelDebug` | everything | + +## Log Methods + +```go +logger.Debug("workspace discovered", "path", "/srv/workspaces") +logger.Info("service started", "service", "audit") +logger.Warn("retrying fetch", "attempt", 2) +logger.Error("fetch failed", "err", err) +logger.Security("sandbox escape detected", "path", attemptedPath) +``` + +## Default Logger + +The package owns a default logger. + +```go +core.SetLevel(core.LevelDebug) +core.SetRedactKeys("token", "password") + +core.Info("service started", "service", "audit") +``` + +## Redaction + +Values for keys listed in `RedactKeys` are replaced with `[REDACTED]`. + +```go +logger.SetRedactKeys("token") +logger.Info("login", "user", "cladius", "token", "secret-value") +``` + +## Output and Rotation + +```go +logger := core.NewLog(core.LogOptions{ + Level: core.LevelInfo, + Output: os.Stderr, +}) +``` + +If you provide `Rotation` and set `RotationWriterFactory`, the logger writes to the rotating writer instead of the plain output stream. + +## Error-Aware Logging + +`LogErr` extracts structured error context before logging: + +```go +le := core.NewLogErr(logger) +le.Log(err) +``` + +`ErrorLog` is the log-and-return wrapper exposed through `c.Log()`. + +## Panic-Aware Logging + +`LogPanic` is the lightweight panic logger: + +```go +defer core.NewLogPanic(logger).Recover() +``` + +It logs the recovered panic but does not manage crash files. For crash reports, use `c.Error().Recover()`. diff --git a/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-design.md b/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-design.md new file mode 100644 index 0000000..0825791 --- /dev/null +++ b/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-design.md @@ -0,0 +1,261 @@ +# Lint Pattern Catalog & Polish Skill Design + +> **Partial implementation (14 Mar 2026):** Layer 1 (`core/lint` -- catalog, matcher, scanner, CLI) is fully implemented and documented at `docs/tools/lint/index.md`. Layer 2 (MCP subsystem in `go-ai`) and Layer 3 (Claude Code polish skill in `core/agent`) are NOT implemented. This plan is retained for those remaining layers. + +**Goal:** A structured pattern catalog (`core/lint`) that captures recurring code quality findings as regex rules, exposes them via MCP tools in `go-ai`, and orchestrates multi-AI code review via a Claude Code skill in `core/agent`. + +**Architecture:** Three layers — a standalone catalog+matcher library (`core/lint`), an MCP subsystem in `go-ai` that exposes lint tools to agents, and a Claude Code plugin in `core/agent` that orchestrates the "polish" workflow (deterministic checks + AI reviewers + feedback loop into the catalog). + +**Tech Stack:** Go (catalog, matcher, CLI, MCP subsystem), YAML (rule definitions), JSONL (findings output, compatible with `~/.core/ai/metrics/`), Claude Code plugin format (hooks.json, commands/*.md, plugin.json). + +--- + +## Context + +During a code review sweep of 18 Go repos (March 2026), AI reviewers (Gemini, Claude) found ~20 recurring patterns: SQL injection, path traversal, XSS, missing constant-time comparison, goroutine leaks, Go 1.26 modernisation opportunities, and more. Many of these patterns repeat across repos. + +Currently these findings exist only as commit messages. This design captures them as a reusable, machine-readable catalog that: +1. Deterministic tools can run immediately (regex matching) +2. MCP-connected agents can query and apply +3. LEM models can train on for "does this comply with CoreGo standards?" judgements +4. Grows automatically as AI reviewers find new patterns + +## Layer 1: `core/lint` — Pattern Catalog & Matcher + +### Repository Structure + +``` +core/lint/ +├── go.mod # forge.lthn.ai/core/lint +├── catalog/ +│ ├── go-security.yaml # SQL injection, path traversal, XSS, constant-time +│ ├── go-modernise.yaml # Go 1.26: slices.Clone, wg.Go, maps.Keys, range-over-int +│ ├── go-correctness.yaml # Deadlocks, goroutine leaks, nil guards, error handling +│ ├── php-security.yaml # XSS, CSRF, mass assignment, SQL injection +│ ├── ts-security.yaml # DOM XSS, prototype pollution +│ └── cpp-safety.yaml # Buffer overflow, use-after-free +├── pkg/lint/ +│ ├── catalog.go # Load + parse YAML catalog files +│ ├── rule.go # Rule struct definition +│ ├── matcher.go # Regex matcher against file contents +│ ├── report.go # Structured findings output (JSON/JSONL/text) +│ ├── catalog_test.go +│ ├── matcher_test.go +│ └── report_test.go +├── cmd/core-lint/ +│ └── main.go # `core-lint check ./...` CLI +└── .core/ + └── build.yaml # Produces core-lint binary +``` + +### Rule Schema (YAML) + +```yaml +- id: go-sec-001 + title: "SQL wildcard injection in LIKE clauses" + severity: high # critical, high, medium, low, info + languages: [go] + tags: [security, injection, owasp-a03] + pattern: 'LIKE\s+\?\s*,\s*["\x60]%\s*\+' + exclude_pattern: 'EscapeLike' # suppress if this also matches + fix: "Use parameterised LIKE with explicit escaping of % and _ characters" + found_in: [go-store] # repos where first discovered + example_bad: | + db.Where("name LIKE ?", "%"+input+"%") + example_good: | + db.Where("name LIKE ?", EscapeLike(input)) + first_seen: "2026-03-09" + detection: regex # future: ast, semantic + auto_fixable: false # future: true when we add codemods +``` + +### Rule Struct (Go) + +```go +type Rule struct { + ID string `yaml:"id"` + Title string `yaml:"title"` + Severity string `yaml:"severity"` + Languages []string `yaml:"languages"` + Tags []string `yaml:"tags"` + Pattern string `yaml:"pattern"` + ExcludePattern string `yaml:"exclude_pattern,omitempty"` + Fix string `yaml:"fix"` + FoundIn []string `yaml:"found_in,omitempty"` + ExampleBad string `yaml:"example_bad,omitempty"` + ExampleGood string `yaml:"example_good,omitempty"` + FirstSeen string `yaml:"first_seen"` + Detection string `yaml:"detection"` // regex | ast | semantic + AutoFixable bool `yaml:"auto_fixable"` +} +``` + +### Finding Struct (Go) + +Designed to align with go-ai's `ScanAlert` shape and `~/.core/ai/metrics/` JSONL format: + +```go +type Finding struct { + RuleID string `json:"rule_id"` + Title string `json:"title"` + Severity string `json:"severity"` + File string `json:"file"` + Line int `json:"line"` + Match string `json:"match"` // matched text + Fix string `json:"fix"` + Repo string `json:"repo,omitempty"` +} +``` + +### CLI Interface + +```bash +# Check current directory against all catalogs for detected languages +core-lint check ./... + +# Check specific languages/catalogs +core-lint check --lang go --catalog go-security ./pkg/... + +# Output as JSON (for piping to other tools) +core-lint check --format json ./... + +# List available rules +core-lint catalog list +core-lint catalog list --lang go --severity high + +# Show a specific rule with examples +core-lint catalog show go-sec-001 +``` + +## Layer 2: `go-ai` Lint MCP Subsystem + +New subsystem registered alongside files/rag/ml/brain: + +```go +type LintSubsystem struct { + catalog *lint.Catalog + root string // workspace root for scanning +} + +func (s *LintSubsystem) Name() string { return "lint" } + +func (s *LintSubsystem) RegisterTools(server *mcp.Server) { + // lint_check - run rules against workspace files + // lint_catalog - list/search available rules + // lint_report - get findings summary for a path +} +``` + +### MCP Tools + +| Tool | Input | Output | Group | +|------|-------|--------|-------| +| `lint_check` | `{path: string, lang?: string, severity?: string}` | `{findings: []Finding}` | lint | +| `lint_catalog` | `{lang?: string, tags?: []string, severity?: string}` | `{rules: []Rule}` | lint | +| `lint_report` | `{path: string, format?: "summary" or "detailed"}` | `{summary: ReportSummary}` | lint | + +This means any MCP-connected agent (Claude, LEM, Codex) can call `lint_check` to scan code against the catalog. + +## Layer 3: `core/agent` Polish Skill + +Claude Code plugin at `core/agent/claude/polish/`: + +``` +core/agent/claude/polish/ +├── plugin.json +├── hooks.json # optional: PostToolUse after git commit +├── commands/ +│ └── polish.md # /polish slash command +└── scripts/ + └── run-lint.sh # shells out to core-lint +``` + +### `/polish` Command Flow + +1. Run `core-lint check ./...` for fast deterministic findings +2. Report findings to user +3. Optionally run AI reviewers (Gemini CLI, Codex) for deeper analysis +4. Deduplicate AI findings against catalog (already-known patterns) +5. Propose new patterns as catalog additions (PR to core/lint) + +### Subagent Configuration (`.core/agents/`) + +Repos can configure polish behaviour: + +```yaml +# any-repo/.core/agents/polish.yaml +languages: [go] +catalogs: [go-security, go-modernise, go-correctness] +reviewers: [gemini] # which AI tools to invoke +exclude: [vendor/, testdata/, *_test.go] +severity_threshold: medium # only report medium+ findings +``` + +## Findings to LEM Pipeline + +``` +core-lint check -> findings.json + | + v +~/.core/ai/metrics/YYYY-MM-DD.jsonl (audit trail) + | + v +LEM training data: + - Rule examples (bad/good pairs) -> supervised training signal + - Finding frequency -> pattern importance weighting + - Rule descriptions -> natural language understanding of "why" + | + v +LEM tool: "does this code comply with CoreGo standards?" + -> queries lint_catalog via MCP + -> applies learned pattern recognition + -> reports violations with rule IDs and fixes +``` + +## Initial Catalog Seed + +From the March 2026 ecosystem sweep: + +| ID | Title | Severity | Language | Found In | +|----|-------|----------|----------|----------| +| go-sec-001 | SQL wildcard injection | high | go | go-store | +| go-sec-002 | Path traversal in cache keys | high | go | go-cache | +| go-sec-003 | XSS in HTML output | high | go | go-html | +| go-sec-004 | Non-constant-time auth comparison | high | go | go-crypt | +| go-sec-005 | Log injection via unescaped input | medium | go | go-log | +| go-sec-006 | Key material in log output | high | go | go-log | +| go-cor-001 | Goroutine leak (no WaitGroup) | high | go | core/go | +| go-cor-002 | Shutdown deadlock (wg.Wait no timeout) | high | go | core/go | +| go-cor-003 | Silent error swallowing | medium | go | go-process, go-ratelimit | +| go-cor-004 | Panic in library code | medium | go | go-i18n | +| go-cor-005 | Delete without path validation | high | go | go-io | +| go-mod-001 | Manual slice clone (append nil pattern) | low | go | core/go | +| go-mod-002 | Manual sort instead of slices.Sorted | low | go | core/go | +| go-mod-003 | Manual reverse loop instead of slices.Backward | low | go | core/go | +| go-mod-004 | sync.WaitGroup Add+Done instead of Go() | low | go | core/go | +| go-mod-005 | Manual map key collection instead of maps.Keys | low | go | core/go | +| go-cor-006 | Missing error return from API calls | medium | go | go-forge, go-git | +| go-cor-007 | Signal handler uses wrong type | medium | go | go-process | + +## Dependencies + +``` +core/lint (standalone, zero core deps) + ^ + | +go-ai/mcp/lint/ (imports core/lint for catalog + matcher) + ^ + | +core/agent/claude/polish/ (shells out to core-lint CLI) +``` + +`core/lint` has no dependency on `core/go` or any other framework module. It is a standalone library + CLI, like `core/go-io`. + +## Future Extensions (Not Built Now) + +- **AST-based detection** (layer 2): Parse Go/PHP AST, match structural patterns +- **Semantic detection** (layer 3): LEM judges code against rule descriptions +- **Auto-fix codemods**: `core-lint fix` applies known fixes automatically +- **CI integration**: GitHub Actions workflow runs `core-lint check` on PRs +- **CodeRabbit integration**: Import CodeRabbit findings as catalog entries +- **Cross-repo dashboard**: Aggregate findings across all repos in workspace diff --git a/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-plan.md b/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-plan.md new file mode 100644 index 0000000..7f1ddec --- /dev/null +++ b/.core/reference/docs/plans/2026-03-09-lint-pattern-catalog-plan.md @@ -0,0 +1,1668 @@ +# Lint Pattern Catalog Implementation Plan + +> **Fully implemented (14 Mar 2026).** All tasks in this plan are complete. The `core/lint` module ships 18 rules across 3 catalogs, with a working CLI and embedded YAML. This plan is retained alongside the design doc, which tracks the remaining MCP and polish skill layers. + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Build `core/lint` — a standalone Go library + CLI that loads YAML pattern catalogs and runs regex-based code checks, seeded with 18 patterns from the March 2026 ecosystem sweep. + +**Architecture:** Standalone Go module (`forge.lthn.ai/core/lint`) with zero framework deps. YAML catalog files define rules (id, severity, regex pattern, fix). `pkg/lint` loads catalogs and matches patterns against files. `cmd/core-lint` is a Cobra CLI. Uses `cli.Main()` + `cli.WithCommands()` from `core/cli`. + +**Tech Stack:** Go 1.26, `gopkg.in/yaml.v3` (YAML parsing), `forge.lthn.ai/core/cli` (CLI framework), `github.com/stretchr/testify` (testing), `embed` (catalog embedding). + +--- + +### Task 1: Create repo and Go module + +**Files:** +- Create: `/Users/snider/Code/core/lint/go.mod` +- Create: `/Users/snider/Code/core/lint/.core/build.yaml` +- Create: `/Users/snider/Code/core/lint/CLAUDE.md` + +**Step 1: Create repo on forge** + +```bash +ssh -p 2223 git@forge.lthn.ai +``` + +If SSH repo creation isn't available, create via Forgejo API: +```bash +curl -X POST "https://forge.lthn.ai/api/v1/orgs/core/repos" \ + -H "Authorization: token $FORGE_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"name":"lint","description":"Pattern catalog & regex matcher for code quality","auto_init":true,"default_branch":"main"}' +``` + +Or manually create on forge.lthn.ai web UI under the `core` org. + +**Step 2: Clone and initialise Go module** + +```bash +cd ~/Code/core +git clone ssh://git@forge.lthn.ai:2223/core/lint.git +cd lint +go mod init forge.lthn.ai/core/lint +``` + +Set Go version in go.mod: +``` +module forge.lthn.ai/core/lint + +go 1.26.0 +``` + +**Step 3: Create `.core/build.yaml`** + +```yaml +version: 1 + +project: + name: core-lint + description: Pattern catalog and regex code checker + main: ./cmd/core-lint + binary: core-lint + +build: + cgo: false + flags: + - -trimpath + ldflags: + - -s + - -w + +targets: + - os: linux + arch: amd64 + - os: linux + arch: arm64 + - os: darwin + arch: arm64 + - os: windows + arch: amd64 +``` + +**Step 4: Create `CLAUDE.md`** + +```markdown +# CLAUDE.md + +## Project Overview + +`core/lint` is a standalone pattern catalog and regex-based code checker. It loads YAML rule definitions and matches them against source files. Zero framework dependencies. + +## Build & Development + +```bash +core go test +core go qa +core build # produces ./bin/core-lint +``` + +## Architecture + +- `catalog/` — YAML rule files (embedded at compile time) +- `pkg/lint/` — Library: Rule, Catalog, Matcher, Report types +- `cmd/core-lint/` — CLI binary using `cli.Main()` + +## Rule Schema + +Each YAML file contains an array of rules with: id, title, severity, languages, tags, pattern (regex), exclude_pattern, fix, example_bad, example_good, detection type. + +## Coding Standards + +- UK English +- `declare(strict_types=1)` equivalent: all functions have typed params/returns +- Tests use testify +- License: EUPL-1.2 +``` + +**Step 5: Add to go.work** + +Add `./core/lint` to `~/Code/go.work` under the Core framework section. + +**Step 6: Commit** + +```bash +git add go.mod .core/ CLAUDE.md +git commit -m "feat: initialise core/lint module" +``` + +--- + +### Task 2: Rule struct and YAML parsing + +**Files:** +- Create: `/Users/snider/Code/core/lint/pkg/lint/rule.go` +- Create: `/Users/snider/Code/core/lint/pkg/lint/rule_test.go` + +**Step 1: Write the failing test** + +```go +package lint + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestParseRules(t *testing.T) { + yaml := ` +- id: test-001 + title: "Test rule" + severity: high + languages: [go] + tags: [security] + pattern: 'fmt\.Println' + fix: "Use structured logging" + detection: regex +` + rules, err := ParseRules([]byte(yaml)) + require.NoError(t, err) + require.Len(t, rules, 1) + assert.Equal(t, "test-001", rules[0].ID) + assert.Equal(t, "high", rules[0].Severity) + assert.Equal(t, []string{"go"}, rules[0].Languages) + assert.Equal(t, `fmt\.Println`, rules[0].Pattern) +} + +func TestParseRules_Invalid(t *testing.T) { + _, err := ParseRules([]byte("not: valid: yaml: [")) + assert.Error(t, err) +} + +func TestRule_Validate(t *testing.T) { + good := Rule{ID: "x-001", Title: "T", Severity: "high", Languages: []string{"go"}, Pattern: "foo", Detection: "regex"} + assert.NoError(t, good.Validate()) + + bad := Rule{} // missing required fields + assert.Error(t, bad.Validate()) +} + +func TestRule_Validate_BadRegex(t *testing.T) { + r := Rule{ID: "x-001", Title: "T", Severity: "high", Languages: []string{"go"}, Pattern: "[invalid", Detection: "regex"} + assert.Error(t, r.Validate()) +} +``` + +**Step 2: Run test to verify it fails** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v` +Expected: FAIL — `ParseRules` and `Rule` not defined + +**Step 3: Write minimal implementation** + +```go +package lint + +import ( + "fmt" + "regexp" + + "gopkg.in/yaml.v3" +) + +// Rule defines a single lint pattern check. +type Rule struct { + ID string `yaml:"id" json:"id"` + Title string `yaml:"title" json:"title"` + Severity string `yaml:"severity" json:"severity"` + Languages []string `yaml:"languages" json:"languages"` + Tags []string `yaml:"tags" json:"tags"` + Pattern string `yaml:"pattern" json:"pattern"` + ExcludePattern string `yaml:"exclude_pattern" json:"exclude_pattern,omitempty"` + Fix string `yaml:"fix" json:"fix"` + FoundIn []string `yaml:"found_in" json:"found_in,omitempty"` + ExampleBad string `yaml:"example_bad" json:"example_bad,omitempty"` + ExampleGood string `yaml:"example_good" json:"example_good,omitempty"` + FirstSeen string `yaml:"first_seen" json:"first_seen,omitempty"` + Detection string `yaml:"detection" json:"detection"` + AutoFixable bool `yaml:"auto_fixable" json:"auto_fixable"` +} + +// Validate checks that a Rule has all required fields and a compilable regex pattern. +func (r *Rule) Validate() error { + if r.ID == "" { + return fmt.Errorf("rule missing id") + } + if r.Title == "" { + return fmt.Errorf("rule %s: missing title", r.ID) + } + if r.Severity == "" { + return fmt.Errorf("rule %s: missing severity", r.ID) + } + if len(r.Languages) == 0 { + return fmt.Errorf("rule %s: missing languages", r.ID) + } + if r.Pattern == "" { + return fmt.Errorf("rule %s: missing pattern", r.ID) + } + if r.Detection == "regex" { + if _, err := regexp.Compile(r.Pattern); err != nil { + return fmt.Errorf("rule %s: invalid regex: %w", r.ID, err) + } + } + return nil +} + +// ParseRules parses YAML bytes into a slice of Rules. +func ParseRules(data []byte) ([]Rule, error) { + var rules []Rule + if err := yaml.Unmarshal(data, &rules); err != nil { + return nil, fmt.Errorf("parse rules: %w", err) + } + return rules, nil +} +``` + +**Step 4: Run test to verify it passes** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v` +Expected: PASS (4 tests) + +**Step 5: Add yaml dependency** + +```bash +cd ~/Code/core/lint && go get gopkg.in/yaml.v3 && go get github.com/stretchr/testify +``` + +**Step 6: Commit** + +```bash +git add pkg/lint/rule.go pkg/lint/rule_test.go go.mod go.sum +git commit -m "feat: add Rule struct with YAML parsing and validation" +``` + +--- + +### Task 3: Catalog loader with embed support + +**Files:** +- Create: `/Users/snider/Code/core/lint/pkg/lint/catalog.go` +- Create: `/Users/snider/Code/core/lint/pkg/lint/catalog_test.go` +- Create: `/Users/snider/Code/core/lint/catalog/go-security.yaml` (minimal test file) + +**Step 1: Create a minimal test catalog file** + +Create `/Users/snider/Code/core/lint/catalog/go-security.yaml`: +```yaml +- id: go-sec-001 + title: "SQL wildcard injection in LIKE clauses" + severity: high + languages: [go] + tags: [security, injection] + pattern: 'LIKE\s+\?\s*,\s*["%].*\+' + fix: "Use parameterised LIKE with EscapeLike()" + found_in: [go-store] + first_seen: "2026-03-09" + detection: regex +``` + +**Step 2: Write the failing test** + +```go +package lint + +import ( + "embed" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestCatalog_LoadDir(t *testing.T) { + // Find the catalog/ dir relative to the module root + dir := filepath.Join("..", "..", "catalog") + cat, err := LoadDir(dir) + require.NoError(t, err) + assert.Greater(t, len(cat.Rules), 0) + assert.Equal(t, "go-sec-001", cat.Rules[0].ID) +} + +func TestCatalog_LoadDir_NotExist(t *testing.T) { + _, err := LoadDir("/nonexistent") + assert.Error(t, err) +} + +func TestCatalog_Filter_Language(t *testing.T) { + cat := &Catalog{Rules: []Rule{ + {ID: "go-001", Languages: []string{"go"}, Severity: "high"}, + {ID: "php-001", Languages: []string{"php"}, Severity: "high"}, + }} + filtered := cat.ForLanguage("go") + assert.Len(t, filtered, 1) + assert.Equal(t, "go-001", filtered[0].ID) +} + +func TestCatalog_Filter_Severity(t *testing.T) { + cat := &Catalog{Rules: []Rule{ + {ID: "a", Severity: "high"}, + {ID: "b", Severity: "low"}, + {ID: "c", Severity: "medium"}, + }} + filtered := cat.AtSeverity("medium") + assert.Len(t, filtered, 2) // high + medium +} + +func TestCatalog_LoadFS(t *testing.T) { + // Write temp yaml + dir := t.TempDir() + data := []byte(`- id: fs-001 + title: "FS test" + severity: low + languages: [go] + tags: [] + pattern: 'test' + fix: "fix" + detection: regex +`) + require.NoError(t, os.WriteFile(filepath.Join(dir, "test.yaml"), data, 0644)) + + cat, err := LoadDir(dir) + require.NoError(t, err) + assert.Len(t, cat.Rules, 1) +} +``` + +**Step 3: Write minimal implementation** + +```go +package lint + +import ( + "embed" + "fmt" + "io/fs" + "os" + "path/filepath" + "slices" + "strings" +) + +// Catalog holds a collection of lint rules loaded from YAML files. +type Catalog struct { + Rules []Rule +} + +// severityOrder maps severity names to numeric priority (higher = more severe). +var severityOrder = map[string]int{ + "critical": 5, + "high": 4, + "medium": 3, + "low": 2, + "info": 1, +} + +// LoadDir loads all .yaml files from a directory path into a Catalog. +func LoadDir(dir string) (*Catalog, error) { + entries, err := os.ReadDir(dir) + if err != nil { + return nil, fmt.Errorf("load catalog dir: %w", err) + } + + cat := &Catalog{} + for _, entry := range entries { + if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".yaml") { + continue + } + data, err := os.ReadFile(filepath.Join(dir, entry.Name())) + if err != nil { + return nil, fmt.Errorf("read %s: %w", entry.Name(), err) + } + rules, err := ParseRules(data) + if err != nil { + return nil, fmt.Errorf("parse %s: %w", entry.Name(), err) + } + cat.Rules = append(cat.Rules, rules...) + } + return cat, nil +} + +// LoadFS loads all .yaml files from an embed.FS into a Catalog. +func LoadFS(fsys embed.FS, dir string) (*Catalog, error) { + cat := &Catalog{} + err := fs.WalkDir(fsys, dir, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return err + } + if d.IsDir() || !strings.HasSuffix(path, ".yaml") { + return nil + } + data, err := fsys.ReadFile(path) + if err != nil { + return fmt.Errorf("read %s: %w", path, err) + } + rules, err := ParseRules(data) + if err != nil { + return fmt.Errorf("parse %s: %w", path, err) + } + cat.Rules = append(cat.Rules, rules...) + return nil + }) + if err != nil { + return nil, err + } + return cat, nil +} + +// ForLanguage returns rules that apply to the given language. +func (c *Catalog) ForLanguage(lang string) []Rule { + var out []Rule + for _, r := range c.Rules { + if slices.Contains(r.Languages, lang) { + out = append(out, r) + } + } + return out +} + +// AtSeverity returns rules at or above the given severity threshold. +func (c *Catalog) AtSeverity(threshold string) []Rule { + minLevel := severityOrder[threshold] + var out []Rule + for _, r := range c.Rules { + if severityOrder[r.Severity] >= minLevel { + out = append(out, r) + } + } + return out +} + +// ByID returns a rule by its ID, or nil if not found. +func (c *Catalog) ByID(id string) *Rule { + for i := range c.Rules { + if c.Rules[i].ID == id { + return &c.Rules[i] + } + } + return nil +} +``` + +**Step 4: Run test to verify it passes** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v` +Expected: PASS (all tests) + +**Step 5: Commit** + +```bash +git add pkg/lint/catalog.go pkg/lint/catalog_test.go catalog/go-security.yaml +git commit -m "feat: add Catalog loader with dir/embed/filter support" +``` + +--- + +### Task 4: Regex matcher + +**Files:** +- Create: `/Users/snider/Code/core/lint/pkg/lint/matcher.go` +- Create: `/Users/snider/Code/core/lint/pkg/lint/matcher_test.go` + +**Step 1: Write the failing test** + +```go +package lint + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestMatcher_Match(t *testing.T) { + rules := []Rule{ + { + ID: "test-001", + Title: "fmt.Println usage", + Severity: "low", + Languages: []string{"go"}, + Pattern: `fmt\.Println`, + Fix: "Use structured logging", + Detection: "regex", + }, + } + m, err := NewMatcher(rules) + require.NoError(t, err) + + content := `package main + +import "fmt" + +func main() { + fmt.Println("hello") +} +` + findings := m.Match("main.go", []byte(content)) + require.Len(t, findings, 1) + assert.Equal(t, "test-001", findings[0].RuleID) + assert.Equal(t, "main.go", findings[0].File) + assert.Equal(t, 6, findings[0].Line) + assert.Contains(t, findings[0].Match, "fmt.Println") +} + +func TestMatcher_ExcludePattern(t *testing.T) { + rules := []Rule{ + { + ID: "test-002", + Title: "Println with exclude", + Severity: "low", + Languages: []string{"go"}, + Pattern: `fmt\.Println`, + ExcludePattern: `// lint:ignore`, + Fix: "Use logging", + Detection: "regex", + }, + } + m, err := NewMatcher(rules) + require.NoError(t, err) + + content := `package main +func a() { fmt.Println("bad") } +func b() { fmt.Println("ok") // lint:ignore } +` + findings := m.Match("main.go", []byte(content)) + // Line 2 matches, line 3 is excluded + assert.Len(t, findings, 1) + assert.Equal(t, 2, findings[0].Line) +} + +func TestMatcher_NoMatch(t *testing.T) { + rules := []Rule{ + {ID: "test-003", Title: "T", Severity: "low", Languages: []string{"go"}, Pattern: `NEVER_MATCH_THIS`, Detection: "regex"}, + } + m, err := NewMatcher(rules) + require.NoError(t, err) + + findings := m.Match("main.go", []byte("package main\n")) + assert.Empty(t, findings) +} + +func TestMatcher_InvalidRegex(t *testing.T) { + rules := []Rule{ + {ID: "bad", Title: "T", Severity: "low", Languages: []string{"go"}, Pattern: `[invalid`, Detection: "regex"}, + } + _, err := NewMatcher(rules) + assert.Error(t, err) +} +``` + +**Step 2: Run test to verify it fails** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestMatcher` +Expected: FAIL — `NewMatcher` not defined + +**Step 3: Write minimal implementation** + +```go +package lint + +import ( + "fmt" + "regexp" + "strings" +) + +// Finding represents a single match of a rule against source code. +type Finding struct { + RuleID string `json:"rule_id"` + Title string `json:"title"` + Severity string `json:"severity"` + File string `json:"file"` + Line int `json:"line"` + Match string `json:"match"` + Fix string `json:"fix"` + Repo string `json:"repo,omitempty"` +} + +// compiledRule is a rule with its regex pre-compiled. +type compiledRule struct { + rule Rule + pattern *regexp.Regexp + exclude *regexp.Regexp +} + +// Matcher runs compiled rules against file contents. +type Matcher struct { + rules []compiledRule +} + +// NewMatcher compiles all rule patterns and returns a Matcher. +func NewMatcher(rules []Rule) (*Matcher, error) { + compiled := make([]compiledRule, 0, len(rules)) + for _, r := range rules { + if r.Detection != "regex" { + continue // skip non-regex rules + } + p, err := regexp.Compile(r.Pattern) + if err != nil { + return nil, fmt.Errorf("rule %s: invalid pattern: %w", r.ID, err) + } + cr := compiledRule{rule: r, pattern: p} + if r.ExcludePattern != "" { + ex, err := regexp.Compile(r.ExcludePattern) + if err != nil { + return nil, fmt.Errorf("rule %s: invalid exclude_pattern: %w", r.ID, err) + } + cr.exclude = ex + } + compiled = append(compiled, cr) + } + return &Matcher{rules: compiled}, nil +} + +// Match checks file contents against all rules and returns findings. +func (m *Matcher) Match(filename string, content []byte) []Finding { + lines := strings.Split(string(content), "\n") + var findings []Finding + + for _, cr := range m.rules { + for i, line := range lines { + if !cr.pattern.MatchString(line) { + continue + } + if cr.exclude != nil && cr.exclude.MatchString(line) { + continue + } + findings = append(findings, Finding{ + RuleID: cr.rule.ID, + Title: cr.rule.Title, + Severity: cr.rule.Severity, + File: filename, + Line: i + 1, + Match: strings.TrimSpace(line), + Fix: cr.rule.Fix, + }) + } + } + return findings +} +``` + +**Step 4: Run test to verify it passes** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestMatcher` +Expected: PASS (4 tests) + +**Step 5: Commit** + +```bash +git add pkg/lint/matcher.go pkg/lint/matcher_test.go +git commit -m "feat: add regex Matcher with exclude pattern support" +``` + +--- + +### Task 5: Report output (JSON, text, JSONL) + +**Files:** +- Create: `/Users/snider/Code/core/lint/pkg/lint/report.go` +- Create: `/Users/snider/Code/core/lint/pkg/lint/report_test.go` + +**Step 1: Write the failing test** + +```go +package lint + +import ( + "bytes" + "encoding/json" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestReport_JSON(t *testing.T) { + findings := []Finding{ + {RuleID: "x-001", Title: "Test", Severity: "high", File: "a.go", Line: 10, Match: "bad code", Fix: "fix it"}, + } + var buf bytes.Buffer + require.NoError(t, WriteJSON(&buf, findings)) + + var parsed []Finding + require.NoError(t, json.Unmarshal(buf.Bytes(), &parsed)) + assert.Len(t, parsed, 1) + assert.Equal(t, "x-001", parsed[0].RuleID) +} + +func TestReport_JSONL(t *testing.T) { + findings := []Finding{ + {RuleID: "a-001", File: "a.go", Line: 1}, + {RuleID: "b-001", File: "b.go", Line: 2}, + } + var buf bytes.Buffer + require.NoError(t, WriteJSONL(&buf, findings)) + + lines := strings.Split(strings.TrimSpace(buf.String()), "\n") + assert.Len(t, lines, 2) +} + +func TestReport_Text(t *testing.T) { + findings := []Finding{ + {RuleID: "x-001", Title: "Test rule", Severity: "high", File: "main.go", Line: 42, Match: "bad()", Fix: "use good()"}, + } + var buf bytes.Buffer + WriteText(&buf, findings) + + out := buf.String() + assert.Contains(t, out, "main.go:42") + assert.Contains(t, out, "x-001") + assert.Contains(t, out, "high") +} + +func TestReport_Summary(t *testing.T) { + findings := []Finding{ + {Severity: "high"}, + {Severity: "high"}, + {Severity: "low"}, + } + s := Summarise(findings) + assert.Equal(t, 3, s.Total) + assert.Equal(t, 2, s.BySeverity["high"]) + assert.Equal(t, 1, s.BySeverity["low"]) +} +``` + +**Step 2: Run test to verify it fails** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestReport` +Expected: FAIL — functions not defined + +**Step 3: Write minimal implementation** + +```go +package lint + +import ( + "encoding/json" + "fmt" + "io" +) + +// Summary holds aggregate stats about findings. +type Summary struct { + Total int `json:"total"` + BySeverity map[string]int `json:"by_severity"` +} + +// Summarise creates a Summary from a list of findings. +func Summarise(findings []Finding) Summary { + s := Summary{ + Total: len(findings), + BySeverity: make(map[string]int), + } + for _, f := range findings { + s.BySeverity[f.Severity]++ + } + return s +} + +// WriteJSON writes findings as a JSON array. +func WriteJSON(w io.Writer, findings []Finding) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(findings) +} + +// WriteJSONL writes findings as newline-delimited JSON (one object per line). +// Compatible with ~/.core/ai/metrics/ format. +func WriteJSONL(w io.Writer, findings []Finding) error { + enc := json.NewEncoder(w) + for _, f := range findings { + if err := enc.Encode(f); err != nil { + return err + } + } + return nil +} + +// WriteText writes findings as human-readable text. +func WriteText(w io.Writer, findings []Finding) { + for _, f := range findings { + fmt.Fprintf(w, "%s:%d [%s] %s (%s)\n", f.File, f.Line, f.Severity, f.Title, f.RuleID) + if f.Fix != "" { + fmt.Fprintf(w, " fix: %s\n", f.Fix) + } + } +} +``` + +**Step 4: Run test to verify it passes** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestReport` +Expected: PASS (4 tests) + +**Step 5: Commit** + +```bash +git add pkg/lint/report.go pkg/lint/report_test.go +git commit -m "feat: add report output (JSON, JSONL, text, summary)" +``` + +--- + +### Task 6: Scanner (walk files + match) + +**Files:** +- Create: `/Users/snider/Code/core/lint/pkg/lint/scanner.go` +- Create: `/Users/snider/Code/core/lint/pkg/lint/scanner_test.go` + +**Step 1: Write the failing test** + +```go +package lint + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestScanner_ScanDir(t *testing.T) { + // Set up temp dir with a .go file containing a known pattern + dir := t.TempDir() + goFile := filepath.Join(dir, "main.go") + require.NoError(t, os.WriteFile(goFile, []byte(`package main + +import "fmt" + +func main() { + fmt.Println("hello") +} +`), 0644)) + + rules := []Rule{ + {ID: "test-001", Title: "Println", Severity: "low", Languages: []string{"go"}, Pattern: `fmt\.Println`, Fix: "log", Detection: "regex"}, + } + + s, err := NewScanner(rules) + require.NoError(t, err) + + findings, err := s.ScanDir(dir) + require.NoError(t, err) + require.Len(t, findings, 1) + assert.Equal(t, "test-001", findings[0].RuleID) +} + +func TestScanner_ScanDir_ExcludesVendor(t *testing.T) { + dir := t.TempDir() + vendor := filepath.Join(dir, "vendor") + require.NoError(t, os.MkdirAll(vendor, 0755)) + require.NoError(t, os.WriteFile(filepath.Join(vendor, "lib.go"), []byte("package lib\nfunc x() { fmt.Println() }\n"), 0644)) + + rules := []Rule{ + {ID: "test-001", Title: "Println", Severity: "low", Languages: []string{"go"}, Pattern: `fmt\.Println`, Fix: "log", Detection: "regex"}, + } + + s, err := NewScanner(rules) + require.NoError(t, err) + + findings, err := s.ScanDir(dir) + require.NoError(t, err) + assert.Empty(t, findings) +} + +func TestScanner_LanguageDetection(t *testing.T) { + assert.Equal(t, "go", DetectLanguage("main.go")) + assert.Equal(t, "php", DetectLanguage("app.php")) + assert.Equal(t, "ts", DetectLanguage("index.ts")) + assert.Equal(t, "ts", DetectLanguage("index.tsx")) + assert.Equal(t, "cpp", DetectLanguage("engine.cpp")) + assert.Equal(t, "cpp", DetectLanguage("engine.cc")) + assert.Equal(t, "", DetectLanguage("README.md")) +} +``` + +**Step 2: Run test to verify it fails** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestScanner` +Expected: FAIL — `NewScanner` not defined + +**Step 3: Write minimal implementation** + +```go +package lint + +import ( + "fmt" + "os" + "path/filepath" + "strings" +) + +// defaultExcludes are directories skipped during scanning. +var defaultExcludes = []string{"vendor", "node_modules", ".git", "testdata", ".core"} + +// extToLang maps file extensions to language identifiers. +var extToLang = map[string]string{ + ".go": "go", + ".php": "php", + ".ts": "ts", + ".tsx": "ts", + ".js": "js", + ".jsx": "js", + ".cpp": "cpp", + ".cc": "cpp", + ".cxx": "cpp", + ".c": "cpp", + ".h": "cpp", + ".hpp": "cpp", +} + +// DetectLanguage returns the language identifier for a filename, or "" if unknown. +func DetectLanguage(filename string) string { + ext := filepath.Ext(filename) + return extToLang[ext] +} + +// Scanner walks directories and matches files against rules. +type Scanner struct { + matcher *Matcher + rules []Rule + excludes []string +} + +// NewScanner creates a Scanner from a set of rules. +func NewScanner(rules []Rule) (*Scanner, error) { + m, err := NewMatcher(rules) + if err != nil { + return nil, err + } + return &Scanner{ + matcher: m, + rules: rules, + excludes: defaultExcludes, + }, nil +} + +// ScanDir walks a directory tree and returns all findings. +func (s *Scanner) ScanDir(root string) ([]Finding, error) { + var all []Finding + + err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error { + if err != nil { + return err + } + + // Skip excluded directories + if d.IsDir() { + for _, ex := range s.excludes { + if d.Name() == ex { + return filepath.SkipDir + } + } + return nil + } + + // Only scan files with known language extensions + lang := DetectLanguage(path) + if lang == "" { + return nil + } + + content, err := os.ReadFile(path) + if err != nil { + return fmt.Errorf("read %s: %w", path, err) + } + + // Make path relative to root for cleaner output + rel, err := filepath.Rel(root, path) + if err != nil { + rel = path + } + + findings := s.matcher.Match(rel, content) + all = append(all, findings...) + return nil + }) + + return all, err +} + +// ScanFile scans a single file and returns findings. +func (s *Scanner) ScanFile(path string) ([]Finding, error) { + content, err := os.ReadFile(path) + if err != nil { + return nil, fmt.Errorf("read %s: %w", path, err) + } + return s.matcher.Match(path, content), nil +} +``` + +**Step 4: Run test to verify it passes** + +Run: `cd ~/Code/core/lint && go test ./pkg/lint/ -v -run TestScanner` +Expected: PASS (3 tests) + +**Step 5: Commit** + +```bash +git add pkg/lint/scanner.go pkg/lint/scanner_test.go +git commit -m "feat: add Scanner with directory walking and language detection" +``` + +--- + +### Task 7: Seed the catalog YAML files + +**Files:** +- Create: `/Users/snider/Code/core/lint/catalog/go-security.yaml` (expand from task 3) +- Create: `/Users/snider/Code/core/lint/catalog/go-correctness.yaml` +- Create: `/Users/snider/Code/core/lint/catalog/go-modernise.yaml` + +**Step 1: Write `catalog/go-security.yaml`** + +```yaml +- id: go-sec-001 + title: "SQL wildcard injection in LIKE clauses" + severity: high + languages: [go] + tags: [security, injection, owasp-a03] + pattern: 'LIKE\s+\?.*["%`]\s*\%.*\+' + exclude_pattern: 'EscapeLike' + fix: "Use parameterised LIKE with explicit escaping of % and _ characters" + found_in: [go-store] + example_bad: | + db.Where("name LIKE ?", "%"+input+"%") + example_good: | + db.Where("name LIKE ?", EscapeLike(input)) + first_seen: "2026-03-09" + detection: regex + +- id: go-sec-002 + title: "Path traversal in file/cache key operations" + severity: high + languages: [go] + tags: [security, path-traversal, owasp-a01] + pattern: 'filepath\.Join\(.*,\s*\w+\)' + exclude_pattern: 'filepath\.Clean|securejoin|ValidatePath' + fix: "Validate path components do not contain .. before joining" + found_in: [go-cache] + example_bad: | + path := filepath.Join(cacheDir, userInput) + example_good: | + if strings.Contains(key, "..") { return ErrInvalidKey } + path := filepath.Join(cacheDir, key) + first_seen: "2026-03-09" + detection: regex + +- id: go-sec-003 + title: "XSS via unescaped HTML output" + severity: high + languages: [go] + tags: [security, xss, owasp-a03] + pattern: 'fmt\.Sprintf\(.*<.*>.*%s' + exclude_pattern: 'html\.EscapeString|template\.HTMLEscapeString' + fix: "Use html.EscapeString() for user-supplied values in HTML output" + found_in: [go-html] + example_bad: | + out := fmt.Sprintf("