diff --git a/docs/go/index.md b/docs/go/index.md index 155a047..40c70ef 100644 --- a/docs/go/index.md +++ b/docs/go/index.md @@ -1,98 +1,96 @@ -# Core Go +--- +title: Core Go Framework +description: Dependency injection and service lifecycle framework for Go. +--- -Core is a Go framework for the host-uk ecosystem - build, release, and deploy Go, Wails, PHP, and container workloads. +# Core Go Framework + +Core (`forge.lthn.ai/core/go`) is a dependency injection and service lifecycle framework for Go. It provides a typed service registry, lifecycle hooks, and a message-passing bus for decoupled communication between services. + +This is the foundation layer of the ecosystem. It has no CLI, no GUI, and minimal dependencies. ## Installation ```bash -# Via Go (recommended) -go install github.com/host-uk/core/cmd/core@latest - -# Or download binary from releases -curl -Lo core https://github.com/host-uk/core/releases/latest/download/core-$(go env GOOS)-$(go env GOARCH) -chmod +x core && sudo mv core /usr/local/bin/ - -# Verify -core doctor +go get forge.lthn.ai/core/go ``` -See [Getting Started](getting-started.md) for all installation options including building from source. +Requires Go 1.26 or later. -## Command Reference +## What It Does -See [CLI](/build/cli/) for full command documentation. +Core solves three problems that every non-trivial Go application eventually faces: -| Command | Description | -|---------|-------------| -| [go](/build/cli/go/) | Go development (test, fmt, lint, cov) | -| [php](/build/cli/php/) | Laravel/PHP development | -| [build](/build/cli/build/) | Build Go, Wails, Docker, LinuxKit projects | -| [ci](/build/cli/ci/) | Publish releases (dry-run by default) | -| [sdk](/build/cli/sdk/) | SDK generation and validation | -| [dev](/build/cli/dev/) | Multi-repo workflow + dev environment | -| [pkg](/build/cli/pkg/) | Package search and install | -| [vm](/build/cli/vm/) | LinuxKit VM management | -| [docs](/build/cli/docs/) | Documentation management | -| [setup](/build/cli/setup/) | Clone repos from registry | -| [doctor](/build/cli/doctor/) | Check development environment | +1. **Service wiring** -- how do you register, retrieve, and type-check services without import cycles? +2. **Lifecycle management** -- how do you start and stop services in the right order? +3. **Decoupled communication** -- how do services talk to each other without knowing each other's types? -## Quick Start +## Packages -```bash -# Go development -core go test # Run tests -core go test --coverage # With coverage -core go fmt # Format code -core go lint # Lint code +| Package | Purpose | +|---------|---------| +| [`pkg/core`](services.md) | DI container, service registry, lifecycle, message bus | +| `pkg/log` | Structured logger service with Core integration | -# Build -core build # Auto-detect and build -core build --targets linux/amd64,darwin/arm64 +## Quick Example -# Release (dry-run by default) -core ci # Preview release -core ci --we-are-go-for-launch # Actually publish +```go +package main -# Multi-repo workflow -core dev work # Status + commit + push -core dev work --status # Just show status +import ( + "context" + "fmt" -# PHP development -core php dev # Start dev environment -core php test # Run tests + "forge.lthn.ai/core/go/pkg/core" + "forge.lthn.ai/core/go/pkg/log" +) + +func main() { + c, err := core.New( + core.WithName("log", log.NewService(log.Options{Level: log.LevelInfo})), + core.WithServiceLock(), // Prevent late registration + ) + if err != nil { + panic(err) + } + + // Start all services + if err := c.ServiceStartup(context.Background(), nil); err != nil { + panic(err) + } + + // Type-safe retrieval + logger, err := core.ServiceFor[*log.Service](c, "log") + if err != nil { + panic(err) + } + fmt.Println("Log level:", logger.Level()) + + // Shut down (reverse order) + _ = c.ServiceShutdown(context.Background()) +} ``` -## Configuration +## Documentation -Core uses `.core/` directory for project configuration: +| Page | Covers | +|------|--------| +| [Getting Started](getting-started.md) | Creating a Core app, registering your first service | +| [Services](services.md) | Service registration, `ServiceRuntime`, factory pattern | +| [Lifecycle](lifecycle.md) | `Startable`/`Stoppable` interfaces, startup/shutdown order | +| [Messaging](messaging.md) | ACTION, QUERY, PERFORM -- the message bus | +| [Configuration](configuration.md) | `WithService`, `WithName`, `WithAssets`, `WithServiceLock` options | +| [Testing](testing.md) | Test naming conventions, test helpers, fuzz testing | +| [Errors](errors.md) | `E()` helper, `Error` struct, unwrapping | -``` -.core/ -├── release.yaml # Release targets and settings -├── build.yaml # Build configuration (optional) -└── linuxkit/ # LinuxKit templates -``` +## Dependencies -And `repos.yaml` in workspace root for multi-repo management. +Core is deliberately minimal: -## Guides +- `forge.lthn.ai/core/go-io` -- abstract storage (local, S3, SFTP, WebDAV) +- `forge.lthn.ai/core/go-log` -- structured logging +- `github.com/stretchr/testify` -- test assertions (test-only) -- [Getting Started](getting-started.md) - Installation and first steps -- [Workflows](workflows.md) - Common task sequences -- [Troubleshooting](troubleshooting.md) - When things go wrong -- [Migration](migration.md) - Moving from legacy tools +## Licence -## Reference - -- [Configuration](configuration.md) - All config options -- [Glossary](glossary.md) - Term definitions - -## Claude Code Skill - -Install the skill to teach Claude Code how to use the Core CLI: - -```bash -curl -fsSL https://raw.githubusercontent.com/host-uk/core/main/.claude/skills/core/install.sh | bash -``` - -See [skill/](skill/) for details. +EUPL-1.2 diff --git a/docs/go/packages/go-ai.md b/docs/go/packages/go-ai.md index 7698acc..caa7b26 100644 --- a/docs/go/packages/go-ai.md +++ b/docs/go/packages/go-ai.md @@ -1,36 +1,130 @@ +--- +title: go-ai Overview +description: The AI integration hub for the Lethean Go ecosystem — MCP server, metrics, and facade. +--- + # go-ai -MCP (Model Context Protocol) hub for the Lethean AI stack. - **Module**: `forge.lthn.ai/core/go-ai` +**Language**: Go 1.26 +**Licence**: EUPL-1.2 -Exposes 49 tools across file operations, directory management, language detection, RAG vector search, ML inference and scoring, process management, WebSocket streaming, browser automation via Chrome DevTools Protocol, JSONL metrics, and an IDE bridge to the Laravel core-agentic backend. The package is a pure library — the Core CLI (`core mcp serve`) imports it and handles transport selection (stdio, TCP, or Unix socket). +go-ai is the **integration hub** for the Lethean AI stack. It imports specialised modules and exposes them as a unified MCP server with IDE bridge support, metrics recording, and a thin AI facade. + +## Architecture + +``` +AI Clients (Claude, Cursor, any MCP-capable IDE) + | MCP JSON-RPC (stdio / TCP / Unix) + v + [ go-ai MCP Server ] <-- this module + | | | + | | +-- ide/ subsystem --> Laravel core-agentic (WebSocket) + | +-- go-rag -----------------> Qdrant + Ollama + +-- go-ml ---------------------------> inference backends (go-mlx, go-rocm, ...) + + Core CLI (forge.lthn.ai/core/cli) bootstraps and wires everything +``` + +go-ai is a pure library module. It contains no `main` package. The Core CLI (`core mcp serve`) imports `forge.lthn.ai/core/go-ai/mcp`, constructs a `mcp.Service`, and calls `Run()`. + +## Package Layout + +``` +go-ai/ ++-- ai/ # AI facade: RAG queries and JSONL metrics +| +-- ai.go # Package documentation and composition overview +| +-- rag.go # QueryRAGForTask() with graceful degradation +| +-- metrics.go # Event, Record(), ReadEvents(), Summary() +| ++-- cmd/ # CLI command registrations +| +-- daemon/ # core daemon (MCP server lifecycle) +| +-- metrics/ # core ai metrics viewer +| +-- rag/ # re-exports go-rag CLI commands +| +-- security/ # security scanning tools (deps, alerts, secrets, scan, jobs) +| +-- lab/ # homelab monitoring dashboard +| +-- embed-bench/ # embedding model benchmark utility +| ++-- docs/ # This documentation +``` + +The MCP server and all its tool subsystems are provided by the separate `forge.lthn.ai/core/mcp` module. go-ai wires that server together with the `ai/` facade and the CLI command registrations. + +## Imported Modules + +| Module | Purpose | +|--------|---------| +| `forge.lthn.ai/core/go-ml` | Inference backends, scoring engine | +| `forge.lthn.ai/core/go-rag` | Vector search, embeddings | +| `forge.lthn.ai/core/go-inference` | Shared TextModel/Backend interfaces | +| `forge.lthn.ai/core/go-process` | Process lifecycle management | +| `forge.lthn.ai/core/go-log` | Structured logging with security levels | +| `forge.lthn.ai/core/go-io` | Sandboxed filesystem abstraction | +| `forge.lthn.ai/core/go-i18n` | Internationalisation | ## Quick Start -```go -import "forge.lthn.ai/core/go-ai/mcp" +go-ai is not run directly. It is consumed by the Core CLI: -svc, err := mcp.New( - mcp.WithWorkspaceRoot("/path/to/project"), - mcp.WithProcessService(ps), -) -// Run as stdio server (default for AI client subprocess integration) -err = svc.Run(ctx) -// Or TCP: MCP_ADDR=127.0.0.1:9100 triggers ServeTCP automatically +```bash +# Start the MCP server on stdio (default) +core mcp serve + +# Start on TCP +core mcp serve --mcp-transport tcp --mcp-addr 127.0.0.1:9100 + +# Run as a background daemon +core daemon start + +# View AI metrics +core ai metrics --since 7d ``` -## Tool Categories +## Documentation -| Category | Tools | Description | -|----------|-------|-------------| -| File | 8 | Read, write, search, glob, patch files | -| Directory | 4 | List, create, move, tree | -| Language | 3 | Detect, grammar, translate | -| RAG | 5 | Ingest, search, embed, index, stats | -| ML | 6 | Generate, score, probe, model management | -| Process | 4 | Start, stop, list, logs | -| WebSocket | 3 | Connect, send, subscribe | -| Browser | 5 | Navigate, click, read, screenshot, evaluate | -| Metrics | 3 | Write, query, dashboard | -| IDE | 8 | Bridge to core-agentic Laravel backend | +| Page | Description | +|------|-------------| +| [MCP Server](mcp-server.md) | Protocol implementation, transports, tool registration | +| [ML Pipeline](ml-pipeline.md) | ML scoring, model management, inference backends | +| [RAG Pipeline](rag.md) | Retrieval-augmented generation, vector search | +| [Agentic Client](agentic.md) | Security scanning, metrics, CLI commands | +| [IDE Bridge](ide-bridge.md) | IDE integration, WebSocket bridge to Laravel | + +## Build and Test + +```bash +go test ./... # Run all tests +go test -run TestName ./... # Run a single test +go test -v -race ./... # Verbose with race detector +go build ./... # Verify compilation (library -- no binary) +go vet ./... # Vet +``` + +Tests follow the `_Good`, `_Bad`, `_Ugly` suffix convention: + +- `_Good` -- Happy path, valid input +- `_Bad` -- Expected error conditions +- `_Ugly` -- Panics and edge cases + +## Dependencies + +### Direct + +| Module | Role | +|--------|------| +| `forge.lthn.ai/core/cli` | CLI framework (cobra-based command registration) | +| `forge.lthn.ai/core/go-api` | API server framework | +| `forge.lthn.ai/core/go-i18n` | Internationalisation strings | +| `forge.lthn.ai/core/go-inference` | Shared inference interfaces | +| `forge.lthn.ai/core/go-io` | Filesystem abstraction | +| `forge.lthn.ai/core/go-log` | Structured logging | +| `forge.lthn.ai/core/go-ml` | ML scoring and inference | +| `forge.lthn.ai/core/go-process` | Process lifecycle | +| `forge.lthn.ai/core/go-rag` | RAG pipeline | +| `github.com/modelcontextprotocol/go-sdk` | MCP Go SDK | +| `github.com/gorilla/websocket` | WebSocket client (IDE bridge) | +| `github.com/gin-gonic/gin` | HTTP router | + +### Indirect (via go-ml and go-rag) + +`go-mlx`, `go-rocm`, `go-duckdb`, `parquet-go`, `ollama`, `qdrant/go-client`, and the Arrow ecosystem are transitive dependencies not imported directly by go-ai. diff --git a/docs/go/packages/go-ansible.md b/docs/go/packages/go-ansible.md index dc70c92..5f77363 100644 --- a/docs/go/packages/go-ansible.md +++ b/docs/go/packages/go-ansible.md @@ -1,15 +1,159 @@ --- title: go-ansible -description: Ansible executor for programmatic playbook and ad-hoc command execution +description: A pure Go Ansible playbook engine -- parses YAML playbooks, inventories, and roles, then executes tasks on remote hosts via SSH without requiring Python. --- # go-ansible -`forge.lthn.ai/core/go-ansible` +`forge.lthn.ai/core/go-ansible` is a pure Go implementation of an Ansible playbook engine. It parses standard Ansible YAML playbooks, inventories, and roles, then executes tasks against remote hosts over SSH -- with no dependency on Python or the upstream `ansible-playbook` binary. -Go wrapper around Ansible for executing playbooks and ad-hoc commands programmatically. Provides a clean API for running Ansible operations with structured output parsing and SSH client abstraction for testing. +## Module Path -## Key Types +``` +forge.lthn.ai/core/go-ansible +``` -- `Executor` — runs Ansible playbooks and ad-hoc commands, captures structured output -- `MockSSHClient` — test double for SSH connections, enables unit testing without live hosts +Requires **Go 1.26+**. + +## Quick Start + +### As a Library + +```go +package main + +import ( + "context" + "fmt" + + ansible "forge.lthn.ai/core/go-ansible" +) + +func main() { + // Create an executor rooted at the playbook directory + executor := ansible.NewExecutor("/path/to/project") + defer executor.Close() + + // Load inventory + if err := executor.SetInventory("/path/to/inventory.yml"); err != nil { + panic(err) + } + + // Optionally set extra variables + executor.SetVar("deploy_version", "1.2.3") + + // Optionally limit to specific hosts + executor.Limit = "web1" + + // Set up callbacks for progress reporting + executor.OnTaskStart = func(host string, task *ansible.Task) { + fmt.Printf("TASK [%s] on %s\n", task.Name, host) + } + executor.OnTaskEnd = func(host string, task *ansible.Task, result *ansible.TaskResult) { + if result.Failed { + fmt.Printf(" FAILED: %s\n", result.Msg) + } else if result.Changed { + fmt.Printf(" changed\n") + } else { + fmt.Printf(" ok\n") + } + } + + // Run the playbook + ctx := context.Background() + if err := executor.Run(ctx, "/path/to/playbook.yml"); err != nil { + panic(err) + } +} +``` + +### As a CLI Command + +The package ships with a CLI integration under `cmd/ansible/` that registers a `core ansible` subcommand: + +```bash +# Run a playbook +core ansible playbooks/deploy.yml -i inventory/production.yml + +# Limit to a single host +core ansible site.yml -l web1 + +# Pass extra variables +core ansible deploy.yml -e "version=1.2.3" -e "env=prod" + +# Dry run (check mode) +core ansible deploy.yml --check + +# Increase verbosity +core ansible deploy.yml -vvv + +# Test SSH connectivity to a host +core ansible test server.example.com -u root -i ~/.ssh/id_ed25519 +``` + +**CLI flags:** + +| Flag | Short | Description | +|------|-------|-------------| +| `--inventory` | `-i` | Inventory file or directory | +| `--limit` | `-l` | Restrict execution to matching hosts | +| `--tags` | `-t` | Only run tasks tagged with these values (comma-separated) | +| `--skip-tags` | | Skip tasks tagged with these values | +| `--extra-vars` | `-e` | Set additional variables (`key=value`, repeatable) | +| `--verbose` | `-v` | Increase verbosity (stack for more: `-vvv`) | +| `--check` | | Dry-run mode -- no changes are made | + +## Package Layout + +``` +go-ansible/ + types.go Core data types: Playbook, Play, Task, Inventory, Host, Facts + parser.go YAML parser for playbooks, inventories, tasks, roles + executor.go Execution engine: module dispatch, templating, conditions, loops + modules.go 41 module implementations (shell, apt, docker-compose, etc.) + ssh.go SSH client with key/password auth, become/sudo, file transfer + types_test.go Tests for data types and YAML unmarshalling + parser_test.go Tests for the YAML parser + executor_test.go Tests for the executor engine + ssh_test.go Tests for SSH client construction + mock_ssh_test.go Mock SSH infrastructure for module tests + modules_*_test.go Module-specific tests (cmd, file, svc, infra, adv) + cmd/ + ansible/ + cmd.go CLI command registration + ansible.go CLI implementation (flags, callbacks, output formatting) +``` + +## Supported Modules + +41 module handlers are implemented, covering the most commonly used Ansible modules: + +| Category | Modules | +|----------|---------| +| **Command execution** | `shell`, `command`, `raw`, `script` | +| **File operations** | `copy`, `template`, `file`, `lineinfile`, `blockinfile`, `stat`, `slurp`, `fetch`, `get_url` | +| **Package management** | `apt`, `apt_key`, `apt_repository`, `package`, `pip` | +| **Service management** | `service`, `systemd` | +| **User and group** | `user`, `group` | +| **HTTP** | `uri` | +| **Source control** | `git` | +| **Archive** | `unarchive` | +| **System** | `hostname`, `sysctl`, `cron`, `reboot`, `setup` | +| **Flow control** | `debug`, `fail`, `assert`, `set_fact`, `pause`, `wait_for`, `meta`, `include_vars` | +| **Community** | `community.general.ufw`, `ansible.posix.authorized_key`, `community.docker.docker_compose` | + +Both fully-qualified collection names (e.g. `ansible.builtin.shell`) and short-form names (e.g. `shell`) are accepted. + +## Dependencies + +| Module | Purpose | +|--------|---------| +| `forge.lthn.ai/core/cli` | CLI framework (command registration, flags, styled output) | +| `forge.lthn.ai/core/go-log` | Structured logging and contextual error helper (`log.E()`) | +| `golang.org/x/crypto` | SSH protocol implementation (`crypto/ssh`, `crypto/ssh/knownhosts`) | +| `gopkg.in/yaml.v3` | YAML parsing for playbooks, inventories, and role files | +| `github.com/stretchr/testify` | Test assertions (test-only) | + +## Licence + +EUPL-1.2 diff --git a/docs/go/packages/go-api.md b/docs/go/packages/go-api.md index cd97286..3dec037 100644 --- a/docs/go/packages/go-api.md +++ b/docs/go/packages/go-api.md @@ -1,16 +1,173 @@ --- title: go-api -description: REST framework with OpenAPI SDK generation, built on Gin +description: Gin-based REST framework with OpenAPI generation, middleware composition, and SDK codegen for the Lethean Go ecosystem. --- + + # go-api -`forge.lthn.ai/core/go-api` +**Module path:** `forge.lthn.ai/core/go-api` +**Language:** Go 1.26 +**Licence:** EUPL-1.2 -HTTP REST framework built on Gin with automatic OpenAPI spec generation and client SDK output. Provides a route group interface for modular endpoint registration and includes health check endpoints out of the box. +go-api is a REST framework built on top of [Gin](https://github.com/gin-gonic/gin). It provides +an `Engine` that subsystems plug into via the `RouteGroup` interface. Each ecosystem package +(go-ai, go-ml, go-rag, and others) registers its own route group, and go-api handles the HTTP +plumbing: middleware composition, response envelopes, WebSocket and SSE integration, GraphQL +hosting, Authentik identity, OpenAPI 3.1 specification generation, and client SDK codegen. -## Key Types +go-api is a library. It has no `main` package and produces no binary on its own. Callers +construct an `Engine`, register route groups, and call `Serve()`. -- `Engine` — HTTP server wrapping Gin with OpenAPI integration and middleware pipeline -- `healthGroup` — built-in health check route group for liveness and readiness probes -- `RouteGroup` — interface for modular route registration +--- + +## Quick Start + +```go +package main + +import ( + "context" + "os/signal" + "syscall" + + api "forge.lthn.ai/core/go-api" +) + +func main() { + engine, _ := api.New( + api.WithAddr(":8080"), + api.WithBearerAuth("my-secret-token"), + api.WithCORS("*"), + api.WithRequestID(), + api.WithSecure(), + api.WithSlog(nil), + api.WithSwagger("My API", "A service description", "1.0.0"), + ) + + engine.Register(myRoutes) // any RouteGroup implementation + + ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) + defer stop() + + _ = engine.Serve(ctx) // blocks until context is cancelled, then shuts down gracefully +} +``` + +The default listen address is `:8080`. A built-in `GET /health` endpoint is always present. +Every feature beyond panic recovery requires an explicit `With*()` option. + +--- + +## Implementing a RouteGroup + +Any type that satisfies the `RouteGroup` interface can register endpoints: + +```go +type Routes struct{ service *mypackage.Service } + +func (r *Routes) Name() string { return "mypackage" } +func (r *Routes) BasePath() string { return "/v1/mypackage" } + +func (r *Routes) RegisterRoutes(rg *gin.RouterGroup) { + rg.GET("/items", r.listItems) + rg.POST("/items", r.createItem) +} + +func (r *Routes) listItems(c *gin.Context) { + items, _ := r.service.List(c.Request.Context()) + c.JSON(200, api.OK(items)) +} +``` + +Register with the engine: + +```go +engine.Register(&Routes{service: svc}) +``` + +--- + +## Package Layout + +| File | Purpose | +|------|---------| +| `api.go` | `Engine` struct, `New()`, `build()`, `Serve()`, `Handler()`, `Channels()` | +| `options.go` | All `With*()` option functions (25 options) | +| `group.go` | `RouteGroup`, `StreamGroup`, `DescribableGroup` interfaces; `RouteDescription` | +| `response.go` | `Response[T]`, `Error`, `Meta`, `OK()`, `Fail()`, `FailWithDetails()`, `Paginated()` | +| `middleware.go` | `bearerAuthMiddleware()`, `requestIDMiddleware()` | +| `authentik.go` | `AuthentikUser`, `AuthentikConfig`, `GetUser()`, `RequireAuth()`, `RequireGroup()` | +| `websocket.go` | `wrapWSHandler()` helper | +| `sse.go` | `SSEBroker`, `NewSSEBroker()`, `Publish()`, `Handler()`, `Drain()`, `ClientCount()` | +| `cache.go` | `cacheStore`, `cacheEntry`, `cacheWriter`, `cacheMiddleware()` | +| `brotli.go` | `brotliHandler`, `newBrotliHandler()`; compression level constants | +| `graphql.go` | `graphqlConfig`, `GraphQLOption`, `WithPlayground()`, `WithGraphQLPath()`, `mountGraphQL()` | +| `i18n.go` | `I18nConfig`, `WithI18n()`, `i18nMiddleware()`, `GetLocale()`, `GetMessage()` | +| `tracing.go` | `WithTracing()`, `NewTracerProvider()` | +| `swagger.go` | `swaggerSpec`, `registerSwagger()`; sequence counter for multi-instance safety | +| `openapi.go` | `SpecBuilder`, `Build()`, `buildPaths()`, `buildTags()`, `envelopeSchema()` | +| `export.go` | `ExportSpec()`, `ExportSpecToFile()` | +| `bridge.go` | `ToolDescriptor`, `ToolBridge`, `NewToolBridge()`, `Add()`, `Describe()`, `Tools()` | +| `codegen.go` | `SDKGenerator`, `Generate()`, `Available()`, `SupportedLanguages()` | +| `cmd/api/` | CLI subcommands: `core api spec` and `core api sdk` | + +--- + +## Dependencies + +### Direct + +| Module | Role | +|--------|------| +| `github.com/gin-gonic/gin` | HTTP router and middleware engine | +| `github.com/gin-contrib/cors` | CORS policy middleware | +| `github.com/gin-contrib/secure` | Security headers (HSTS, X-Frame-Options, nosniff) | +| `github.com/gin-contrib/gzip` | Gzip response compression | +| `github.com/gin-contrib/slog` | Structured request logging via `log/slog` | +| `github.com/gin-contrib/timeout` | Per-request deadline enforcement | +| `github.com/gin-contrib/static` | Static file serving | +| `github.com/gin-contrib/sessions` | Cookie-backed server sessions | +| `github.com/gin-contrib/authz` | Casbin policy-based authorisation | +| `github.com/gin-contrib/httpsign` | HTTP Signatures verification | +| `github.com/gin-contrib/location/v2` | Reverse proxy header detection | +| `github.com/gin-contrib/pprof` | Go profiling endpoints | +| `github.com/gin-contrib/expvar` | Runtime metrics endpoint | +| `github.com/casbin/casbin/v2` | Policy-based access control engine | +| `github.com/coreos/go-oidc/v3` | OIDC provider discovery and JWT validation | +| `github.com/andybalholm/brotli` | Brotli compression | +| `github.com/gorilla/websocket` | WebSocket upgrade support | +| `github.com/swaggo/gin-swagger` | Swagger UI handler | +| `github.com/swaggo/files` | Swagger UI static assets | +| `github.com/swaggo/swag` | Swagger spec registry | +| `github.com/99designs/gqlgen` | GraphQL schema execution (gqlgen) | +| `go.opentelemetry.io/otel` | OpenTelemetry tracing SDK | +| `go.opentelemetry.io/contrib/.../otelgin` | OpenTelemetry Gin instrumentation | +| `golang.org/x/text` | BCP 47 language tag matching | +| `gopkg.in/yaml.v3` | YAML export of OpenAPI specs | +| `forge.lthn.ai/core/cli` | CLI command registration (for `cmd/api/` subcommands) | + +### Ecosystem position + +go-api sits at the base of the Lethean HTTP stack. It has no imports from other Lethean +ecosystem modules (beyond `core/cli` for the CLI subcommands). Other packages import go-api +to expose their functionality as REST endpoints: + +``` +Application main / Core CLI + | + v + go-api Engine <-- this module + | | | + | | +-- OpenAPI spec --> SDKGenerator --> openapi-generator-cli + | +-- ToolBridge --> go-ai / go-ml / go-rag route groups + +-- RouteGroups ----------> any package implementing RouteGroup +``` + +--- + +## Further Reading + +- [Architecture](architecture.md) -- internals, key types, data flow, middleware stack +- [Development](development.md) -- building, testing, contributing, coding standards diff --git a/docs/go/packages/go-blockchain.md b/docs/go/packages/go-blockchain.md index c225189..3543414 100644 --- a/docs/go/packages/go-blockchain.md +++ b/docs/go/packages/go-blockchain.md @@ -1,42 +1,132 @@ -# go-blockchain +--- +title: Lethean Go Blockchain +description: Pure Go implementation of the Lethean CryptoNote/Zano-fork blockchain protocol. +--- -Pure Go implementation of the Lethean blockchain protocol. +# Lethean Go Blockchain -**Module**: `forge.lthn.ai/core/go-blockchain` +`go-blockchain` is a Go reimplementation of the Lethean blockchain protocol. It provides pure-Go implementations of chain logic, data structures, consensus rules, wallet operations, and networking, delegating only mathematically complex cryptographic operations (ring signatures, Bulletproofs+, Zarcanum proofs) to a cleaned C++ library via CGo. -Provides chain configuration, core cryptographic data types, CryptoNote wire serialisation, and LWMA difficulty adjustment for the Lethean CryptoNote/Zano-fork chain. Lineage: CryptoNote to IntenseCoin (2017) to Lethean to Zano rebase. +**Module path:** `forge.lthn.ai/core/go-blockchain` + +**Licence:** [European Union Public Licence (EUPL) version 1.2](https://joinup.ec.europa.eu/software/page/eupl/licence-eupl) + +## Lineage + +``` +CryptoNote (van Saberhagen, 2013) + | +IntenseCoin (2017) + | +Lethean (2017-present) + | +Zano rebase (2025) -- privacy upgrades: Zarcanum, CLSAG, Bulletproofs+, confidential assets + | +go-blockchain -- Go reimplementation of the Zano-fork protocol +``` + +The Lethean mainnet launched on **2026-02-12** with genesis timestamp `1770897600` (12:00 UTC). The chain runs a hybrid PoW/PoS consensus with 120-second block targets. + +## Package Structure + +``` +go-blockchain/ + config/ Chain parameters (mainnet/testnet), hardfork schedule + types/ Core data types: Hash, PublicKey, Address, Block, Transaction + wire/ Binary serialisation (consensus-critical, bit-identical to C++) + crypto/ CGo bridge to libcryptonote (ring sigs, BP+, Zarcanum, stealth) + difficulty/ PoW + PoS difficulty adjustment (LWMA variant) + consensus/ Three-layer block/transaction validation + chain/ Blockchain storage, block/tx validation, mempool + p2p/ Levin TCP protocol, peer discovery, handshake + rpc/ Daemon and wallet JSON-RPC client + wallet/ Key management, output scanning, tx construction + mining/ Solo PoW miner (RandomX nonce grinding) + tui/ Terminal dashboard (bubbletea + lipgloss) +``` + +## Design Principles + +1. **Consensus-critical code must be bit-identical** to the C++ implementation. The `wire/` package produces exactly the same binary output as the C++ serialisation for the same input. + +2. **No global state.** Chain parameters are passed via `config.ChainConfig` structs, not package-level globals. `Mainnet` and `Testnet` are pre-defined instances. + +3. **Interfaces at boundaries.** The `chain/` package defines interfaces for storage backends; the `wallet/` package uses Scanner, Signer, Builder, and RingSelector interfaces for v1/v2+ extensibility. + +4. **Test against real chain data.** Wherever possible, tests use actual mainnet block and transaction hex blobs as test vectors, ensuring compatibility with the C++ node. ## Quick Start ```go import ( + "fmt" + "forge.lthn.ai/core/go-blockchain/config" + "forge.lthn.ai/core/go-blockchain/rpc" "forge.lthn.ai/core/go-blockchain/types" - "forge.lthn.ai/core/go-blockchain/wire" - "forge.lthn.ai/core/go-blockchain/difficulty" ) -// Query the active hardfork version at a given block height -version := config.VersionAtHeight(config.MainnetForks, 10081) // returns HF2 +// Query the daemon +client := rpc.NewClient("http://localhost:36941") +info, err := client.GetInfo() +if err != nil { + panic(err) +} +fmt.Printf("Height: %d, Difficulty: %d\n", info.Height, info.PowDifficulty) -// Encode and decode a Lethean address -addr := &types.Address{SpendPublicKey: spendKey, ViewPublicKey: viewKey} -encoded := addr.Encode(config.AddressPrefix) -decoded, prefix, err := types.DecodeAddress(encoded) +// Decode an address +addr, prefix, err := types.DecodeAddress("iTHN...") +if err != nil { + panic(err) +} +fmt.Printf("Spend key: %s\n", addr.SpendPublicKey) +fmt.Printf("Auditable: %v\n", addr.IsAuditable()) -// Varint encoding for the wire protocol -buf := wire.EncodeVarint(0x1eaf7) -val, n, err := wire.DecodeVarint(buf) - -// Calculate next block difficulty -nextDiff := difficulty.NextDifficulty(timestamps, cumulativeDiffs, 120) +// Check hardfork version at a given height +version := config.VersionAtHeight(config.MainnetForks, 15000) +fmt.Printf("Active hardfork at height 15000: HF%d\n", version) ``` -## Packages +## CGo Boundary -| Package | Description | -|---------|-------------| -| `config` | Hardfork schedule, network parameters, address prefixes | -| `types` | Addresses, keys, hashes, amounts | -| `wire` | CryptoNote binary serialisation, varint codec | -| `difficulty` | LWMA difficulty adjustment algorithm | +The `crypto/` package is the **only** package that crosses the CGo boundary. All other packages are pure Go. + +``` +Go side C++ side (libcryptonote + librandomx) ++---------+ +---------------------------+ +| crypto/ | --- CGo calls ---> | cn_fast_hash() | +| | | generate_key_derivation | +| | | generate_key_image | +| | | check_ring_signature | +| | | CLSAG_GG/GGX/GGXXG_verify| +| | | bulletproof_plus_verify | +| | | zarcanum_verify | +| | | randomx_hash | ++---------+ +---------------------------+ +``` + +When CGo is disabled, stub implementations return errors, allowing the rest of the codebase to compile and run tests that do not require real cryptographic operations. + +## Development Phases + +The project follows a 9-phase development plan. See the [wiki Development Phases page](https://forge.lthn.ai/core/go-blockchain/wiki/Development-Phases) for detailed phase descriptions. + +| Phase | Scope | Status | +|-------|-------|--------| +| 0 | Config + Types | Complete | +| 1 | Wire Serialisation | Complete | +| 2 | CGo Crypto Bridge | Complete | +| 3 | P2P Protocol | Complete | +| 4 | RPC Client | Complete | +| 5 | Chain Storage | Complete | +| 6 | Wallet Core | Complete | +| 7 | Consensus Rules | Complete | +| 8 | Mining | Complete | + +## Further Reading + +- [Architecture](architecture.md) -- Package dependencies, CGo boundary, data structures +- [Cryptography](cryptography.md) -- Crypto primitives, hashing, signatures, proofs +- [Networking](networking.md) -- P2P protocol, peer discovery, message types +- [RPC Reference](rpc.md) -- Daemon and wallet JSON-RPC API +- [Chain Parameters](parameters.md) -- Tokenomics, emission, hardfork schedule diff --git a/docs/go/packages/go-build.md b/docs/go/packages/go-build.md index 5e1d3ce..e33888a 100644 --- a/docs/go/packages/go-build.md +++ b/docs/go/packages/go-build.md @@ -1,16 +1,208 @@ --- title: go-build -description: Build system, release publishers, and SDK generation +description: Build system, release pipeline, and SDK generation for the Core ecosystem. --- # go-build -`forge.lthn.ai/core/go-build` +`forge.lthn.ai/core/go-build` is the build, release, and SDK generation toolkit for Core projects. It provides: -Build system that auto-detects project types (Go, Wails, Docker, LinuxKit, C++) and produces cross-compiled binaries, archives, and checksums. Includes release publishers for GitHub, Docker registries, npm, Homebrew, Scoop, AUR, and Chocolatey. Also provides SDK generation for API clients. +- **Auto-detecting builders** for Go, Wails, Docker, LinuxKit, C++, and Taskfile projects +- **Cross-compilation** with per-target archiving (tar.gz, tar.xz, zip) and SHA-256 checksums +- **Code signing** -- macOS codesign with notarisation, GPG detached signatures, Windows signtool (placeholder) +- **Release automation** -- semantic versioning from git tags, conventional-commit changelogs, multi-target publishing +- **SDK generation** -- OpenAPI spec diffing for breaking-change detection, code generation for TypeScript, Python, Go, and PHP +- **CLI integration** -- registers `core build`, `core ci`, and `core sdk` commands via the Core CLI framework -## Packages +## Module Path -- `pkg/build/` — project-type detection, cross-compilation, archiving -- `pkg/release/` — changelog generation, publisher pipeline -- `pkg/sdk/` — client SDK generation from OpenAPI specs +``` +forge.lthn.ai/core/go-build +``` + +Requires **Go 1.26+**. + +## Quick Start + +### Build a project + +From any project directory containing a recognised marker file: + +```bash +core build # Auto-detect type, build for configured targets +core build --targets linux/amd64 # Single target +core build --ci # JSON output for CI pipelines +core build --verbose # Detailed step-by-step output +``` + +The builder is chosen by marker-file priority: + +| Marker file | Builder | +|-------------------|------------| +| `wails.json` | Wails | +| `go.mod` | Go | +| `package.json` | Node (stub)| +| `composer.json` | PHP (stub) | +| `CMakeLists.txt` | C++ | +| `Dockerfile` | Docker | +| `linuxkit.yml` | LinuxKit | +| `Taskfile.yml` | Taskfile | + +### Release artifacts + +```bash +core build release --we-are-go-for-launch # Build + archive + checksum + publish +core build release # Dry-run (default without the flag) +core build release --draft --prerelease # Mark as draft pre-release +``` + +### Publish pre-built artifacts + +After `core build` has populated `dist/`: + +```bash +core ci # Dry-run publish from dist/ +core ci --we-are-go-for-launch # Actually publish +core ci --version v1.2.3 # Override version +``` + +### Generate changelogs + +```bash +core ci changelog # From latest tag to HEAD +core ci changelog --from v0.1.0 --to v0.2.0 +core ci version # Show determined next version +core ci init # Scaffold .core/release.yaml +``` + +### SDK operations + +```bash +core build sdk # Generate SDKs for all configured languages +core build sdk --lang typescript # Single language +core sdk diff --base v1.0.0 --spec api/openapi.yaml # Breaking-change check +core sdk validate # Validate OpenAPI spec +``` + +## Package Layout + +``` +forge.lthn.ai/core/go-build/ +| +|-- cmd/ +| |-- build/ CLI commands for `core build` (build, from-path, pwa, sdk, release) +| |-- ci/ CLI commands for `core ci` (init, changelog, version, publish) +| +-- sdk/ CLI commands for `core sdk` (diff, validate) +| ++-- pkg/ + |-- build/ Core build types, config loading, discovery, archiving, checksums + | |-- builders/ Builder implementations (Go, Wails, Docker, LinuxKit, C++, Taskfile) + | +-- signing/ Code-signing implementations (macOS codesign, GPG, Windows stub) + | + |-- release/ Release orchestration, versioning, changelog, config + | +-- publishers/ Publisher implementations (GitHub, Docker, npm, Homebrew, Scoop, AUR, Chocolatey, LinuxKit) + | + +-- sdk/ OpenAPI SDK generation and breaking-change diffing + +-- generators/ Language generators (TypeScript, Python, Go, PHP) +``` + +## Configuration Files + +Build and release behaviour is driven by two YAML files in the `.core/` directory. + +### `.core/build.yaml` + +Controls compilation targets, flags, and signing: + +```yaml +version: 1 +project: + name: myapp + description: My application + main: ./cmd/myapp + binary: myapp +build: + cgo: false + flags: ["-trimpath"] + ldflags: ["-s", "-w"] + env: [] +targets: + - os: linux + arch: amd64 + - os: linux + arch: arm64 + - os: darwin + arch: arm64 + - os: windows + arch: amd64 +sign: + enabled: true + gpg: + key: $GPG_KEY_ID + macos: + identity: $CODESIGN_IDENTITY + notarize: false + apple_id: $APPLE_ID + team_id: $APPLE_TEAM_ID + app_password: $APPLE_APP_PASSWORD +``` + +When no `.core/build.yaml` exists, sensible defaults apply (CGO off, `-trimpath -s -w`, four standard targets). + +### `.core/release.yaml` + +Controls versioning, changelog filtering, publishers, and SDK generation: + +```yaml +version: 1 +project: + name: myapp + repository: owner/repo +build: + targets: + - os: linux + arch: amd64 + - os: darwin + arch: arm64 +publishers: + - type: github + draft: false + prerelease: false + - type: homebrew + tap: owner/homebrew-tap + - type: docker + registry: ghcr.io + image: owner/myapp + tags: ["latest", "{{.Version}}"] +changelog: + include: [feat, fix, perf, refactor] + exclude: [chore, docs, style, test, ci] +sdk: + spec: api/openapi.yaml + languages: [typescript, python, go, php] + output: sdk + diff: + enabled: true + fail_on_breaking: false +``` + +## Dependencies + +| Dependency | Purpose | +|---|---| +| `forge.lthn.ai/core/cli` | CLI command registration and TUI styling | +| `forge.lthn.ai/core/go-io` | Filesystem abstraction (`io.Medium`, `io.Local`) | +| `forge.lthn.ai/core/go-i18n` | Internationalised CLI labels | +| `forge.lthn.ai/core/go-log` | Structured error logging | +| `github.com/Snider/Borg` | XZ compression for tar.xz archives | +| `github.com/getkin/kin-openapi` | OpenAPI spec loading and validation | +| `github.com/oasdiff/oasdiff` | OpenAPI diff and breaking-change detection | +| `gopkg.in/yaml.v3` | YAML config parsing | +| `github.com/leaanthony/debme` | Embedded filesystem anchoring (PWA templates) | +| `github.com/leaanthony/gosod` | Template extraction for PWA builds | +| `golang.org/x/net` | HTML parsing for PWA manifest detection | +| `golang.org/x/text` | Changelog section title casing | + +## Licence + +EUPL-1.2 diff --git a/docs/go/packages/go-cache.md b/docs/go/packages/go-cache.md index a5eaaee..76cbfe8 100644 --- a/docs/go/packages/go-cache.md +++ b/docs/go/packages/go-cache.md @@ -1,15 +1,111 @@ --- title: go-cache -description: In-memory cache with expiry support +description: File-based caching with TTL expiry, storage-agnostic via the go-io Medium interface. --- # go-cache -`forge.lthn.ai/core/go-cache` +`go-cache` is a lightweight, storage-agnostic caching library for Go. It stores +JSON-serialised entries with automatic TTL expiry and path-traversal protection. -Lightweight in-memory cache for Go applications. Stores key-value pairs with optional expiry. +**Module path:** `forge.lthn.ai/core/go-cache` -## Key Types +**Licence:** EUPL-1.2 -- `Cache` — thread-safe in-memory key-value store -- `Entry` — single cached item with value and expiry metadata + +## Quick Start + +```go +import ( + "fmt" + "time" + + "forge.lthn.ai/core/go-cache" +) + +func main() { + // Create a cache with default settings: + // - storage: local filesystem (io.Local) + // - directory: .core/cache/ in the working directory + // - TTL: 1 hour + c, err := cache.New(nil, "", 0) + if err != nil { + panic(err) + } + + // Store a value + err = c.Set("user/profile", map[string]string{ + "name": "Alice", + "role": "admin", + }) + if err != nil { + panic(err) + } + + // Retrieve it (returns false if missing or expired) + var profile map[string]string + found, err := c.Get("user/profile", &profile) + if err != nil { + panic(err) + } + if found { + fmt.Println(profile["name"]) // Alice + } +} +``` + + +## Package Layout + +| File | Purpose | +|-----------------|-------------------------------------------------------------| +| `cache.go` | Core types (`Cache`, `Entry`), CRUD operations, key helpers | +| `cache_test.go` | Tests covering set/get, expiry, delete, clear, defaults | +| `go.mod` | Module definition (Go 1.26) | + + +## Dependencies + +| Module | Version | Role | +|-------------------------------|---------|---------------------------------------------| +| `forge.lthn.ai/core/go-io` | v0.0.3 | Storage abstraction (`Medium` interface) | +| `forge.lthn.ai/core/go-log` | v0.0.1 | Structured logging (indirect, via `go-io`) | + +There are no other runtime dependencies. The test suite uses the standard +library only (plus the `MockMedium` from `go-io`). + + +## Key Concepts + +### Storage Backends + +The cache does not read or write files directly. All I/O goes through the +`io.Medium` interface defined in `go-io`. This means the same cache logic works +against: + +- **Local filesystem** (`io.Local`) -- the default +- **SQLite KV store** (`store.Medium` from `go-io/store`) +- **S3-compatible storage** (`go-io/s3`) +- **In-memory mock** (`io.NewMockMedium()`) -- ideal for tests + +Pass any `Medium` implementation as the first argument to `cache.New()`. + +### TTL and Expiry + +Every entry records both `cached_at` and `expires_at` timestamps. On `Get()`, +if the current time is past `expires_at`, the entry is treated as a cache miss +-- no stale data is ever returned. The default TTL is one hour +(`cache.DefaultTTL`). + +### GitHub Cache Keys + +The package includes two helper functions that produce consistent cache keys +for GitHub API data: + +```go +cache.GitHubReposKey("host-uk") // "github/host-uk/repos" +cache.GitHubRepoKey("host-uk", "core") // "github/host-uk/core/meta" +``` + +These are convenience helpers used by other packages in the ecosystem (such as +`go-devops`) to avoid key duplication when caching GitHub responses. diff --git a/docs/go/packages/go-config.md b/docs/go/packages/go-config.md index 162c498..27ba6bc 100644 --- a/docs/go/packages/go-config.md +++ b/docs/go/packages/go-config.md @@ -1,16 +1,141 @@ --- title: go-config -description: Configuration service for the Core DI container +description: Layered configuration management for the Core framework with file, environment, and in-memory resolution. --- # go-config -`forge.lthn.ai/core/go-config` +`forge.lthn.ai/core/go-config` provides layered configuration management for applications built on the Core framework. It resolves values through a priority chain -- defaults, file, environment variables, flags -- so that the same codebase works identically across local development, CI, and production without code changes. -Configuration management service that integrates with the Core dependency injection container. Handles loading, merging, and accessing typed configuration values for services registered in the Core runtime. +## Module Path -## Key Types +``` +forge.lthn.ai/core/go-config +``` -- `Config` — configuration data holder with typed accessors -- `Service` — Core-integrated service that manages configuration lifecycle -- `ServiceOptions` — options for configuring the service registration +Requires **Go 1.26+**. + +## Quick Start + +### Standalone usage + +```go +package main + +import ( + "fmt" + config "forge.lthn.ai/core/go-config" +) + +func main() { + cfg, err := config.New() // loads ~/.core/config.yaml if it exists + if err != nil { + panic(err) + } + + // Write a value and persist it + _ = cfg.Set("dev.editor", "vim") + _ = cfg.Commit() + + // Read it back + var editor string + _ = cfg.Get("dev.editor", &editor) + fmt.Println(editor) // "vim" +} +``` + +### As a Core framework service + +```go +import ( + config "forge.lthn.ai/core/go-config" + "forge.lthn.ai/core/go/pkg/core" +) + +app, _ := core.New( + core.WithService(config.NewConfigService), +) +// The config service loads automatically during OnStartup. +// Retrieve it later via core.ServiceFor[*config.Service](app). +``` + +## Package Layout + +| File | Purpose | +|-----------------|----------------------------------------------------------------| +| `config.go` | Core `Config` struct -- layered Get/Set, file load, commit | +| `env.go` | Environment variable iteration and prefix-based loading | +| `service.go` | Framework service wrapper with lifecycle (`Startable`) support | +| `config_test.go`| Tests following the `_Good` / `_Bad` / `_Ugly` convention | + +## Dependencies + +| Module | Role | +|-----------------------------------|-----------------------------------------| +| `forge.lthn.ai/core/go` | Core framework (`core.Config` interface, `ServiceRuntime`) | +| `forge.lthn.ai/core/go-io` | Storage abstraction (`Medium` for reading/writing files) | +| `forge.lthn.ai/core/go-log` | Contextual error helper (`E()`) | +| `github.com/spf13/viper` | Underlying configuration engine | +| `gopkg.in/yaml.v3` | YAML serialisation for `Commit()` | + +## Configuration Priority + +Values are resolved in ascending priority order: + +1. **Defaults** -- hardcoded fallbacks (via `Set()` before any file load) +2. **File** -- YAML loaded from `~/.core/config.yaml` (or a custom path) +3. **Environment variables** -- prefixed with `CORE_CONFIG_` by default +4. **Explicit Set()** -- in-memory overrides applied at runtime + +Environment variables always override file values. An explicit `Set()` call overrides everything. + +## Key Access + +All keys use **dot notation** for nested values: + +```go +cfg.Set("a.b.c", "deep") + +var val string +cfg.Get("a.b.c", &val) // "deep" +``` + +This maps to YAML structure: + +```yaml +a: + b: + c: deep +``` + +## Environment Variable Mapping + +Environment variables are mapped to dot-notation keys by: + +1. Stripping the prefix (default `CORE_CONFIG_`) +2. Lowercasing +3. Replacing `_` with `.` + +For example, `CORE_CONFIG_DEV_EDITOR=nano` resolves to key `dev.editor` with value `"nano"`. + +You can change the prefix with `WithEnvPrefix`: + +```go +cfg, _ := config.New(config.WithEnvPrefix("MYAPP")) +// MYAPP_SETTING=secret -> key "setting" +``` + +## Persisting Changes + +`Set()` only writes to memory. Call `Commit()` to flush changes to disk: + +```go +cfg.Set("dev.editor", "vim") +cfg.Commit() // writes to ~/.core/config.yaml +``` + +`Commit()` only persists values that were loaded from the file or explicitly set via `Set()`. Environment variable values are never leaked into the config file. + +## Licence + +EUPL-1.2 diff --git a/docs/go/packages/go-container.md b/docs/go/packages/go-container.md index 65dac04..1afee3b 100644 --- a/docs/go/packages/go-container.md +++ b/docs/go/packages/go-container.md @@ -1,18 +1,146 @@ --- title: go-container -description: Container runtime, LinuxKit builder, and dev environment management +description: Container runtime, LinuxKit image builder, and portable development environment management for Go. --- # go-container -`forge.lthn.ai/core/go-container` +`forge.lthn.ai/core/go-container` provides a container runtime built on LinuxKit and lightweight hypervisors. It manages the full lifecycle of LinuxKit virtual machines -- from building images with embedded templates, to running them via QEMU or Hyperkit, to offering a portable development environment with shell access, project mounting, test execution, and Claude AI integration. -Container runtime abstraction supporting multiple hypervisors for running LinuxKit-based images and managing development environments. Handles image building, VM lifecycle, and port forwarding across QEMU and HyperKit backends. +This is **not** a Docker wrapper. It runs real VMs from LinuxKit images (ISO, qcow2, VMDK, raw) using platform-native acceleration (KVM on Linux, HVF on macOS, Hyperkit where available). -## Key Types -- `Container` — manages container lifecycle (create, start, stop, remove) -- `RunOptions` — configuration for container execution (ports, volumes, environment) -- `HypervisorOptions` — shared configuration across hypervisor backends -- `QemuHypervisor` — QEMU-based VM backend -- `HyperkitHypervisor` — HyperKit-based VM backend (macOS) +## Module path + +``` +forge.lthn.ai/core/go-container +``` + +Requires **Go 1.26+**. + + +## Quick start + +### Run a VM from an image + +```go +import ( + "context" + container "forge.lthn.ai/core/go-container" + "forge.lthn.ai/core/go-io" +) + +manager, err := container.NewLinuxKitManager(io.Local) +if err != nil { + log.Fatal(err) +} + +ctx := context.Background() +c, err := manager.Run(ctx, "/path/to/image.qcow2", container.RunOptions{ + Name: "my-vm", + Memory: 2048, + CPUs: 2, + SSHPort: 2222, + Detach: true, +}) +if err != nil { + log.Fatal(err) +} + +fmt.Printf("Started container %s (PID %d)\n", c.ID, c.PID) +``` + +### Use the development environment + +```go +import ( + "forge.lthn.ai/core/go-container/devenv" + "forge.lthn.ai/core/go-io" +) + +dev, err := devenv.New(io.Local) +if err != nil { + log.Fatal(err) +} + +// Boot the dev environment (downloads image if needed) +ctx := context.Background() +err = dev.Boot(ctx, devenv.DefaultBootOptions()) + +// Open an SSH shell +err = dev.Shell(ctx, devenv.ShellOptions{}) + +// Run tests inside the VM +err = dev.Test(ctx, "/path/to/project", devenv.TestOptions{}) +``` + +### Build and run from a LinuxKit template + +```go +import container "forge.lthn.ai/core/go-container" + +// List available templates (built-in + user-defined) +templates := container.ListTemplates() + +// Apply variables to a template +content, err := container.ApplyTemplate("core-dev", map[string]string{ + "SSH_KEY": "ssh-ed25519 AAAA...", + "MEMORY": "4096", + "HOSTNAME": "my-dev-box", +}) +``` + + +## Package layout + +| Package | Import path | Purpose | +|---------|-------------|---------| +| `container` (root) | `forge.lthn.ai/core/go-container` | Container struct, Manager interface, hypervisor abstraction, LinuxKit manager, state persistence, template engine | +| `devenv` | `forge.lthn.ai/core/go-container/devenv` | Portable dev environment orchestration: boot, shell, serve, test, Claude sandbox, image management | +| `sources` | `forge.lthn.ai/core/go-container/sources` | Image download backends: CDN and GitHub Releases with progress reporting | +| `cmd/vm` | `forge.lthn.ai/core/go-container/cmd/vm` | CLI commands (`core vm run`, `core vm ps`, `core vm stop`, `core vm logs`, `core vm exec`, `core vm templates`) | + + +## Dependencies + +| Module | Purpose | +|--------|---------| +| `forge.lthn.ai/core/go-io` | File system abstraction (`Medium` interface), process utilities | +| `forge.lthn.ai/core/go-config` | Configuration loading (used by `devenv` for `~/.core/config.yaml`) | +| `forge.lthn.ai/core/go-i18n` | Internationalised UI strings (used by `cmd/vm`) | +| `forge.lthn.ai/core/cli` | CLI framework (used by `cmd/vm` for command registration) | +| `github.com/stretchr/testify` | Test assertions | +| `gopkg.in/yaml.v3` | YAML parsing for test configuration | + +The root `container` package has only two direct dependencies: `go-io` and the standard library. The `devenv` and `cmd/vm` packages pull in the heavier dependencies. + + +## CLI commands + +When registered via `cmd/vm`, the following commands become available under `core vm`: + +| Command | Description | +|---------|-------------| +| `core vm run [image]` | Run a VM from an image file or `--template` | +| `core vm ps` | List running VMs (`-a` for all including stopped) | +| `core vm stop ` | Stop a running VM by ID or name (supports partial matching) | +| `core vm logs ` | View VM logs (`-f` to follow) | +| `core vm exec ` | Execute a command inside the VM via SSH | +| `core vm templates` | List available LinuxKit templates | +| `core vm templates show ` | Display a template's full YAML | +| `core vm templates vars ` | Show a template's required and optional variables | + + +## Built-in templates + +Two LinuxKit templates are embedded in the binary: + +- **core-dev** -- Full development environment with Go, Node.js, PHP, Docker-in-LinuxKit, and SSH access +- **server-php** -- Production PHP server with FrankenPHP, Caddy reverse proxy, and health checks + +User-defined templates can be placed in `.core/linuxkit/` (workspace-relative) or `~/.core/linuxkit/` (global). They are discovered automatically and merged with the built-in set. + + +## Licence + +EUPL-1.2. See [LICENSE](../LICENSE) for the full text. diff --git a/docs/go/packages/go-crypt.md b/docs/go/packages/go-crypt.md index 7f1f3ff..3916142 100644 --- a/docs/go/packages/go-crypt.md +++ b/docs/go/packages/go-crypt.md @@ -1,10 +1,19 @@ +--- +title: go-crypt +description: Cryptographic primitives, authentication, and trust policy engine for the Lethean agent platform. +--- + # go-crypt -Cryptographic primitives, authentication, and trust policy engine. - **Module**: `forge.lthn.ai/core/go-crypt` +**Licence**: EUPL-1.2 +**Language**: Go 1.26 -Provides symmetric encryption (ChaCha20-Poly1305 and AES-256-GCM with Argon2id KDF), OpenPGP challenge-response authentication with online and air-gapped courier modes, Argon2id password hashing, RSA-OAEP key generation, RFC-0004 deterministic content hashing, and a three-tier agent trust policy engine with an audit log and approval queue. +Cryptographic primitives, authentication, and trust policy engine for the +Lethean agent platform. Provides symmetric encryption, password hashing, +OpenPGP authentication with both online and air-gapped modes, RSA key +management, deterministic content hashing, and a three-tier agent access +control system with an audit log and approval queue. ## Quick Start @@ -14,25 +23,141 @@ import ( "forge.lthn.ai/core/go-crypt/auth" "forge.lthn.ai/core/go-crypt/trust" ) - -// Encrypt with ChaCha20-Poly1305 + Argon2id KDF -ciphertext, err := crypt.Encrypt(plaintext, passphrase) - -// OpenPGP authentication -a := auth.New(medium, auth.WithSessionStore(auth.NewSQLiteSessionStore(dbPath))) -session, err := a.Login(userID, password) - -// Trust policy evaluation -engine := trust.NewPolicyEngine(registry) -decision := engine.Evaluate("Charon", "repo.push", "core/go-crypt") ``` -## Packages +### Encrypt and Decrypt Data -| Package | Description | -|---------|-------------| -| `crypt` | ChaCha20-Poly1305, AES-256-GCM, Argon2id KDF | -| `auth` | OpenPGP challenge-response, session management | -| `trust` | Three-tier policy engine, audit log, approval queue | -| `hash` | RFC-0004 deterministic content hashing | -| `keys` | RSA-OAEP key generation | +The default cipher is XChaCha20-Poly1305 with Argon2id key derivation. A +random salt and nonce are generated automatically and prepended to the +ciphertext. + +```go +// Encrypt with ChaCha20-Poly1305 + Argon2id KDF +ciphertext, err := crypt.Encrypt(plaintext, []byte("my passphrase")) + +// Decrypt +plaintext, err := crypt.Decrypt(ciphertext, []byte("my passphrase")) + +// Or use AES-256-GCM instead +ciphertext, err := crypt.EncryptAES(plaintext, []byte("my passphrase")) +plaintext, err := crypt.DecryptAES(ciphertext, []byte("my passphrase")) +``` + +### Hash and Verify Passwords + +```go +// Hash with Argon2id (recommended) +hash, err := crypt.HashPassword("hunter2") +// Returns: $argon2id$v=19$m=65536,t=3,p=4$$ + +// Verify (constant-time comparison) +match, err := crypt.VerifyPassword("hunter2", hash) +``` + +### OpenPGP Authentication + +```go +// Create an authenticator backed by a storage medium +a := auth.New(medium, + auth.WithSessionStore(sqliteStore), + auth.WithSessionTTL(8 * time.Hour), +) + +// Register a user (generates PGP keypair, stores credentials) +user, err := a.Register("alice", "password123") + +// Password-based login (bypasses PGP challenge-response) +session, err := a.Login(userID, "password123") + +// Validate a session token +session, err := a.ValidateSession(token) +``` + +### Trust Policy Evaluation + +```go +// Set up a registry and register agents +registry := trust.NewRegistry() +registry.Register(trust.Agent{ + Name: "Athena", + Tier: trust.TierFull, +}) +registry.Register(trust.Agent{ + Name: "Clotho", + Tier: trust.TierVerified, + ScopedRepos: []string{"core/*"}, +}) + +// Evaluate capabilities +engine := trust.NewPolicyEngine(registry) +result := engine.Evaluate("Athena", trust.CapPushRepo, "core/go-crypt") +// result.Decision == trust.Allow + +result = engine.Evaluate("Clotho", trust.CapMergePR, "core/go-crypt") +// result.Decision == trust.NeedsApproval +``` + +## Package Layout + +| Package | Import Path | Description | +|---------|-------------|-------------| +| `crypt` | `go-crypt/crypt` | High-level encrypt/decrypt (ChaCha20 + AES), password hashing, HMAC, checksums, key derivation | +| `crypt/chachapoly` | `go-crypt/crypt/chachapoly` | Standalone ChaCha20-Poly1305 AEAD wrapper | +| `crypt/lthn` | `go-crypt/crypt/lthn` | RFC-0004 quasi-salted deterministic hash for content identifiers | +| `crypt/pgp` | `go-crypt/crypt/pgp` | OpenPGP key generation, encryption, decryption, signing, verification | +| `crypt/rsa` | `go-crypt/crypt/rsa` | RSA-OAEP-SHA256 key generation and encryption (2048+ bit) | +| `crypt/openpgp` | `go-crypt/crypt/openpgp` | Service wrapper implementing the `core.Crypt` interface with IPC support | +| `auth` | `go-crypt/auth` | OpenPGP challenge-response authentication, session management, key rotation/revocation | +| `trust` | `go-crypt/trust` | Agent trust model, policy engine, approval queue, audit log | +| `cmd/crypt` | `go-crypt/cmd/crypt` | CLI commands: `crypt encrypt`, `crypt decrypt`, `crypt hash`, `crypt keygen`, `crypt checksum` | + +## CLI Commands + +The `cmd/crypt` package registers a `crypt` command group with the `core` CLI: + +```bash +# Encrypt a file (ChaCha20-Poly1305 by default) +core crypt encrypt myfile.txt -p "passphrase" +core crypt encrypt myfile.txt --aes -p "passphrase" + +# Decrypt +core crypt decrypt myfile.txt.enc -p "passphrase" + +# Hash a password +core crypt hash "my password" # Argon2id +core crypt hash "my password" --bcrypt # Bcrypt + +# Verify a password against a hash +core crypt hash "my password" --verify '$argon2id$v=19$...' + +# Generate a random key +core crypt keygen # 32 bytes, hex +core crypt keygen -l 64 --base64 # 64 bytes, base64 + +# Compute file checksums +core crypt checksum myfile.txt # SHA-256 +core crypt checksum myfile.txt --sha512 +core crypt checksum myfile.txt --verify "abc123..." +``` + +## Dependencies + +| Module | Role | +|--------|------| +| `forge.lthn.ai/core/go` | Framework: `core.E` error helper, `core.Crypt` interface, `io.Medium` storage abstraction | +| `forge.lthn.ai/core/go-store` | SQLite KV store for persistent session storage | +| `forge.lthn.ai/core/go-io` | `io.Medium` interface used by the auth package | +| `forge.lthn.ai/core/go-log` | Contextual error wrapping via `core.E()` | +| `forge.lthn.ai/core/cli` | CLI framework for the `cmd/crypt` commands | +| `github.com/ProtonMail/go-crypto` | OpenPGP implementation (actively maintained, post-quantum research) | +| `golang.org/x/crypto` | Argon2id, ChaCha20-Poly1305, scrypt, HKDF, bcrypt | +| `github.com/stretchr/testify` | Test assertions (`assert`, `require`) | + +No C toolchain or CGo is required. All cryptographic operations use pure Go +implementations. + +## Further Reading + +- [Architecture](architecture.md) -- internals, data flow, algorithm reference +- [Development](development.md) -- building, testing, contributing +- [History](history.md) -- completed phases, security audit findings, known limitations diff --git a/docs/go/packages/go-devops.md b/docs/go/packages/go-devops.md index d0444ab..b3fb457 100644 --- a/docs/go/packages/go-devops.md +++ b/docs/go/packages/go-devops.md @@ -1,38 +1,146 @@ +--- +title: go-devops +description: Build system, release publishers, infrastructure management, and DevOps tooling for the Lethean ecosystem. +--- + # go-devops -Infrastructure and build automation library for the Lethean ecosystem. +`forge.lthn.ai/core/go-devops` is the build, release, and infrastructure automation library for the Lethean ecosystem. It replaces goreleaser with a native Go pipeline that auto-detects project types, cross-compiles, signs artefacts, generates changelogs, and publishes to eight distribution targets. **Module**: `forge.lthn.ai/core/go-devops` +**Go**: 1.26 +**Licence**: EUPL-1.2 -Provides a native Go Ansible playbook executor (~30 modules over SSH without shelling out), a multi-target build pipeline with project type auto-detection (Go, Wails, Docker, C++, LinuxKit, Taskfile), code signing (macOS codesign, GPG, Windows signtool), release orchestration with changelog generation and eight publisher backends (GitHub Releases, Docker, Homebrew, npm, AUR, Scoop, Chocolatey, LinuxKit), Hetzner Cloud and Robot API clients, CloudNS DNS management, container/VM management via QEMU and Hyperkit, an OpenAPI SDK generator (TypeScript, Python, Go, PHP), and a developer toolkit with cyclomatic complexity analysis, vulnerability scanning, and coverage trending. +## What it does -## Quick Start +| Area | Summary | +|------|---------| +| **Build system** | Auto-detect project type from marker files, cross-compile for multiple OS/arch targets, archive and checksum artefacts | +| **Code signing** | macOS `codesign`, GPG detached signatures, Windows `signtool` | +| **Release publishers** | GitHub Releases, Docker, Homebrew, npm, AUR, Scoop, Chocolatey, LinuxKit | +| **SDK generation** | Generate typed API clients from OpenAPI specs (TypeScript, Python, Go, PHP) with breaking change detection | +| **Ansible executor** | Native Go playbook runner with ~30 modules over SSH — no `ansible-playbook` shell-out | +| **Infrastructure** | Hetzner Cloud/Robot provisioning, CloudNS DNS management | +| **Container/VM** | LinuxKit-based VMs via QEMU (Linux) or Hyperkit (macOS) | +| **Developer toolkit** | Cyclomatic complexity analysis, vulnerability scanning, coverage trending, secret scanning | +| **Doc sync** | Collect documentation from multi-repo workspaces into a central location | -```go -import ( - "forge.lthn.ai/core/go-devops/ansible" - "forge.lthn.ai/core/go-devops/build" - "forge.lthn.ai/core/go-devops/release" -) +## Package layout -// Run an Ansible playbook over SSH -pb, _ := ansible.ParsePlaybook("playbooks/deploy.yml") -inv, _ := ansible.ParseInventory("inventory.yml") -pb.Run(ctx, inv) - -// Build and release -artifacts, _ := build.Build(ctx, ".") -release.Publish(ctx, releaseCfg, false) +``` +go-devops/ +├── ansible/ Ansible playbook execution engine (native Go, no shell-out) +├── build/ Build system: project detection, archives, checksums +│ ├── builders/ Builders: Go, Wails, Docker, C++, LinuxKit, Taskfile +│ ├── signing/ Code signing: macOS codesign, GPG, Windows signtool +│ └── buildcmd/ CLI handlers for core build / core release +├── container/ LinuxKit VM management, hypervisor abstraction +├── deploy/ Deployment integrations (Coolify PaaS, embedded Python) +├── devkit/ Code quality, security, coverage trending +├── devops/ Portable dev environment management +│ └── sources/ Image download: GitHub Releases, S3/CDN +├── infra/ Infrastructure APIs: Hetzner Cloud, Hetzner Robot, CloudNS +├── release/ Release orchestration: version, changelog, publishing +│ └── publishers/ 8 publisher backends +├── sdk/ OpenAPI SDK generation and breaking change detection +│ └── generators/ Language generators: TypeScript, Python, Go, PHP +├── snapshot/ Frozen release manifest generation (core.json) +└── cmd/ CLI command registrations + ├── dev/ Multi-repo workflow commands (work, health, commit, push, pull) + ├── docs/ Documentation sync and listing + ├── deploy/ Coolify deployment commands + ├── setup/ Repository and CI bootstrapping + └── gitcmd/ Git helpers ``` -## Key Packages +## CLI commands -| Package | Description | -|---------|-------------| -| `ansible` | Native Go Ansible executor (~30 modules) | -| `build` | Multi-target build pipeline with auto-detection | -| `release` | Changelog generation + 8 publisher backends | -| `hetzner` | Hetzner Cloud and Robot API clients | -| `cloudns` | CloudNS DNS management | -| `sdk` | OpenAPI SDK generator | -| `devkit` | Complexity analysis, vuln scanning, coverage | +go-devops registers commands into the `core` CLI binary (built from `forge.lthn.ai/core/cli`). Key commands: + +```bash +# Build +core build # Auto-detect project type, build for configured targets +core build --ci # All targets, JSON output +core build sdk # Generate SDKs from OpenAPI spec + +# Release +core build release # Build + changelog + publish (requires --we-are-go-for-launch) + +# Multi-repo development +core dev health # Quick summary across all repos +core dev work # Combined status, commit, push workflow +core dev commit # Claude-assisted commits for dirty repos +core dev push # Push repos with unpushed commits +core dev pull # Pull repos behind remote + +# GitHub integration +core dev issues # List open issues across repos +core dev reviews # PRs needing review +core dev ci # GitHub Actions status + +# Documentation +core docs list # Scan repos for docs +core docs sync # Copy docs to central location +core docs sync --target gohelp # Sync to go-help format + +# Deployment +core deploy servers # List Coolify servers +core deploy apps # List Coolify applications + +# Setup +core setup repo # Generate .core/ configuration for a repo +core setup ci # Bootstrap CI configuration +``` + +## Configuration + +Two YAML files in `.core/` at the project root control build and release behaviour: + +| File | Purpose | +|------|---------| +| `.core/build.yaml` | Project name, binary, build flags, cross-compilation targets | +| `.core/release.yaml` | Repository, changelog rules, publisher configs, SDK settings | + +See [Build System](build-system.md) and [Publishers](publishers.md) for full configuration reference. + +## Core interfaces + +Every extensible subsystem is defined by a small interface: + +```go +// Builder — project type plugin (build/builders/) +type Builder interface { + Name() string + Detect(fs io.Medium, dir string) (bool, error) + Build(ctx context.Context, cfg *Config, targets []Target) ([]Artifact, error) +} + +// Publisher — distribution target plugin (release/publishers/) +type Publisher interface { + Name() string + Publish(ctx context.Context, release *Release, pubCfg PublisherConfig, + relCfg ReleaseConfig, dryRun bool) error +} + +// Generator — SDK language generator (sdk/generators/) +type Generator interface { + Language() string + Generate(ctx context.Context, spec, outputDir string, config *Config) error +} + +// Signer — code signing plugin (build/signing/) +type Signer interface { + Name() string + Available() bool + Sign(filePath, keyID string) ([]byte, error) +} +``` + +## Further reading + +- [Build System](build-system.md) — Builders, project detection, `.core/build.yaml` reference +- [Publishers](publishers.md) — Release publishers, `.core/release.yaml` reference +- [SDK Generation](sdk-generation.md) — OpenAPI client generation and breaking change detection +- [Doc Sync](sync.md) — Documentation sync across multi-repo workspaces +- [Architecture](architecture.md) — Full architecture deep-dive (Ansible, infra, devkit, containers) +- [Development Guide](development.md) — Building, testing, coding standards diff --git a/docs/go/packages/go-forge.md b/docs/go/packages/go-forge.md index 6a81476..53ce237 100644 --- a/docs/go/packages/go-forge.md +++ b/docs/go/packages/go-forge.md @@ -1,18 +1,151 @@ --- title: go-forge -description: Forgejo API client covering actions, admin, repos, branches, and more +description: Full-coverage Go client for the Forgejo API with generics-based CRUD, pagination, and code-generated types. --- # go-forge -`forge.lthn.ai/core/go-forge` +`forge.lthn.ai/core/go-forge` is a Go client library for the [Forgejo](https://forgejo.org) REST API. It provides typed access to 18 API domains (repositories, issues, pull requests, organisations, and more) through a single top-level `Forge` client. Types are generated directly from Forgejo's `swagger.v1.json` specification, keeping the library in lockstep with the server. -Comprehensive API client for Forgejo (Gitea-compatible) instances. Covers the full Forgejo REST API surface including repository management, CI/CD actions, admin operations, branch protection, and user management. Handles authentication, rate limiting, and pagination. +**Module path:** `forge.lthn.ai/core/go-forge` +**Go version:** 1.26+ +**Licence:** EUPL-1.2 -## Key Types -- `ActionsService` — CI/CD actions and workflow management -- `AdminService` — server administration operations -- `BranchService` — branch creation, protection, and deletion -- `APIError` — structured error from the Forgejo API -- `RateLimit` — rate limit state tracking for API calls +## Quick start + +```go +package main + +import ( + "context" + "fmt" + "log" + + "forge.lthn.ai/core/go-forge" +) + +func main() { + // Create a client with your Forgejo URL and API token. + f := forge.NewForge("https://forge.lthn.ai", "your-token") + + ctx := context.Background() + + // List repositories for an organisation (first page, 50 per page). + result, err := f.Repos.List(ctx, forge.Params{"org": "core"}, forge.DefaultList) + if err != nil { + log.Fatal(err) + } + for _, repo := range result.Items { + fmt.Println(repo.Name) + } + + // Get a single repository. + repo, err := f.Repos.Get(ctx, forge.Params{"owner": "core", "repo": "go-forge"}) + if err != nil { + log.Fatal(err) + } + fmt.Printf("%s — %s\n", repo.FullName, repo.Description) +} +``` + +### Configuration from environment + +If you prefer to resolve the URL and token from environment variables rather than hard-coding them, use `NewForgeFromConfig`: + +```go +// Priority: flags > env (FORGE_URL, FORGE_TOKEN) > defaults (http://localhost:3000) +f, err := forge.NewForgeFromConfig("", "", forge.WithUserAgent("my-tool/1.0")) +if err != nil { + log.Fatal(err) // no token configured +} +``` + +Environment variables: + +| Variable | Purpose | Default | +|---------------|--------------------------------------|--------------------------| +| `FORGE_URL` | Base URL of the Forgejo instance | `http://localhost:3000` | +| `FORGE_TOKEN` | API token for authentication | (none -- required) | + + +## Package layout + +``` +go-forge/ +├── client.go HTTP client, auth, error handling, rate limits +├── config.go Config resolution: flags > env > defaults +├── forge.go Top-level Forge struct aggregating all 18 services +├── resource.go Generic Resource[T, C, U] for CRUD operations +├── pagination.go ListPage, ListAll, ListIter — paginated requests +├── params.go Path variable resolution ({owner}/{repo} -> values) +├── repos.go RepoService — repositories, forks, transfers, mirrors +├── issues.go IssueService — issues, comments, labels, reactions +├── pulls.go PullService — pull requests, merges, reviews +├── orgs.go OrgService — organisations, members +├── users.go UserService — users, followers, stars +├── teams.go TeamService — teams, members, repositories +├── admin.go AdminService — site admin, cron, user management +├── branches.go BranchService — branches, branch protections +├── releases.go ReleaseService — releases, assets, tags +├── labels.go LabelService — repo and org labels +├── webhooks.go WebhookService — repo and org webhooks +├── notifications.go NotificationService — notifications, threads +├── packages.go PackageService — package registry +├── actions.go ActionsService — CI/CD secrets, variables, dispatches +├── contents.go ContentService — file read/write/delete +├── wiki.go WikiService — wiki pages +├── commits.go CommitService — statuses, notes +├── misc.go MiscService — markdown, licences, gitignore, version +├── types/ 229 generated Go types from swagger.v1.json +│ ├── generate.go go:generate directive +│ ├── repo.go Repository, CreateRepoOption, EditRepoOption, ... +│ ├── issue.go Issue, CreateIssueOption, ... +│ ├── pr.go PullRequest, CreatePullRequestOption, ... +│ └── ... (36 files total, grouped by domain) +├── cmd/forgegen/ Code generator: swagger spec -> types/*.go +│ ├── main.go CLI entry point +│ ├── parser.go Swagger spec parsing, type extraction, CRUD pair detection +│ └── generator.go Template-based Go source file generation +└── testdata/ + └── swagger.v1.json Forgejo API specification (input for codegen) +``` + + +## Services + +The `Forge` struct exposes 18 service fields, each handling a different API domain: + +| Service | Struct | Embedding | Domain | +|-----------------|---------------------|----------------------------------|--------------------------------------| +| `Repos` | `RepoService` | `Resource[Repository, ...]` | Repositories, forks, transfers | +| `Issues` | `IssueService` | `Resource[Issue, ...]` | Issues, comments, labels, reactions | +| `Pulls` | `PullService` | `Resource[PullRequest, ...]` | Pull requests, merges, reviews | +| `Orgs` | `OrgService` | `Resource[Organization, ...]` | Organisations, members | +| `Users` | `UserService` | `Resource[User, ...]` | Users, followers, stars | +| `Teams` | `TeamService` | `Resource[Team, ...]` | Teams, members, repos | +| `Admin` | `AdminService` | (standalone) | Site admin, cron, user management | +| `Branches` | `BranchService` | `Resource[Branch, ...]` | Branches, protections | +| `Releases` | `ReleaseService` | `Resource[Release, ...]` | Releases, assets, tags | +| `Labels` | `LabelService` | (standalone) | Repo and org labels | +| `Webhooks` | `WebhookService` | `Resource[Hook, ...]` | Repo and org webhooks | +| `Notifications` | `NotificationService` | (standalone) | Notifications, threads | +| `Packages` | `PackageService` | (standalone) | Package registry | +| `Actions` | `ActionsService` | (standalone) | CI/CD secrets, variables, dispatches | +| `Contents` | `ContentService` | (standalone) | File read/write/delete | +| `Wiki` | `WikiService` | (standalone) | Wiki pages | +| `Commits` | `CommitService` | (standalone) | Commit statuses, git notes | +| `Misc` | `MiscService` | (standalone) | Markdown, licences, gitignore, version | + +Services that embed `Resource[T, C, U]` inherit `List`, `ListAll`, `Iter`, `Get`, `Create`, `Update`, and `Delete` methods automatically. Standalone services have hand-written methods because their API endpoints are heterogeneous and do not fit a uniform CRUD pattern. + + +## Dependencies + +This module has **zero external dependencies**. It relies solely on the Go standard library (`net/http`, `encoding/json`, `context`, `iter`, etc.) and requires Go 1.26 or later. + +``` +module forge.lthn.ai/core/go-forge + +go 1.26.0 +``` diff --git a/docs/go/packages/go-git.md b/docs/go/packages/go-git.md index 4c5d9ea..4d64d3f 100644 --- a/docs/go/packages/go-git.md +++ b/docs/go/packages/go-git.md @@ -1,18 +1,125 @@ --- title: go-git -description: Git operations helper for status, push, and repository queries +description: Multi-repository Git operations library for Go with parallel status checking and Core framework integration. --- # go-git -`forge.lthn.ai/core/go-git` +**Module:** `forge.lthn.ai/core/go-git` -Helper library for common Git operations. Provides structured output for repository status, push results, and error handling around Git CLI commands. +**Go version:** 1.26+ -## Key Types +**Licence:** [EUPL-1.2](../LICENSE.md) -- `RepoStatus` — parsed state of a Git repository (branch, dirty files, ahead/behind) -- `StatusOptions` — options controlling what status information to collect -- `PushResult` — structured result of a push operation -- `GitError` — typed error wrapping Git CLI failures with exit codes -- `QueryStatus` — batch query result for multiple repositories +## What it does + +go-git is a Go library for orchestrating Git operations across multiple repositories. It was extracted from `forge.lthn.ai/core/go-scm/git/` into a standalone module. + +The library provides two layers: + +1. **Standalone functions** -- pure Git operations that depend only on the standard library. +2. **Core service integration** -- a `Service` type that plugs into the Core DI framework, exposing Git operations via the query/task message bus. + +Typical use cases include multi-repo status dashboards, batch push/pull workflows, and CI tooling that needs to inspect many repositories at once. + +## Quick start + +### Standalone usage (no framework) + +```go +package main + +import ( + "context" + "fmt" + + git "forge.lthn.ai/core/go-git" +) + +func main() { + statuses := git.Status(context.Background(), git.StatusOptions{ + Paths: []string{"/home/dev/repo-a", "/home/dev/repo-b"}, + Names: map[string]string{ + "/home/dev/repo-a": "repo-a", + "/home/dev/repo-b": "repo-b", + }, + }) + + for _, s := range statuses { + if s.Error != nil { + fmt.Printf("%s: error: %v\n", s.Name, s.Error) + continue + } + fmt.Printf("%s [%s]: modified=%d untracked=%d staged=%d ahead=%d behind=%d\n", + s.Name, s.Branch, s.Modified, s.Untracked, s.Staged, s.Ahead, s.Behind) + } +} +``` + +### With the Core framework + +```go +package main + +import ( + "fmt" + + "forge.lthn.ai/core/go/pkg/core" + git "forge.lthn.ai/core/go-git" +) + +func main() { + c, err := core.New( + core.WithService(git.NewService(git.ServiceOptions{ + WorkDir: "/home/dev/projects", + })), + ) + if err != nil { + panic(err) + } + + // Query status via the message bus. + result, err := c.Query(git.QueryStatus{ + Paths: []string{"/home/dev/projects/repo-a"}, + Names: map[string]string{"/home/dev/projects/repo-a": "repo-a"}, + }) + if err != nil { + panic(err) + } + + statuses := result.([]git.RepoStatus) + for _, s := range statuses { + fmt.Printf("%s: dirty=%v ahead=%v\n", s.Name, s.IsDirty(), s.HasUnpushed()) + } +} +``` + +## Package layout + +| File | Purpose | +|------|---------| +| `git.go` | Standalone Git operations -- `Status`, `Push`, `Pull`, `PushMultiple`, error types. Zero framework dependencies. | +| `service.go` | Core framework integration -- `Service`, query types (`QueryStatus`, `QueryDirtyRepos`, `QueryAheadRepos`), task types (`TaskPush`, `TaskPull`, `TaskPushMultiple`). | +| `git_test.go` | Tests for standalone operations using real temporary Git repositories. | +| `service_test.go` | Tests for `Service` filtering helpers (`DirtyRepos`, `AheadRepos`, iterators). | +| `service_extra_test.go` | Integration tests for `Service` query/task handlers against the Core framework. | + +## Dependencies + +| Dependency | Purpose | +|------------|---------| +| `forge.lthn.ai/core/go/pkg/core` | DI container, `ServiceRuntime`, query/task bus (used only by `service.go`). | +| `github.com/stretchr/testify` | Assertions in tests (test-only). | + +The standalone layer (`git.go`) uses only the Go standard library. It shells out to the system `git` binary -- there is no embedded Git implementation. + +## Build targets + +Defined in `.core/build.yaml`: + +| OS | Architecture | +|----|-------------| +| Linux | amd64 | +| Linux | arm64 | +| Darwin | arm64 | +| Windows | amd64 | diff --git a/docs/go/packages/go-html.md b/docs/go/packages/go-html.md index 75f5ad7..c7b12c4 100644 --- a/docs/go/packages/go-html.md +++ b/docs/go/packages/go-html.md @@ -1,35 +1,80 @@ +--- +title: go-html +description: HLCRF DOM compositor with grammar pipeline integration for type-safe server-side HTML generation and optional WASM client rendering. +--- + # go-html -HLCRF DOM compositor with grammar pipeline integration. +`go-html` is a pure-Go library for building HTML documents as type-safe node trees and rendering them to string output. It provides a five-slot layout compositor (Header, Left, Content, Right, Footer -- abbreviated HLCRF), a responsive multi-variant wrapper, a server-side grammar analysis pipeline, a Web Component code generator, and an optional WASM module for client-side rendering. -**Module**: `forge.lthn.ai/core/go-html` - -Provides a type-safe node tree (El, Text, Raw, If, Each, Switch, Entitled), a five-slot Header/Left/Content/Right/Footer layout compositor with deterministic `data-block` path IDs and ARIA roles, a responsive multi-variant wrapper, a server-side grammar pipeline (StripTags, GrammarImprint via go-i18n reversal, CompareVariants), a build-time Web Component codegen CLI, and a WASM module (2.90 MB raw, 842 KB gzip) exposing `renderToString()`. +**Module path:** `forge.lthn.ai/core/go-html` +**Go version:** 1.26 +**Licence:** EUPL-1.2 ## Quick Start ```go -import "forge.lthn.ai/core/go-html" +package main -page := html.NewLayout("HCF"). - H(html.El("nav", html.Text("i18n.label.navigation"))). - C(html.El("main", - html.El("h1", html.Text("i18n.label.welcome")), - html.Each(items, func(item Item) html.Node { - return html.El("li", html.Text(item.Name)) - }), - )). - F(html.El("footer", html.Text("i18n.label.copyright"))) +import html "forge.lthn.ai/core/go-html" -rendered := page.Render(html.NewContext("en-GB")) +func main() { + page := html.NewLayout("HCF"). + H(html.El("nav", html.Text("nav.label"))). + C(html.El("article", + html.El("h1", html.Text("page.title")), + html.Each(items, func(item Item) html.Node { + return html.El("li", html.Text(item.Name)) + }), + )). + F(html.El("footer", html.Text("footer.copyright"))) + + output := page.Render(html.NewContext()) +} ``` -## Layout Slots +This builds a Header-Content-Footer layout with semantic HTML elements (`
`, `
`, `