# go-api Architecture **Module**: `forge.lthn.ai/core/go-api` **Language**: Go 1.25 **Licence**: EUPL-1.2 **LOC**: ~2.1 K non-test, 176 tests passing --- ## 1. Overview `go-api` is the Gin-based REST framework for the Lethean Go ecosystem. Subsystems across the ecosystem plug into a central `Engine` via the `RouteGroup` interface, and go-api handles the HTTP plumbing: middleware composition, response envelopes, WebSocket and SSE integration, GraphQL, Authentik identity, OpenAPI spec generation, and SDK codegen. ### Position in the Lethean Stack ``` AI Clients / Browsers / SDK consumers | HTTP / WebSocket / SSE v [ go-api Engine ] ← this module | | | | | └─ OpenAPI spec → SDKGenerator → openapi-generator-cli | └─ ToolBridge ──→ go-ai / go-ml / go-rag route groups └─ RouteGroups ─────────→ any package that implements RouteGroup Core CLI or standalone main() bootstraps and wires everything ``` go-api is a library. It contains no `main` package. Callers construct an `Engine`, register route groups, and call `Serve()`. --- ## 2. Engine ### 2.1 The Engine Struct `Engine` is the central container. It owns the listen address, the ordered list of registered route groups, the middleware chain, and all optional integrations: ```go type Engine struct { addr string groups []RouteGroup middlewares []gin.HandlerFunc wsHandler http.Handler sseBroker *SSEBroker swaggerEnabled bool swaggerTitle string swaggerDesc string swaggerVersion string pprofEnabled bool expvarEnabled bool graphql *graphqlConfig } ``` ### 2.2 Construction `New()` applies functional options and returns a configured engine. The default listen address is `:8080`. No middleware is added automatically beyond Gin's built-in panic recovery; every feature requires an explicit `With*()` option. ```go engine, err := api.New( api.WithAddr(":9000"), api.WithBearerAuth("secret"), api.WithCORS("*"), api.WithRequestID(), api.WithSlog(nil), api.WithSwagger("My API", "Description", "1.0.0"), ) engine.Register(myRoutes) engine.Serve(ctx) ``` ### 2.3 Build Sequence `Engine.build()` (called internally by `Handler()` and `Serve()`) assembles a fresh `*gin.Engine` each time. The order is fixed: 1. `gin.Recovery()` — panic recovery (always first). 2. User middleware in registration order — all `With*()` options that append to `e.middlewares`. 3. Built-in `GET /health` endpoint — always present. 4. Route groups — each mounted at its `BasePath()`. 5. WebSocket handler at `GET /ws` — when `WithWSHandler()` was called. 6. SSE broker at `GET /events` — when `WithSSE()` was called. 7. GraphQL endpoint — when `WithGraphQL()` was called. 8. Swagger UI at `GET /swagger/*any` — when `WithSwagger()` was called. 9. pprof endpoints at `GET /debug/pprof/*` — when `WithPprof()` was called. 10. expvar endpoint at `GET /debug/vars` — when `WithExpvar()` was called. ### 2.4 Graceful Shutdown `Serve()` starts an `http.Server` in a goroutine and blocks on `ctx.Done()`. When the context is cancelled, a 10-second `Shutdown` deadline is applied. In-flight requests complete or time out before the server exits. --- ## 3. RouteGroup / DescribableGroup Interfaces Three interfaces form the extension point model: ```go // RouteGroup is the minimum interface. All subsystems implement this. type RouteGroup interface { Name() string BasePath() string RegisterRoutes(rg *gin.RouterGroup) } // StreamGroup is optionally implemented by groups that publish WebSocket channels. type StreamGroup interface { Channels() []string } // DescribableGroup extends RouteGroup with OpenAPI metadata. // Groups implementing this will appear in the generated spec. type DescribableGroup interface { RouteGroup Describe() []RouteDescription } ``` `RouteDescription` carries the HTTP method, path, summary, description, tags, and JSON Schema maps for the request body and response data. The `SpecBuilder` consumes these to generate the OpenAPI 3.1 paths object. `Engine.Channels()` iterates all registered groups and collects channel names from those that implement `StreamGroup`. This list is used when initialising a WebSocket hub. --- ## 4. Middleware Stack — 21 With*() Options All middleware options append to `Engine.middlewares` in the order they are passed to `New()`. They are applied after `gin.Recovery()` but before any route handler. | Option | Purpose | Key detail | |--------|---------|-----------| | `WithAddr(addr)` | Listen address | Default `:8080` | | `WithBearerAuth(token)` | Static bearer token auth | Skips `/health` and `/swagger` | | `WithRequestID()` | X-Request-ID propagation | Preserves client-supplied IDs | | `WithCORS(origins...)` | CORS policy | `"*"` enables `AllowAllOrigins`; 12-hour `MaxAge` | | `WithMiddleware(mw...)` | Arbitrary Gin middleware | Escape hatch | | `WithStatic(prefix, root)` | Static file serving | Directory listing disabled | | `WithWSHandler(h)` | WebSocket at `/ws` | Wraps any `http.Handler` | | `WithAuthentik(cfg)` | Authentik forward-auth + OIDC | Permissive; sets context, never rejects | | `WithSwagger(title, desc, ver)` | Swagger UI at `/swagger/` | Runtime spec via `SpecBuilder` | | `WithPprof()` | Go profiling at `/debug/pprof/` | WARNING: do not expose in production | | `WithExpvar()` | Runtime metrics at `/debug/vars` | WARNING: do not expose in production | | `WithSecure()` | Security headers | HSTS 1yr, X-Frame-Options DENY, nosniff | | `WithGzip(level...)` | Gzip response compression | Default compression if level omitted | | `WithBrotli(level...)` | Brotli response compression | Default compression if level omitted | | `WithSlog(logger)` | Structured request logging | Falls back to `slog.Default()` if nil | | `WithTimeout(d)` | Per-request deadline | 504 error envelope on timeout | | `WithCache(ttl)` | In-memory GET response caching | `X-Cache: HIT` on cache hits | | `WithSessions(name, secret)` | Cookie-backed server sessions | gin-contrib/sessions | | `WithAuthz(enforcer)` | Casbin policy-based authorisation | Subject from HTTP Basic Auth | | `WithHTTPSign(secrets, opts...)` | HTTP Signatures verification | draft-cavage-http-signatures | | `WithSSE(broker)` | Server-Sent Events at `/events` | `?channel=` filtering | | `WithLocation()` | Reverse proxy header detection | X-Forwarded-Proto / X-Forwarded-Host | | `WithI18n(cfg...)` | Accept-Language locale detection | BCP 47 matching via x/text/language | | `WithTracing(name, opts...)` | OpenTelemetry distributed tracing | otelgin + W3C traceparent | | `WithGraphQL(schema, opts...)` | GraphQL endpoint | gqlgen; optional playground UI | --- ## 5. Response[T] Envelope All API responses share a single generic envelope: ```go type Response[T any] struct { Success bool `json:"success"` Data T `json:"data,omitempty"` Error *Error `json:"error,omitempty"` Meta *Meta `json:"meta,omitempty"` } ``` Constructor helpers: | Helper | Usage | |--------|-------| | `OK(data)` | Successful response with typed data | | `Fail(code, message)` | Error response with code and message | | `FailWithDetails(code, message, details)` | Error with additional detail payload | | `Paginated(data, page, perPage, total)` | Successful response with pagination meta | `Meta` carries `request_id`, `duration`, `page`, `per_page`, and `total`. `Error` carries `code`, `message`, and an optional `details` field for structured validation errors. --- ## 6. Authentik Integration The `WithAuthentik()` option installs a permissive identity middleware that runs on every non-public request. It has two extraction paths: **Path 1 — Forward-auth headers (TrustedProxy: true):** When Traefik or another reverse proxy is configured with Authentik forward-auth, it injects `X-authentik-username`, `X-authentik-email`, `X-authentik-name`, `X-authentik-uid`, `X-authentik-groups` (pipe-separated), `X-authentik-entitlements`, and `X-authentik-jwt` headers. The middleware reads these and populates an `AuthentikUser` in the Gin context. **Path 2 — OIDC JWT validation:** For direct API clients that present a `Bearer` token, the middleware validates the JWT against the configured OIDC issuer. Providers are cached by issuer URL to avoid repeated discovery requests. In both paths, if extraction fails the request continues unauthenticated (fail-open). Handlers check identity with: ```go user := api.GetUser(c) // returns nil when unauthenticated ``` For protected routes, apply guards: ```go rg.GET("/private", api.RequireAuth(), handler) // 401 if no user rg.GET("/admin", api.RequireGroup("admins"), handler) // 403 if wrong group ``` `AuthentikConfig.PublicPaths` lists additional paths exempt from header extraction. `/health` and paths starting with `/swagger` are always exempt. --- ## 7. WebSocket and SSE ### WebSocket `WithWSHandler(h)` mounts any `http.Handler` at `GET /ws`. The intended pairing is a `go-ws` hub: ```go hub := ws.NewHub() go hub.Run(ctx) engine, _ := api.New(api.WithWSHandler(hub.Handler())) ``` Groups implementing `StreamGroup` declare channel names, which `Engine.Channels()` aggregates. ### Server-Sent Events `SSEBroker` manages persistent SSE connections. Clients connect to `GET /events` and optionally subscribe to a named channel via `?channel=`. Clients with no channel parameter receive events on all channels. ```go broker := api.NewSSEBroker() engine, _ := api.New(api.WithSSE(broker)) // In a handler or background goroutine: broker.Publish("deployments", "deploy.started", payload) ``` Each client has a 64-event buffer. Overflow events are dropped without blocking the publisher. `SSEBroker.Drain()` signals all clients to disconnect on graceful shutdown. --- ## 8. GraphQL `WithGraphQL()` mounts a gqlgen `ExecutableSchema` at `/graphql` (or a custom path via `WithGraphQLPath()`). An optional `WithPlayground()` adds the interactive playground at `{path}/playground`. ```go engine, _ := api.New( api.WithGraphQL(schema, api.WithPlayground(), api.WithGraphQLPath("/gql"), ), ) ``` The endpoint accepts all HTTP methods (POST for queries and mutations, GET for playground redirects and introspection). --- ## 9. OpenAPI Spec Generation ### SpecBuilder `SpecBuilder` generates an OpenAPI 3.1 JSON document from registered route groups: ```go builder := &api.SpecBuilder{ Title: "My API", Description: "...", Version: "1.0.0", } data, _ := builder.Build(engine.Groups()) ``` The built document includes: - The `GET /health` endpoint under the `system` tag. - One path entry per `RouteDescription` returned by `DescribableGroup.Describe()`. - `#/components/schemas/Error` and `#/components/schemas/Meta` shared schemas. - All response bodies wrapped in the `Response[T]` envelope schema. - Tags derived from every registered group's `Name()`. Groups that implement `RouteGroup` but not `DescribableGroup` contribute a tag but no paths. ### Export Two convenience functions write the spec to an `io.Writer` or directly to a file: ```go // Write JSON or YAML to a writer: api.ExportSpec(os.Stdout, "yaml", builder, engine.Groups()) // Write to a file (parent dirs created automatically): api.ExportSpecToFile("./api/openapi.yaml", "yaml", builder, engine.Groups()) ``` ### Swagger UI When `WithSwagger()` is active, the spec is built lazily on first access by a `swaggerSpec` wrapper and registered in the global `swag` registry with a unique sequence-based instance name. Multiple `Engine` instances in the same process (common in tests) do not collide in the registry. --- ## 10. ToolBridge `ToolBridge` converts tool descriptors — as produced by go-ai, go-ml, and similar packages — into `POST /{tool_name}` REST endpoints, and implements `DescribableGroup` so those endpoints appear in the generated OpenAPI spec. ```go bridge := api.NewToolBridge("/v1/tools") bridge.Add(api.ToolDescriptor{ Name: "file_read", Description: "Read a file", Group: "files", InputSchema: map[string]any{"type": "object", "properties": ...}, }, fileReadHandler) engine.Register(bridge) ``` Each registered tool becomes a `POST /v1/tools/{tool_name}` endpoint. `ToolBridge.Tools()` returns the full list of descriptors, making it easy to enumerate tools from the outside. --- ## 11. SDK Codegen `SDKGenerator` wraps `openapi-generator-cli` to generate client SDKs from an exported spec: ```go gen := &api.SDKGenerator{ SpecPath: "./api/openapi.yaml", OutputDir: "./sdk", PackageName: "myapi", } if gen.Available() { _ = gen.Generate(ctx, "typescript-fetch") _ = gen.Generate(ctx, "python") } ``` Supported target languages: `go`, `typescript-fetch`, `typescript-axios`, `python`, `java`, `csharp`, `ruby`, `swift`, `kotlin`, `rust`, `php`. `SupportedLanguages()` returns the full list in sorted order. `SDKGenerator.Available()` reports whether `openapi-generator-cli` is on `PATH` without running it. --- ## 12. In-Memory Cache `WithCache(ttl)` installs a URL-keyed in-memory response cache scoped to GET requests. Successful 2xx responses are stored. Cached responses are served with an `X-Cache: HIT` header and bypass all downstream handlers. Expired entries are evicted lazily on the next access for the same key. The cache is not shared across `Engine` instances and has no size limit. --- ## 13. OpenTelemetry Tracing `WithTracing(serviceName)` adds otelgin middleware that creates a span for each request, tagged with HTTP method, route template, and response status code. Trace context is propagated via the W3C `traceparent` header. `NewTracerProvider(exporter)` is a convenience helper for tests and simple single-service deployments that constructs a synchronous `TracerProvider` and installs it globally. Production deployments should build a batching provider with appropriate resource attributes. --- ## 14. Internationalisation `WithI18n(cfg)` parses the `Accept-Language` header on every request and stores the resolved BCP 47 locale tag in the Gin context. The `golang.org/x/text/language` matcher handles quality-weighted negotiation and script/region subtag matching against the configured supported locales. Handlers retrieve the locale and optional localised messages: ```go locale := api.GetLocale(c) // e.g. "en", "fr" msg, ok := api.GetMessage(c, "greeting") // from configured Messages map ``` The built-in message map is a lightweight bridge. The `go-i18n` grammar engine is the intended replacement for production-grade localisation. --- ## 15. Package Layout ``` go-api/ ├── api.go — Engine struct, New(), build(), Serve(), Handler(), Addr(), Channels() ├── options.go — All With*() option functions ├── group.go — RouteGroup, StreamGroup, DescribableGroup interfaces; RouteDescription ├── response.go — Response[T], Error, Meta, OK(), Fail(), FailWithDetails(), Paginated() ├── middleware.go — bearerAuthMiddleware(), requestIDMiddleware() ├── authentik.go — AuthentikUser, AuthentikConfig, GetUser(), RequireAuth(), RequireGroup() ├── websocket.go — wrapWSHandler() helper ├── sse.go — SSEBroker, NewSSEBroker(), Publish(), Handler(), Drain() ├── cache.go — cacheStore, cacheEntry, cacheWriter, cacheMiddleware() ├── brotli.go — brotliHandler, newBrotliHandler(); BrotliDefault/BestSpeed/BestCompression constants ├── graphql.go — graphqlConfig, GraphQLOption, WithPlayground(), WithGraphQLPath(), mountGraphQL() ├── i18n.go — I18nConfig, WithI18n(), i18nMiddleware(), GetLocale(), GetMessage() ├── tracing.go — WithTracing(), NewTracerProvider() ├── swagger.go — swaggerSpec, registerSwagger(); swaggerSeq for multi-instance safety ├── openapi.go — SpecBuilder, Build(), buildPaths(), buildTags(), envelopeSchema() ├── export.go — ExportSpec(), ExportSpecToFile() ├── bridge.go — ToolDescriptor, ToolBridge, NewToolBridge(), Add(), Describe(), Tools() └── codegen.go — SDKGenerator, Generate(), buildArgs(), Available(), SupportedLanguages() ``` --- ## 16. Dependency Diagram ### Direct Dependencies | Module | Role | |--------|------| | `github.com/gin-gonic/gin` | HTTP router and middleware engine | | `github.com/gin-contrib/cors` | CORS policy middleware | | `github.com/gin-contrib/secure` | Security headers | | `github.com/gin-contrib/gzip` | Gzip compression | | `github.com/gin-contrib/slog` | Structured logging | | `github.com/gin-contrib/timeout` | Per-request deadlines | | `github.com/gin-contrib/static` | Static file serving | | `github.com/gin-contrib/sessions` | Cookie-backed sessions | | `github.com/gin-contrib/authz` | Casbin authorisation | | `github.com/gin-contrib/httpsign` | HTTP Signatures | | `github.com/gin-contrib/location/v2` | Reverse proxy header detection | | `github.com/gin-contrib/pprof` | Go profiling endpoints | | `github.com/gin-contrib/expvar` | Runtime metrics endpoint | | `github.com/casbin/casbin/v2` | Policy-based access control engine | | `github.com/coreos/go-oidc/v3` | OIDC JWT validation for Authentik | | `github.com/andybalholm/brotli` | Brotli compression | | `github.com/gorilla/websocket` | WebSocket upgrade | | `github.com/swaggo/gin-swagger` | Swagger UI handler | | `github.com/swaggo/files` | Swagger UI static assets | | `github.com/swaggo/swag` | Swagger spec registry | | `github.com/99designs/gqlgen` | GraphQL schema execution | | `go.opentelemetry.io/otel` | OpenTelemetry tracing | | `go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin` | OTel + Gin | | `golang.org/x/text/language` | BCP 47 language matching | | `gopkg.in/yaml.v3` | YAML export of OpenAPI spec | ### What Imports go-api go-api is imported by packages that need to expose REST endpoints. Within the Lethean ecosystem the expected importers are: ``` Core CLI (forge.lthn.ai/core/go) └─ go-api ← wires route groups from go-ai, go-ml, go-rag into an Engine go-ai └─ go-api ← ToolBridge converts MCP tool descriptors to REST endpoints go-ml └─ go-api ← may register inference/scoring route groups Application main packages └─ go-api ← top-level server bootstrap ``` go-api itself has no imports from other Lethean ecosystem modules. It is the stable base that others import, not a consumer of them.