# go-api — Project History and Known Limitations Module: `forge.lthn.ai/core/go-api` --- ## Origins `go-api` was created as the dedicated HTTP framework for the Lethean Go ecosystem. The motivation was to give every Go package in the stack a consistent way to expose REST endpoints without each package taking its own opinion on routing, middleware, response formatting, or OpenAPI generation. It was scaffolded independently from the start — it was never extracted from a monolith — and has no `forge.lthn.ai/core/*` dependencies. This keeps it at the bottom of the import graph: every other package can import go-api, but go-api imports nothing from the ecosystem. --- ## Development Phases ### Phase 1 — Core Engine (36 tests) Commits `889391a` through `22f8a69` The initial phase established the foundational abstractions that all subsequent work builds on. **Scaffold** (`889391a`): Module path `forge.lthn.ai/core/go-api` created. `go.mod` initialised with Gin as the only direct dependency. **Response envelope** (`7835837`): `Response[T]`, `Error`, and `Meta` types defined. `OK()`, `Fail()`, and `Paginated()` helpers added. The generic envelope was established from the start rather than retrofitted. **RouteGroup interface** (`6f5fb69`): `RouteGroup` (Name, BasePath, RegisterRoutes) and `StreamGroup` (Channels) interfaces defined in `group.go`. The interface-driven extension model was the core design decision of Phase 1. **Engine** (`db75c88`): `Engine` struct added with `New()`, `Register()`, `Handler()`, `Serve()`, and graceful shutdown. Default listen address `:8080`. Built-in `GET /health` endpoint. Panic recovery via `gin.Recovery()` always applied first. **Bearer auth, request ID, CORS** (`d21734d`): First three middleware options: `WithBearerAuth()`, `WithRequestID()`, `WithCORS()`. The functional `Option` type and the `With*()` pattern were established here. **Swagger UI** (`22f8a69`): `WithSwagger()` added. The initial implementation served a static Swagger UI backed by a placeholder spec; this was later replaced in Phase 3. **WebSocket** (`22f8a69`): `WithWSHandler()` added, mounting any `http.Handler` at `GET /ws`. `Engine.Channels()` added to aggregate channel names from `StreamGroup` implementations. By the end of Phase 1, the module had 36 tests covering the engine lifecycle, health endpoint, response helpers, bearer auth, request ID propagation, CORS, and WebSocket mounting. --- ### Phase 2 — 21 Middleware Options (143 tests) Commits `d760e77` through `8ba1716` Phase 2 expanded the middleware library in four waves, reaching 21 `With*()` options total. **Wave 1 — Security and Identity** (`d760e77` through `8f3e496`): The Authentik integration was the most significant addition of this wave. - `WithAuthentik()` — permissive forward-auth middleware. Reads `X-authentik-*` headers when `TrustedProxy: true`, validates JWTs via OIDC discovery when `Issuer` and `ClientID` are set. Fail-open: unauthenticated requests are never rejected by this middleware alone. - `RequireAuth()`, `RequireGroup()` — explicit guards for protected routes, returning 401 and 403 respectively via the standard `Fail()` envelope. - `GetUser()` — context accessor for the current `AuthentikUser`. - `AuthentikUser` — carries Username, Email, Name, UID, Groups, Entitlements, and JWT. `HasGroup()` convenience method added. - Live integration tests added in `authentik_integration_test.go`, guarded by environment variables. - `WithSecure()` — HSTS, X-Frame-Options DENY, X-Content-Type-Options nosniff, strict referrer policy. SSL redirect deliberately omitted to work correctly behind a TLS-terminating proxy. **Wave 2 — Compression and Logging** (`6521b90` through `6bb7195`): - `WithTimeout(d)` — per-request deadline via gin-contrib/timeout. Returns 504 with the standard Fail() envelope on expiry. - `WithGzip(level...)` — gzip response compression; defaults to `gzip.DefaultCompression`. - `WithBrotli(level...)` — Brotli compression via `andybalholm/brotli`. Custom `brotliHandler` wrapping `brotli.HTTPCompressor`. - `WithSlog(logger)` — structured request logging via gin-contrib/slog. Falls back to `slog.Default()` when nil is passed. - `WithStatic(prefix, root)` — static file serving via gin-contrib/static; directory listing disabled. **Wave 3 — Auth, Caching, Streaming** (`0ab962a` through `7b3f99e`): - `WithCache(ttl)` — in-memory GET response cache. Custom `cacheWriter` intercepts the response body without affecting the downstream handler. `X-Cache: HIT` on served cache entries. - `WithSessions(name, secret)` — cookie-backed server sessions via gin-contrib/sessions. - `WithAuthz(enforcer)` — Casbin policy-based authorisation via gin-contrib/authz. Subject from HTTP Basic Auth. - `WithHTTPSign(secrets, opts...)` — HTTP Signatures verification via gin-contrib/httpsign. - `WithSSE(broker)` — Server-Sent Events at `GET /events`. `SSEBroker` added with `Publish()`, channel filtering, 64-event per-client buffer, and `Drain()` for graceful shutdown. **Wave 4 — Infrastructure and Protocol** (`a612d85` through `8ba1716`): - `WithLocation()` — reverse proxy header detection via gin-contrib/location/v2. - `WithI18n(cfg...)` — Accept-Language parsing and BCP 47 locale matching via `golang.org/x/text/language`. `GetLocale()` and `GetMessage()` context accessors added. - `WithGraphQL(schema, opts...)` — gqlgen `ExecutableSchema` mounting. `WithPlayground()` and `WithGraphQLPath()` sub-options. Playground at `{path}/playground`. - `WithPprof()` — Go runtime profiling at `/debug/pprof/`. - `WithExpvar()` — expvar runtime metrics at `/debug/vars`. - `WithTracing(name, opts...)` — OpenTelemetry distributed tracing via otelgin. `NewTracerProvider()` convenience helper added. W3C `traceparent` propagation. At the end of Phase 2, the module had 143 tests. --- ### Phase 3 — OpenAPI, ToolBridge, SDK Codegen (176 tests) Commits `465bd60` through `1910aec` Phase 3 upgraded the Swagger integration from a placeholder to a full runtime OpenAPI 3.1 pipeline and added two new subsystems: `ToolBridge` and `SDKGenerator`. **DescribableGroup interface** (`465bd60`): `DescribableGroup` added to `group.go`, extending `RouteGroup` with `Describe() []RouteDescription`. `RouteDescription` carries HTTP method, path, summary, description, tags, and JSON Schema maps for request body and response data. This was the contract that `SpecBuilder` and `ToolBridge` would both consume. **ToolBridge** (`2b63c7b`): `ToolBridge` added to `bridge.go`. Converts `ToolDescriptor` values into `POST /{tool_name}` Gin routes and implements `DescribableGroup` so those routes appear in the OpenAPI spec. Designed to bridge the MCP tool model (as used by go-ai) into the REST world. `Tools()` accessor added for external enumeration. **SpecBuilder** (`3e96f9b`): `SpecBuilder` added to `openapi.go`. Generates a complete OpenAPI 3.1 JSON document from registered `RouteGroup` and `DescribableGroup` values. Includes the built-in `GET /health` endpoint, shared `Error` and `Meta` component schemas, and the `Response[T]` envelope schema wrapping every response body. Tags are derived from all group names, not just describable ones. **Spec export** (`e94283b`): `ExportSpec()` and `ExportSpecToFile()` added to `export.go`. Supports `"json"` and `"yaml"` output formats. YAML output is produced by unmarshalling the JSON then re-encoding with `gopkg.in/yaml.v3` at 2-space indentation. Parent directories created automatically by `ExportSpecToFile()`. **Swagger refactor** (`303779f`): `registerSwagger()` in `swagger.go` rewritten to use `SpecBuilder` rather than the previous placeholder. A `swaggerSpec` wrapper satisfies the `swag.Spec` interface and builds the spec lazily on first access via `sync.Once`. A `swaggerSeq` atomic counter assigns unique instance names so multiple `Engine` instances in the same test binary do not collide in the global `swag` registry. **SDK codegen** (`a09a4e9`, `1910aec`): `SDKGenerator` added to `codegen.go`. Wraps `openapi-generator-cli` to generate client SDKs for 11 target languages. `SupportedLanguages()` returns the list in sorted order (the sort was added in `1910aec` to ensure deterministic output in tests and documentation). At the end of Phase 3, the module has 176 tests. --- ## Known Limitations ### 1. Cache has no size limit `WithCache(ttl)` stores all successful GET responses in memory with no maximum entry count or total size bound. For a server receiving requests to many distinct URLs, the cache will grow without bound. A LRU eviction policy or a configurable maximum is the natural next step. ### 2. SDK codegen requires an external binary `SDKGenerator.Generate()` shells out to `openapi-generator-cli`. This requires a JVM and the openapi-generator JAR to be installed on the host. `Available()` checks whether the CLI is on `PATH` but there is no programmatic fallback. Packaging `openapi-generator-cli` via a Docker wrapper or replacing it with a pure-Go generator would remove this external dependency. ### 3. OpenAPI spec generation is build-time only `SpecBuilder.Build()` generates the spec from `Describe()` return values, which are static at the time of construction. Dynamic route generation (for example, routes registered after `New()` returns) is not reflected in the spec. This matches the current design — all groups must be registered before `Serve()` is called — but it would conflict with any future dynamic route registration model. ### 4. i18n message map is a lightweight bridge only `WithI18n()` accepts a `Messages map[string]map[string]string` for simple key-value lookups. It does not support pluralisation, gender inflection, argument interpolation, or any of the grammar features provided by `go-i18n`. Applications requiring production-grade localisation should use `go-i18n` directly and use `GetLocale()` to pass the detected locale to it. ### 5. Authentik JWT validation performs OIDC discovery on first request `getOIDCProvider()` performs an OIDC discovery request on first use and caches the resulting `*oidc.Provider` by issuer URL. This is lazy — the first request to a non-public path will incur a network round-trip to the issuer. A warm-up call during application startup would eliminate this latency from the first real request. ### 6. ToolBridge has no input validation `ToolBridge.Add()` accepts a `ToolDescriptor` with `InputSchema` and `OutputSchema` maps, but these are used only for OpenAPI documentation. The registered `gin.HandlerFunc` is responsible for its own input validation. There is no automatic binding or validation of incoming request bodies against the declared JSON Schema. ### 7. SSEBroker.Drain() does not wait for clients to disconnect `Drain()` closes all client event channels to signal disconnection but returns immediately without waiting for client goroutines to exit. In a graceful shutdown sequence, there is a brief window where client HTTP connections remain open. The engine's 10-second shutdown deadline covers this window in practice, but there is no explicit coordination.