go-api/docs/history.md

go-api — Project History and Known Limitations

Module: forge.lthn.ai/core/go-api

---

Origins

go-api was created as the dedicated HTTP framework for the Lethean Go ecosystem. The motivation was to give every Go package in the stack a consistent way to expose REST endpoints without each package taking its own opinion on routing, middleware, response formatting, or OpenAPI generation. It was scaffolded independently from the start — it was never extracted from a monolith — and has no forge.lthn.ai/core/* dependencies. This keeps it at the bottom of the import graph: every other package can import go-api, but go-api imports nothing from the ecosystem.

---

Development Phases

Phase 1 — Core Engine (36 tests)

Commits 889391a through 22f8a69

The initial phase established the foundational abstractions that all subsequent work builds on.

Scaffold (889391a): Module path forge.lthn.ai/core/go-api created. go.mod initialised with Gin as the only direct dependency.

Response envelope (7835837): Response[T], Error, and Meta types defined. OK(), Fail(), and Paginated() helpers added. The generic envelope was established from the start rather than retrofitted.

RouteGroup interface (6f5fb69): RouteGroup (Name, BasePath, RegisterRoutes) and StreamGroup (Channels) interfaces defined in group.go. The interface-driven extension model was the core design decision of Phase 1.

Engine (db75c88): Engine struct added with New(), Register(), Handler(), Serve(), and graceful shutdown. Default listen address :8080. Built-in GET /health endpoint. Panic recovery via gin.Recovery() always applied first.

Bearer auth, request ID, CORS (d21734d): First three middleware options: WithBearerAuth(), WithRequestID(), WithCORS(). The functional Option type and the With*() pattern were established here.

Swagger UI (22f8a69): WithSwagger() added. The initial implementation served a static Swagger UI backed by a placeholder spec; this was later replaced in Phase 3.

WebSocket (22f8a69): WithWSHandler() added, mounting any http.Handler at GET /ws. Engine.Channels() added to aggregate channel names from StreamGroup implementations.

By the end of Phase 1, the module had 36 tests covering the engine lifecycle, health endpoint, response helpers, bearer auth, request ID propagation, CORS, and WebSocket mounting.

---

Phase 2 — 21 Middleware Options (143 tests)

Commits d760e77 through 8ba1716

Phase 2 expanded the middleware library in four waves, reaching 21 With*() options total.

Wave 1 — Security and Identity (d760e77 through 8f3e496):

The Authentik integration was the most significant addition of this wave.

• WithAuthentik() — permissive forward-auth middleware. Reads X-authentik-* headers when TrustedProxy: true, validates JWTs via OIDC discovery when Issuer and ClientID are set. Fail-open: unauthenticated requests are never rejected by this middleware alone.
• RequireAuth(), RequireGroup() — explicit guards for protected routes, returning 401 and 403 respectively via the standard Fail() envelope.
• GetUser() — context accessor for the current AuthentikUser.
• AuthentikUser — carries Username, Email, Name, UID, Groups, Entitlements, and JWT. HasGroup() convenience method added.
• Live integration tests added in authentik_integration_test.go, guarded by environment variables.
• WithSecure() — HSTS, X-Frame-Options DENY, X-Content-Type-Options nosniff, strict referrer policy. SSL redirect deliberately omitted to work correctly behind a TLS-terminating proxy.

Wave 2 — Compression and Logging (6521b90 through 6bb7195):

• WithTimeout(d) — per-request deadline via gin-contrib/timeout. Returns 504 with the standard Fail() envelope on expiry.
• WithGzip(level...) — gzip response compression; defaults to gzip.DefaultCompression.
• WithBrotli(level...) — Brotli compression via andybalholm/brotli. Custom brotliHandler wrapping brotli.HTTPCompressor.
• WithSlog(logger) — structured request logging via gin-contrib/slog. Falls back to slog.Default() when nil is passed.
• WithStatic(prefix, root) — static file serving via gin-contrib/static; directory listing disabled.

Wave 3 — Auth, Caching, Streaming (0ab962a through 7b3f99e):

• WithCache(ttl) — in-memory GET response cache. Custom cacheWriter intercepts the response body without affecting the downstream handler. X-Cache: HIT on served cache entries.
• WithSessions(name, secret) — cookie-backed server sessions via gin-contrib/sessions.
• WithAuthz(enforcer) — Casbin policy-based authorisation via gin-contrib/authz. Subject from HTTP Basic Auth.
• WithHTTPSign(secrets, opts...) — HTTP Signatures verification via gin-contrib/httpsign.
• WithSSE(broker) — Server-Sent Events at GET /events. SSEBroker added with Publish(), channel filtering, 64-event per-client buffer, and Drain() for graceful shutdown.

Wave 4 — Infrastructure and Protocol (a612d85 through 8ba1716):

• WithLocation() — reverse proxy header detection via gin-contrib/location/v2.
• WithI18n(cfg...) — Accept-Language parsing and BCP 47 locale matching via golang.org/x/text/language. GetLocale() and GetMessage() context accessors added.
• WithGraphQL(schema, opts...) — gqlgen ExecutableSchema mounting. WithPlayground() and WithGraphQLPath() sub-options. Playground at {path}/playground.
• WithPprof() — Go runtime profiling at /debug/pprof/.
• WithExpvar() — expvar runtime metrics at /debug/vars.
• WithTracing(name, opts...) — OpenTelemetry distributed tracing via otelgin. NewTracerProvider() convenience helper added. W3C traceparent propagation.

At the end of Phase 2, the module had 143 tests.

---

Phase 3 — OpenAPI, ToolBridge, SDK Codegen (176 tests)

Commits 465bd60 through 1910aec

Phase 3 upgraded the Swagger integration from a placeholder to a full runtime OpenAPI 3.1 pipeline and added two new subsystems: ToolBridge and SDKGenerator.

DescribableGroup interface (465bd60): DescribableGroup added to group.go, extending RouteGroup with Describe() []RouteDescription. RouteDescription carries HTTP method, path, summary, description, tags, and JSON Schema maps for request body and response data. This was the contract that SpecBuilder and ToolBridge would both consume.

ToolBridge (2b63c7b): ToolBridge added to bridge.go. Converts ToolDescriptor values into POST /{tool_name} Gin routes and implements DescribableGroup so those routes appear in the OpenAPI spec. Designed to bridge the MCP tool model (as used by go-ai) into the REST world. Tools() accessor added for external enumeration.

SpecBuilder (3e96f9b): SpecBuilder added to openapi.go. Generates a complete OpenAPI 3.1 JSON document from registered RouteGroup and DescribableGroup values. Includes the built-in GET /health endpoint, shared Error and Meta component schemas, and the Response[T] envelope schema wrapping every response body. Tags are derived from all group names, not just describable ones.

Spec export (e94283b): ExportSpec() and ExportSpecToFile() added to export.go. Supports "json" and "yaml" output formats. YAML output is produced by unmarshalling the JSON then re-encoding with gopkg.in/yaml.v3 at 2-space indentation. Parent directories created automatically by ExportSpecToFile().

Swagger refactor (303779f): registerSwagger() in swagger.go rewritten to use SpecBuilder rather than the previous placeholder. A swaggerSpec wrapper satisfies the swag.Spec interface and builds the spec lazily on first access via sync.Once. A swaggerSeq atomic counter assigns unique instance names so multiple Engine instances in the same test binary do not collide in the global swag registry.

SDK codegen (a09a4e9, 1910aec): SDKGenerator added to codegen.go. Wraps openapi-generator-cli to generate client SDKs for 11 target languages. SupportedLanguages() returns the list in sorted order (the sort was added in 1910aec to ensure deterministic output in tests and documentation).

At the end of Phase 3, the module has 176 tests.

---

Known Limitations

1. Cache has no size limit

WithCache(ttl) stores all successful GET responses in memory with no maximum entry count or total size bound. For a server receiving requests to many distinct URLs, the cache will grow without bound. A LRU eviction policy or a configurable maximum is the natural next step.

2. SDK codegen requires an external binary

SDKGenerator.Generate() shells out to openapi-generator-cli. This requires a JVM and the openapi-generator JAR to be installed on the host. Available() checks whether the CLI is on PATH but there is no programmatic fallback. Packaging openapi-generator-cli via a Docker wrapper or replacing it with a pure-Go generator would remove this external dependency.

3. OpenAPI spec generation is build-time only

SpecBuilder.Build() generates the spec from Describe() return values, which are static at the time of construction. Dynamic route generation (for example, routes registered after New() returns) is not reflected in the spec. This matches the current design — all groups must be registered before Serve() is called — but it would conflict with any future dynamic route registration model.

4. i18n message map is a lightweight bridge only

WithI18n() accepts a Messages map[string]map[string]string for simple key-value lookups. It does not support pluralisation, gender inflection, argument interpolation, or any of the grammar features provided by go-i18n. Applications requiring production-grade localisation should use go-i18n directly and use GetLocale() to pass the detected locale to it.

5. Authentik JWT validation performs OIDC discovery on first request

getOIDCProvider() performs an OIDC discovery request on first use and caches the resulting *oidc.Provider by issuer URL. This is lazy — the first request to a non-public path will incur a network round-trip to the issuer. A warm-up call during application startup would eliminate this latency from the first real request.

6. ToolBridge has no input validation

ToolBridge.Add() accepts a ToolDescriptor with InputSchema and OutputSchema maps, but these are used only for OpenAPI documentation. The registered gin.HandlerFunc is responsible for its own input validation. There is no automatic binding or validation of incoming request bodies against the declared JSON Schema.

7. SSEBroker.Drain() does not wait for clients to disconnect

Drain() closes all client event channels to signal disconnection but returns immediately without waiting for client goroutines to exit. In a graceful shutdown sequence, there is a brief window where client HTTP connections remain open. The engine's 10-second shutdown deadline covers this window in practice, but there is no explicit coordination.