0d3479839d
Standard docs trio matching the pattern used across all Go ecosystem repos. Covers Engine architecture, 21 With*() options, RouteGroup interfaces, OpenAPI generation pipeline, and three-phase development history (176 tests). Co-Authored-By: Virgil <virgil@lethean.io>
219 lines
11 KiB
Markdown
219 lines
11 KiB
Markdown
<!-- SPDX-License-Identifier: EUPL-1.2 -->
|
|
|
|
# go-api — Project History and Known Limitations
|
|
|
|
Module: `forge.lthn.ai/core/go-api`
|
|
|
|
---
|
|
|
|
## Origins
|
|
|
|
`go-api` was created as the dedicated HTTP framework for the Lethean Go ecosystem. The motivation
|
|
was to give every Go package in the stack a consistent way to expose REST endpoints without each
|
|
package taking its own opinion on routing, middleware, response formatting, or OpenAPI generation.
|
|
It was scaffolded independently from the start — it was never extracted from a monolith — and has
|
|
no `forge.lthn.ai/core/*` dependencies. This keeps it at the bottom of the import graph: every
|
|
other package can import go-api, but go-api imports nothing from the ecosystem.
|
|
|
|
---
|
|
|
|
## Development Phases
|
|
|
|
### Phase 1 — Core Engine (36 tests)
|
|
|
|
Commits `889391a` through `22f8a69`
|
|
|
|
The initial phase established the foundational abstractions that all subsequent work builds on.
|
|
|
|
**Scaffold** (`889391a`):
|
|
Module path `forge.lthn.ai/core/go-api` created. `go.mod` initialised with Gin as the only
|
|
direct dependency.
|
|
|
|
**Response envelope** (`7835837`):
|
|
`Response[T]`, `Error`, and `Meta` types defined. `OK()`, `Fail()`, and `Paginated()` helpers
|
|
added. The generic envelope was established from the start rather than retrofitted.
|
|
|
|
**RouteGroup interface** (`6f5fb69`):
|
|
`RouteGroup` (Name, BasePath, RegisterRoutes) and `StreamGroup` (Channels) interfaces defined
|
|
in `group.go`. The interface-driven extension model was the core design decision of Phase 1.
|
|
|
|
**Engine** (`db75c88`):
|
|
`Engine` struct added with `New()`, `Register()`, `Handler()`, `Serve()`, and graceful shutdown.
|
|
Default listen address `:8080`. Built-in `GET /health` endpoint. Panic recovery via `gin.Recovery()`
|
|
always applied first.
|
|
|
|
**Bearer auth, request ID, CORS** (`d21734d`):
|
|
First three middleware options: `WithBearerAuth()`, `WithRequestID()`, `WithCORS()`.
|
|
The functional `Option` type and the `With*()` pattern were established here.
|
|
|
|
**Swagger UI** (`22f8a69`):
|
|
`WithSwagger()` added. The initial implementation served a static Swagger UI backed by a placeholder
|
|
spec; this was later replaced in Phase 3.
|
|
|
|
**WebSocket** (`22f8a69`):
|
|
`WithWSHandler()` added, mounting any `http.Handler` at `GET /ws`. `Engine.Channels()` added to
|
|
aggregate channel names from `StreamGroup` implementations.
|
|
|
|
By the end of Phase 1, the module had 36 tests covering the engine lifecycle, health endpoint,
|
|
response helpers, bearer auth, request ID propagation, CORS, and WebSocket mounting.
|
|
|
|
---
|
|
|
|
### Phase 2 — 21 Middleware Options (143 tests)
|
|
|
|
Commits `d760e77` through `8ba1716`
|
|
|
|
Phase 2 expanded the middleware library in four waves, reaching 21 `With*()` options total.
|
|
|
|
**Wave 1 — Security and Identity** (`d760e77` through `8f3e496`):
|
|
|
|
The Authentik integration was the most significant addition of this wave.
|
|
|
|
- `WithAuthentik()` — permissive forward-auth middleware. Reads `X-authentik-*` headers when
|
|
`TrustedProxy: true`, validates JWTs via OIDC discovery when `Issuer` and `ClientID` are set.
|
|
Fail-open: unauthenticated requests are never rejected by this middleware alone.
|
|
- `RequireAuth()`, `RequireGroup()` — explicit guards for protected routes, returning 401 and 403
|
|
respectively via the standard `Fail()` envelope.
|
|
- `GetUser()` — context accessor for the current `AuthentikUser`.
|
|
- `AuthentikUser` — carries Username, Email, Name, UID, Groups, Entitlements, and JWT. `HasGroup()`
|
|
convenience method added.
|
|
- Live integration tests added in `authentik_integration_test.go`, guarded by environment variables.
|
|
- `WithSecure()` — HSTS, X-Frame-Options DENY, X-Content-Type-Options nosniff, strict referrer
|
|
policy. SSL redirect deliberately omitted to work correctly behind a TLS-terminating proxy.
|
|
|
|
**Wave 2 — Compression and Logging** (`6521b90` through `6bb7195`):
|
|
|
|
- `WithTimeout(d)` — per-request deadline via gin-contrib/timeout. Returns 504 with the standard
|
|
Fail() envelope on expiry.
|
|
- `WithGzip(level...)` — gzip response compression; defaults to `gzip.DefaultCompression`.
|
|
- `WithBrotli(level...)` — Brotli compression via `andybalholm/brotli`. Custom `brotliHandler`
|
|
wrapping `brotli.HTTPCompressor`.
|
|
- `WithSlog(logger)` — structured request logging via gin-contrib/slog. Falls back to
|
|
`slog.Default()` when nil is passed.
|
|
- `WithStatic(prefix, root)` — static file serving via gin-contrib/static; directory listing
|
|
disabled.
|
|
|
|
**Wave 3 — Auth, Caching, Streaming** (`0ab962a` through `7b3f99e`):
|
|
|
|
- `WithCache(ttl)` — in-memory GET response cache. Custom `cacheWriter` intercepts the response
|
|
body without affecting the downstream handler. `X-Cache: HIT` on served cache entries.
|
|
- `WithSessions(name, secret)` — cookie-backed server sessions via gin-contrib/sessions.
|
|
- `WithAuthz(enforcer)` — Casbin policy-based authorisation via gin-contrib/authz. Subject from
|
|
HTTP Basic Auth.
|
|
- `WithHTTPSign(secrets, opts...)` — HTTP Signatures verification via gin-contrib/httpsign.
|
|
- `WithSSE(broker)` — Server-Sent Events at `GET /events`. `SSEBroker` added with `Publish()`,
|
|
channel filtering, 64-event per-client buffer, and `Drain()` for graceful shutdown.
|
|
|
|
**Wave 4 — Infrastructure and Protocol** (`a612d85` through `8ba1716`):
|
|
|
|
- `WithLocation()` — reverse proxy header detection via gin-contrib/location/v2.
|
|
- `WithI18n(cfg...)` — Accept-Language parsing and BCP 47 locale matching via
|
|
`golang.org/x/text/language`. `GetLocale()` and `GetMessage()` context accessors added.
|
|
- `WithGraphQL(schema, opts...)` — gqlgen `ExecutableSchema` mounting. `WithPlayground()` and
|
|
`WithGraphQLPath()` sub-options. Playground at `{path}/playground`.
|
|
- `WithPprof()` — Go runtime profiling at `/debug/pprof/`.
|
|
- `WithExpvar()` — expvar runtime metrics at `/debug/vars`.
|
|
- `WithTracing(name, opts...)` — OpenTelemetry distributed tracing via otelgin. `NewTracerProvider()`
|
|
convenience helper added. W3C `traceparent` propagation.
|
|
|
|
At the end of Phase 2, the module had 143 tests.
|
|
|
|
---
|
|
|
|
### Phase 3 — OpenAPI, ToolBridge, SDK Codegen (176 tests)
|
|
|
|
Commits `465bd60` through `1910aec`
|
|
|
|
Phase 3 upgraded the Swagger integration from a placeholder to a full runtime OpenAPI 3.1 pipeline
|
|
and added two new subsystems: `ToolBridge` and `SDKGenerator`.
|
|
|
|
**DescribableGroup interface** (`465bd60`):
|
|
`DescribableGroup` added to `group.go`, extending `RouteGroup` with `Describe() []RouteDescription`.
|
|
`RouteDescription` carries HTTP method, path, summary, description, tags, and JSON Schema maps
|
|
for request body and response data. This was the contract that `SpecBuilder` and `ToolBridge`
|
|
would both consume.
|
|
|
|
**ToolBridge** (`2b63c7b`):
|
|
`ToolBridge` added to `bridge.go`. Converts `ToolDescriptor` values into `POST /{tool_name}`
|
|
Gin routes and implements `DescribableGroup` so those routes appear in the OpenAPI spec. Designed
|
|
to bridge the MCP tool model (as used by go-ai) into the REST world. `Tools()` accessor added
|
|
for external enumeration.
|
|
|
|
**SpecBuilder** (`3e96f9b`):
|
|
`SpecBuilder` added to `openapi.go`. Generates a complete OpenAPI 3.1 JSON document from registered
|
|
`RouteGroup` and `DescribableGroup` values. Includes the built-in `GET /health` endpoint, shared
|
|
`Error` and `Meta` component schemas, and the `Response[T]` envelope schema wrapping every response
|
|
body. Tags are derived from all group names, not just describable ones.
|
|
|
|
**Spec export** (`e94283b`):
|
|
`ExportSpec()` and `ExportSpecToFile()` added to `export.go`. Supports `"json"` and `"yaml"`
|
|
output formats. YAML output is produced by unmarshalling the JSON then re-encoding with
|
|
`gopkg.in/yaml.v3` at 2-space indentation. Parent directories created automatically by
|
|
`ExportSpecToFile()`.
|
|
|
|
**Swagger refactor** (`303779f`):
|
|
`registerSwagger()` in `swagger.go` rewritten to use `SpecBuilder` rather than the previous
|
|
placeholder. A `swaggerSpec` wrapper satisfies the `swag.Spec` interface and builds the spec
|
|
lazily on first access via `sync.Once`. A `swaggerSeq` atomic counter assigns unique instance
|
|
names so multiple `Engine` instances in the same test binary do not collide in the global
|
|
`swag` registry.
|
|
|
|
**SDK codegen** (`a09a4e9`, `1910aec`):
|
|
`SDKGenerator` added to `codegen.go`. Wraps `openapi-generator-cli` to generate client SDKs
|
|
for 11 target languages. `SupportedLanguages()` returns the list in sorted order (the sort was
|
|
added in `1910aec` to ensure deterministic output in tests and documentation).
|
|
|
|
At the end of Phase 3, the module has 176 tests.
|
|
|
|
---
|
|
|
|
## Known Limitations
|
|
|
|
### 1. Cache has no size limit
|
|
|
|
`WithCache(ttl)` stores all successful GET responses in memory with no maximum entry count or
|
|
total size bound. For a server receiving requests to many distinct URLs, the cache will grow
|
|
without bound. A LRU eviction policy or a configurable maximum is the natural next step.
|
|
|
|
### 2. SDK codegen requires an external binary
|
|
|
|
`SDKGenerator.Generate()` shells out to `openapi-generator-cli`. This requires a JVM and the
|
|
openapi-generator JAR to be installed on the host. `Available()` checks whether the CLI is on
|
|
`PATH` but there is no programmatic fallback. Packaging `openapi-generator-cli` via a Docker
|
|
wrapper or replacing it with a pure-Go generator would remove this external dependency.
|
|
|
|
### 3. OpenAPI spec generation is build-time only
|
|
|
|
`SpecBuilder.Build()` generates the spec from `Describe()` return values, which are static at
|
|
the time of construction. Dynamic route generation (for example, routes registered after
|
|
`New()` returns) is not reflected in the spec. This matches the current design — all groups
|
|
must be registered before `Serve()` is called — but it would conflict with any future dynamic
|
|
route registration model.
|
|
|
|
### 4. i18n message map is a lightweight bridge only
|
|
|
|
`WithI18n()` accepts a `Messages map[string]map[string]string` for simple key-value lookups.
|
|
It does not support pluralisation, gender inflection, argument interpolation, or any of the
|
|
grammar features provided by `go-i18n`. Applications requiring production-grade localisation
|
|
should use `go-i18n` directly and use `GetLocale()` to pass the detected locale to it.
|
|
|
|
### 5. Authentik JWT validation performs OIDC discovery on first request
|
|
|
|
`getOIDCProvider()` performs an OIDC discovery request on first use and caches the resulting
|
|
`*oidc.Provider` by issuer URL. This is lazy — the first request to a non-public path will
|
|
incur a network round-trip to the issuer. A warm-up call during application startup would
|
|
eliminate this latency from the first real request.
|
|
|
|
### 6. ToolBridge has no input validation
|
|
|
|
`ToolBridge.Add()` accepts a `ToolDescriptor` with `InputSchema` and `OutputSchema` maps, but
|
|
these are used only for OpenAPI documentation. The registered `gin.HandlerFunc` is responsible
|
|
for its own input validation. There is no automatic binding or validation of incoming request
|
|
bodies against the declared JSON Schema.
|
|
|
|
### 7. SSEBroker.Drain() does not wait for clients to disconnect
|
|
|
|
`Drain()` closes all client event channels to signal disconnection but returns immediately
|
|
without waiting for client goroutines to exit. In a graceful shutdown sequence, there is a
|
|
brief window where client HTTP connections remain open. The engine's 10-second shutdown
|
|
deadline covers this window in practice, but there is no explicit coordination.
|