From 86b8a9136c3b4c48fd00590f299f8b415598ee39 Mon Sep 17 00:00:00 2001
From: Snider <snider@host.uk.com>
Date: Fri, 6 Mar 2026 16:25:13 +0000
Subject: [PATCH] docs: add core/docs help engine design

Merge go-help into core/docs as pkg/help, replace Hugo/Docsy with
native static generator + go-html HLCRF layout. Content owned by
package repos (docs/ convention), collected by go-devops for builds.

Co-Authored-By: Virgil <virgil@lethean.io>
---
 .../2026-03-06-docs-help-engine-design.md     | 123 ++++++++++++++++++
 1 file changed, 123 insertions(+)
 create mode 100644 docs/plans/2026-03-06-docs-help-engine-design.md

diff --git a/docs/plans/2026-03-06-docs-help-engine-design.md b/docs/plans/2026-03-06-docs-help-engine-design.md
new file mode 100644
index 0000000..be4897a
--- /dev/null
+++ b/docs/plans/2026-03-06-docs-help-engine-design.md
@@ -0,0 +1,123 @@
+# Design: core/docs with go-help Engine
+
+**Date**: 6 Mar 2026
+**Status**: Approved
+**Replaces**: Hugo/Docsy docs-site, standalone go-help repo
+**Domain**: core.help (BunnyCDN)
+
+## Problem
+
+Documentation lives in three disconnected repos: go-help (engine), docs (content), docs-site (Hugo wrapper). Hugo needs Node/PostCSS in CI, adds Docsy as a Go module dependency, and produces a site that can't be served from inside a Go binary. We want one tool that embeds docs in binaries AND generates the public site.
+
+## Decision
+
+Merge go-help into `core/docs` as `pkg/help`. Replace Hugo/Docsy with go-help's native static generator. Use go-html's HLCRF compositor for the layout wrapper. Archive go-help and docs-site on forge.
+
+## Architecture
+
+```
+core/docs/                          (forge.lthn.ai/core/docs)
+  pkg/help/                         go-help library
+    catalog.go                      topic store + search
+    search.go                       inverted index, stemming, fuzzy
+    parser.go                       YAML frontmatter + section extraction
+    render.go                       goldmark markdown -> HTML
+    server.go                       HTTP server (HTML + JSON API)
+    generate.go                     static site generator
+    layout.go                       NEW: go-html HLCRF wrapper
+    ingest.go                       CLI help text -> Topic
+    stemmer.go                      Porter-style stemmer
+    templates/                      embedded HTML templates (updated)
+  content/                          aggregated docs tree (built by core docs collect)
+    cli/
+    go/
+    php/
+    api/
+    packages/
+    deploy/
+  cmd/docs-server/                  optional standalone server binary
+  go.mod                            forge.lthn.ai/core/docs
+```
+
+### Dependency chain
+
+```
+pkg/help -> go-html -> go-i18n -> go-inference
+pkg/help -> goldmark
+pkg/help -> yaml.v3
+```
+
+No database, no network dependency, no external service at runtime.
+
+## Content Ownership
+
+Docs live in their package repos under a `docs/` folder (convention, zero config). The `core/docs/content/` tree is the aggregated output, built by go-devops:
+
+1. `core docs collect` — walks workspace, copies each repo's `docs/` into `content/{repo-name}/`
+2. `core docs build` — go-help parses collected tree, generates static site to `dist/`
+3. `core docs deploy` — pushes `dist/` to BunnyCDN at core.help
+
+For binary builds (CoreGUI, core-php), the binary embeds only the docs from its own deps (determined by go.mod imports, not the full workspace).
+
+## Dual-Mode Serving
+
+### In-app (CoreGUI webview / `core help serve`)
+
+`pkg/help.NewServer()` serves from `//go:embed` content. No network. Runs on localhost, CoreGUI opens a webview to it. Fragment URLs work: `#rate-limiting` scrolls to the section.
+
+### Public (core.help)
+
+Static HTML generated by `pkg/help.Generate()`, deployed to BunnyCDN. Same URLs, same fragments. External docs can deep-link: `core.help/topics/cli-dev-work#sync-interval`.
+
+## Layout: go-html HLCRF
+
+Replaces go-help's current `html/template` files with go-html compositor:
+
+| Slot | Element | Content |
+|------|---------|---------|
+| H | `<header role="banner">` | Nav bar + search input + "core.help" branding |
+| L | `<aside role="complementary">` | Topic tree grouped by tag, collapsible sections |
+| C | `<main role="main">` | Rendered Markdown with section anchors (id attrs on headings) |
+| F | `<footer role="contentinfo">` | Licence, version, links |
+
+Semantic HTML with ARIA roles. Deterministic `data-block` IDs for CSS targeting. Dark theme CSS stays inline (no external stylesheets).
+
+## Fragment Linking
+
+go-help's `GenerateID()` produces URL-safe slugs from headings:
+- `"Rate Limiting"` -> `rate-limiting`
+- `"API / Rate Limits"` -> `api-rate-limits`
+
+Every heading rendered with an `id` attribute. Works identically in-app and on the public site.
+
+The JSON API (`/api/topics/{id}`) returns sections with IDs, so a client can request a subsection programmatically.
+
+## Search
+
+No changes to the search engine:
+- Inverted index with stemming and fuzzy matching (Levenshtein, max distance 2)
+- Phrase search via quoted strings
+- Relevance scoring: title 10x, phrase 8x, section 5x, tag 3x
+- Static site: `search-index.json` with client-side JS
+- In-app server: server-side `catalog.Search()`
+
+## Repo Lifecycle
+
+| Repo | Action |
+|------|--------|
+| go-help | Archive on forge after merge |
+| docs-site | Archive on forge |
+| docs | Becomes Go module, gains `pkg/help/` + `go.mod` |
+
+## Not In Scope
+
+- i18n for docs content (English only; go-i18n is a transitive dep of go-html, not used for content)
+- Authentication (core.help is public, in-app is local)
+- Wiki sync (separate concern: kb.yaml / OpenBrain)
+- Database (catalog is in-memory)
+
+## Testing
+
+- All existing go-help tests (~94% coverage) move to `pkg/help/`
+- New tests for go-html HLCRF layout rendering (output structure, section anchor IDs)
+- Integration test: parse `docs/` folder -> generate static site -> verify file structure + search index