go/docs/ecosystem.md

Core Go Ecosystem

The Core Go ecosystem is a set of 19 standalone Go modules that form the infrastructure backbone for the host-uk platform and the Lethean network. All modules are hosted under the forge.lthn.ai/core/ organisation. Each module has its own repository, independent versioning, and a docs/ directory.

The CLI framework documented in the rest of this site (forge.lthn.ai/core/cli) is one node in this graph. The satellite packages listed here are separate repositories that the CLI imports or that stand alone as libraries.

---

Module Index

Package	Module Path	Managed By
go-inference	forge.lthn.ai/core/go-inference	Virgil
go-mlx	forge.lthn.ai/core/go-mlx	Virgil
go-rocm	forge.lthn.ai/core/go-rocm	Charon
go-ml	forge.lthn.ai/core/go-ml	Virgil
go-ai	forge.lthn.ai/core/go-ai	Virgil
go-agentic	forge.lthn.ai/core/go-agentic	Charon
go-rag	forge.lthn.ai/core/go-rag	Charon
go-i18n	forge.lthn.ai/core/go-i18n	Virgil
go-html	forge.lthn.ai/core/go-html	Charon
go-crypt	forge.lthn.ai/core/go-crypt	Virgil
go-scm	forge.lthn.ai/core/go-scm	Charon
go-p2p	forge.lthn.ai/core/go-p2p	Charon
go-devops	forge.lthn.ai/core/go-devops	Virgil
go-help	forge.lthn.ai/core/go-help	Charon
go-ratelimit	forge.lthn.ai/core/go-ratelimit	Charon
go-session	forge.lthn.ai/core/go-session	Charon
go-store	forge.lthn.ai/core/go-store	Charon
go-ws	forge.lthn.ai/core/go-ws	Charon
go-webview	forge.lthn.ai/core/go-webview	Charon

---

Dependency Graph

The graph below shows import relationships. An arrow A → B means A imports B.

go-inference          (no dependencies — foundation contract)
     ↑
     ├── go-mlx       (CGO, Apple Silicon Metal GPU)
     ├── go-rocm      (AMD ROCm, llama-server subprocess)
     └── go-ml        (scoring engine, backends, orchestrator)
              ↑
              └── go-ai  (MCP hub, 49 tools)
                    ↑
                    └── go-agentic  (service lifecycle, allowances)

go-rag               (Qdrant + Ollama, standalone)
     ↑
     └── go-ai

go-i18n              (grammar engine, standalone; Phase 2a imports go-mlx)

go-crypt             (standalone)
     ↑
     ├── go-p2p       (UEPS wire protocol)
     └── go-scm       (AgentCI dispatch)

go-store             (SQLite KV, standalone)
     ↑
     ├── go-ratelimit (sliding window limiter)
     ├── go-session   (transcript parser)
     └── go-agentic

go-ws                (WebSocket hub, standalone)
     ↑
     └── go-ai

go-webview           (CDP client, standalone)
     ↑
     └── go-ai

go-html              (DOM compositor, standalone)

go-help              (help catalogue, standalone)

go-devops            (Ansible, build, infrastructure — imports go-scm)

The CLI framework (forge.lthn.ai/core/cli) has internal equivalents of several of these packages (pkg/rag, pkg/ws, pkg/webview, pkg/i18n) that were developed in parallel. The satellite packages are the canonical standalone versions intended for use outside the CLI binary.

---

Package Descriptions

go-inference

Module: forge.lthn.ai/core/go-inference

Zero-dependency interface package that defines the common contract for all inference backends in the ecosystem:

• TextModel — the top-level model interface (Generate, Stream, Close)
• Backend — hardware/runtime abstraction (Metal, ROCm, CPU, remote)
• Token — streaming token type with metadata

No concrete implementations live here. Any package that needs to call inference without depending on a specific hardware library imports go-inference and receives an implementation at runtime.

---

go-mlx

Module: forge.lthn.ai/core/go-mlx

Native Metal GPU inference for Apple Silicon using CGO bindings to mlx-c (the C API for Apple's MLX framework). Implements the go-inference interfaces.

Build requirements:

• macOS 13+ (Ventura) on Apple Silicon
• mlx-c installed (brew install mlx)
• CGO enabled: CGO_CFLAGS and CGO_LDFLAGS must reference the mlx-c headers and library

Features:

• Loads GGUF and MLX-format models
• Streaming token generation directly on GPU
• Quantised model support (Q4, Q8)
• Phase 4 backend abstraction in progress — will allow hot-swapping backends at runtime

Local path: /Users/snider/Code/go-mlx

---

go-rocm

Module: forge.lthn.ai/core/go-rocm

AMD ROCm GPU inference for Linux. Rather than using CGO, this package manages a llama-server subprocess (from llama.cpp) compiled with ROCm support and communicates over its HTTP API.

Features:

• Subprocess lifecycle management (start, health-check, restart on crash)
• OpenAI-compatible HTTP client wrapping llama-server's API
• Implements go-inference interfaces
• Targeted at the homelab RX 7800 XT running Ubuntu 24.04

Managed by Charon (Linux homelab).

---

go-ml

Module: forge.lthn.ai/core/go-ml

Scoring engine, backend registry, and agent orchestration layer. The hub that connects models from go-mlx, go-rocm, and future backends into a unified interface.

Features:

• Backend registry: register multiple inference backends, select by capability
• Scoring pipeline: evaluate model outputs against rubrics
• Agent orchestrator: coordinate multi-step inference tasks
• ~3.5K LOC

---

go-ai

Module: forge.lthn.ai/core/go-ai

MCP (Model Context Protocol) server hub with 49 registered tools. Acts as the primary facade for AI capabilities in the ecosystem.

Features:

• 49 MCP tools covering file operations, RAG, metrics, process management, WebSocket, and CDP/webview
• Imports go-ml, go-rag, go-mlx
• Can run as stdio MCP server or TCP MCP server
• AI usage metrics recorded to JSONL

Run the MCP server:

# stdio (for Claude Desktop / Claude Code)
core mcp serve

# TCP
MCP_ADDR=:9000 core mcp serve

---

go-agentic

Module: forge.lthn.ai/core/go-agentic

Service lifecycle and allowance management for autonomous agents. Handles:

• Agent session tracking and state persistence
• Allowance system: budget constraints on tool calls, token usage, and wall-clock time
• Integration with go-store for persistence
• REST client for the PHP core-agentic backend

Managed by Charon.

---

go-rag

Module: forge.lthn.ai/core/go-rag

Retrieval-Augmented Generation pipeline using Qdrant for vector storage and Ollama for embeddings.

Features:

• ChunkMarkdown: semantic splitting by H2 headers and paragraphs with overlap
• Ingest: crawl a directory of Markdown files, embed, and store in Qdrant
• Query: semantic search returning ranked QueryResult slices
• FormatResultsContext: formats results as XML tags for LLM prompt injection
• Clients: QdrantClient and OllamaClient wrapping their respective Go SDKs

Managed by Charon.

---

go-i18n

Module: forge.lthn.ai/core/go-i18n

Grammar engine for natural-language generation. Goes beyond key-value lookup tables to handle pluralisation, verb conjugation, past tense, gerunds, and semantic sentence construction ("Subject verbed object").

Features:

• T(key, args...) — main translation function
• S(noun, value) — semantic subject with grammatical context
• Language rules defined in JSON; algorithmic fallbacks for irregular verbs
• GrammarImprint: a linguistic hash (reversal of the grammar engine) used as a semantic fingerprint — part of the Lethean identity verification stack
• Phase 2a (imports go-mlx for language model-assisted reversal) currently blocked on go-mlx Phase 4

Local path: /Users/snider/Code/go-i18n

---

go-html

Module: forge.lthn.ai/core/go-html

HLCRF DOM compositor — a programmatic HTML/DOM construction library targeting both server-side rendering and WASM (browser).

HLCRF stands for Header, Left, Content, Right, Footer — the region layout model used throughout the CLI's terminal UI and web rendering layer.

Features:

• Composable region-based layout (mirrors the terminal Composite in pkg/cli)
• WASM build target: runs in the browser without JavaScript
• Used by the LEM Chat UI and web SDK generation

Managed by Charon.

---

go-crypt

Module: forge.lthn.ai/core/go-crypt

Cryptographic primitives, authentication, and trust policy enforcement.

Features:

• Password hashing (Argon2id with tuned parameters)
• Symmetric encryption (ChaCha20-Poly1305, AES-GCM)
• Key derivation (HKDF, Scrypt)
• OpenPGP challenge-response authentication
• Trust policies: define and evaluate access rules
• Foundation for the UEPS (User-controlled Encryption Policy System) wire protocol in go-p2p

---

go-scm

Module: forge.lthn.ai/core/go-scm

Source control management and CI integration, including the AgentCI dispatch system.

Features:

• Forgejo and Gitea API clients (typed wrappers)
• GitHub integration via the gh CLI
• AgentCI: dispatches AI work items to agent runners over SSH using Charm stack libraries (soft-serve, keygen, melt, wishlist)
• PR lifecycle management: create, review, merge, label
• JSONL job journal for audit trails

Managed by Charon.

---

go-p2p

Module: forge.lthn.ai/core/go-p2p

Peer-to-peer mesh networking implementing the UEPS (User-controlled Encryption Policy System) wire protocol.

Features:

• UEPS: consent-gated TLV frames with Ed25519 consent tokens and an Intent-Broker
• Peer discovery and mesh routing
• Encrypted relay transport
• Integration with go-crypt for all cryptographic operations

This is a core component of the Lethean Web3 network layer.

Managed by Charon (Linux homelab).

---

go-devops

Module: forge.lthn.ai/core/go-devops

Infrastructure automation, build tooling, and release pipeline utilities, intended as a standalone library form of what the Core CLI provides as commands.

Features:

• Ansible-lite engine (native Go SSH playbook execution)
• LinuxKit image building and VM lifecycle
• Multi-target binary build and release
• Integration with go-scm for repository operations

---

go-help

Module: forge.lthn.ai/core/go-help

Embedded documentation catalogue with full-text search and an optional HTTP server for serving help content.

Features:

• YAML-frontmatter Markdown topic parsing
• In-memory reverse index with title/heading/body scoring
• Snippet extraction with keyword highlighting
• HTTP server mode: serve the catalogue as a documentation site
• Used by the core pkg search command and the pkg/help package inside the CLI

Managed by Charon.

---

go-ratelimit

Module: forge.lthn.ai/core/go-ratelimit

Sliding-window rate limiter with a SQLite persistence backend.

Features:

• Token bucket and sliding-window algorithms
• SQLite backend via go-store for durable rate state across restarts
• HTTP middleware helper
• Used by go-ai and go-agentic to enforce per-agent API quotas

Managed by Charon.

---

go-session

Module: forge.lthn.ai/core/go-session

Claude Code JSONL transcript parser and visualisation toolkit (standalone version of pkg/session inside the CLI).

Features:

• ParseTranscript(path): reads .jsonl session files and reconstructs tool use timelines
• ListSessions(dir): scans a Claude projects directory for session files
• Search(dir, query): full-text search across sessions
• RenderHTML(sess, path): single-file HTML visualisation
• RenderMP4(sess, path): terminal video replay via VHS

Managed by Charon.

---

go-store

Module: forge.lthn.ai/core/go-store

SQLite-backed key-value store with reactive change notification.

Features:

• Get, Set, Delete, List over typed keys
• Watch(key, handler): register a callback that fires on change
• OnChange(handler): subscribe to all changes
• Used by go-ratelimit, go-session, and go-agentic for lightweight persistence

Managed by Charon.

---

go-ws

Module: forge.lthn.ai/core/go-ws

WebSocket hub with channel-based subscriptions and an optional Redis pub/sub bridge for multi-instance deployments.

Features:

• Hub pattern: central registry of connected clients
• Channel routing: SendToChannel(topic, msg) delivers only to subscribers
• Redis bridge: publish messages from one instance, receive on all
• HTTP handler: hub.Handler() for embedding in any Go HTTP server
• SendProcessOutput(id, line): convenience method for streaming process logs

Managed by Charon.

---

go-webview

Module: forge.lthn.ai/core/go-webview

Chrome DevTools Protocol (CDP) client for browser automation, testing, and AI-driven web interaction (standalone version of pkg/webview inside the CLI).

Features:

• Navigation, click, type, screenshot
• Evaluate(script): arbitrary JavaScript execution with result capture
• Console capture and filtering
• Angular-aware helpers: WaitForAngular(), GetNgModel(selector)
• ActionSequence: chain interactions into a single call
• Used by go-ai to expose browser tools to MCP agents

Managed by Charon.

---

Forge Repository Paths

All repositories are hosted at forge.lthn.ai (Forgejo). SSH access uses port 2223:

ssh://git@forge.lthn.ai:2223/core/go-inference.git
ssh://git@forge.lthn.ai:2223/core/go-mlx.git
ssh://git@forge.lthn.ai:2223/core/go-rocm.git
ssh://git@forge.lthn.ai:2223/core/go-ml.git
ssh://git@forge.lthn.ai:2223/core/go-ai.git
ssh://git@forge.lthn.ai:2223/core/go-agentic.git
ssh://git@forge.lthn.ai:2223/core/go-rag.git
ssh://git@forge.lthn.ai:2223/core/go-i18n.git
ssh://git@forge.lthn.ai:2223/core/go-html.git
ssh://git@forge.lthn.ai:2223/core/go-crypt.git
ssh://git@forge.lthn.ai:2223/core/go-scm.git
ssh://git@forge.lthn.ai:2223/core/go-p2p.git
ssh://git@forge.lthn.ai:2223/core/go-devops.git
ssh://git@forge.lthn.ai:2223/core/go-help.git
ssh://git@forge.lthn.ai:2223/core/go-ratelimit.git
ssh://git@forge.lthn.ai:2223/core/go-session.git
ssh://git@forge.lthn.ai:2223/core/go-store.git
ssh://git@forge.lthn.ai:2223/core/go-ws.git
ssh://git@forge.lthn.ai:2223/core/go-webview.git

HTTPS authentication is not available on Forge. Always use SSH remotes.

---

Go Workspace Setup

The satellite packages can be used together in a Go workspace. After cloning the repositories you need:

go work init
go work use ./go-inference ./go-mlx ./go-rag ./go-ai   # add as needed
go work sync

The CLI repository already uses a Go workspace that includes cmd/core-gui, cmd/bugseti, and cmd/examples/*.

---

See Also

• index.md — Main documentation hub
• getting-started.md — CLI installation
• configuration.md — repos.yaml registry format