core/go
8
0
Fork 0
generated from lthn/LEM
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: remove dispatch queue, add ecosystem map and docs index

Browse source 
Delete TODO.md (satellite dispatch queue — all tasks complete) and
FINDINGS.md (go-i18n research — absorbed into go-i18n docs).
Add docs/ecosystem.md mapping all 19 satellite Go packages with
dependency graph and Forge SSH URLs. Rewrite docs/index.md as a
proper documentation hub with section organisation.

Co-Authored-By: Virgil <virgil@lethean.io>
This commit is contained in:
Snider 2026-02-20 15:10:56 +00:00
parent 19521c8f18
commit bcf2d3be48
4 changed files with 632 additions and 195 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

77
FINDINGS.md
View file

@ -1,77 +0,0 @@
# FINDINGS.md — Core Go Research
## go-i18n (forge.lthn.ai/core/go-i18n)
**Explored**: 2026-02-19
**Location**: `/Users/snider/Code/host-uk/go-i18n`
**Module**: `forge.lthn.ai/core/go-i18n`
**State**: 20 commits on main, clean, all tests pass
**Lines**: ~5,800 across 32 files (14 test files)
**Deps**: only `golang.org/x/text`
### What It Is
A **grammar engine** — not a translation file manager. Provides:
1. **Forward composition**: `PastTense()`, `Gerund()`, `Pluralize()`, `Article()`, handlers
2. **Reverse grammar**: Tokeniser reads grammar tables backwards to extract structure
3. **GrammarImprint**: Feature vector projection (content → grammar fingerprint, lossy)
4. **Multiplier**: Deterministic training data augmentation (no LLM)
Consumers (core/cli, apps) bring their own translation files. go-i18n provides the grammar primitives.
### Current Capabilities
| Feature | Status | Notes |
|---------|--------|-------|
| Grammar primitives (past/gerund/plural/article) | Working | 100 irregular verbs, 40 irregular nouns |
| Magic namespace handlers (i18n.label/progress/count/done/fail/numeric) | Working | 6 handler types |
| Service + message lookup | Working | Thread-safe, fallback chain |
| Subject builder (S()) | Working | Fluent API with count/gender/location/formality |
| Plural categories (CLDR) | Working | 7+ languages |
| RTL/LTR detection | Working | 12+ RTL languages |
| Number formatting | Working | Locale-specific separators |
| Reversal tokeniser | Working | 3-tier: JSON → irregular → regular morphology |
| GrammarImprint similarity | Working | Weighted cosine (verbs 30%, tense 20%, nouns 25%) |
| Multiplier expand | Working | Tense + number flipping, dedup, round-trip verify |
### What's Missing / Incomplete
| Gap | Priority | Notes |
|-----|----------|-------|
| Reference distribution builder | High | Process scored seeds → calibrate imprints |
| Non-English grammar tables | Medium | Only en.json exists, reversal needs gram.* per language |
| Ambiguity resolution | Medium | "run", "file", "test" are both verb and noun |
| Domain vocabulary expansion | Low | 150+ words, needs legal/medical/financial |
| Poindexter integration | Deferred | Awaiting Poindexter library |
| TIM container image | Deferred | Distroless Go binary for confidential compute |
### Key Architecture Decisions
- **Bijective grammar tables**: Forward and reverse use same JSON → reversal is deterministic
- **Lossy projection**: GrammarImprint intentionally loses content, preserves only structure
- **No LLM dependency**: Multiplier generates variants purely from morphological rules
- **Consumer translations are external**: go-i18n doesn't ship or manage app-specific locale files
- **gram.* keys are sacred**: Agents MUST NOT flatten — grammar engine depends on nested structure
### pkg/i18n in core/go
- Full i18n framework with 34 locale files — but locale data is bad/stale
- Only imported by `pkg/cli/` which has been extracted to `core/cli`
- Effectively orphaned in core/go
- Can be removed once core/cli imports go-i18n directly
- The locale files need full rework, not migration
---
## CoreDeno (PR #9 — merged)
**Explored**: 2026-02-19
Deno sidecar for core-gui JS runtime. Go↔Deno bidirectional bridge:
- Go→Deno: JSON-RPC over Unix socket (module lifecycle)
- Deno→Go: gRPC over Unix socket (file I/O, store, manifest)
- Each module in isolated Deno Worker with declared permissions
- Marketplace: git clone + ed25519 manifest verification + SQLite registry
10 security/correctness issues found and fixed in review.

52
TODO.md
View file

@ -1,52 +0,0 @@
# TODO.md — Core Go Dispatch Queue
Tasks dispatched from core/go orchestration to satellite repos.
Format: `- [ ] REPO: task description` / `- [x]` when done.
---
## go-i18n (forge.lthn.ai/core/go-i18n)
### Phase 1: Harden the Engine
- [ ] **go-i18n: Add CLAUDE.md** — Document the grammar engine contract: what it is (grammar primitives + reversal), what it isn't (translation file manager). Include build/test commands, the gram.* sacred rule, and the agent-flattening prohibition.
- [ ] **go-i18n: Ambiguity resolution for dual-class words** — Words like "run", "file", "test", "check", "build" are both verb and noun. Tokeniser currently picks first match. Need context-aware disambiguation (look at surrounding tokens: article before → noun, after subject → verb).
- [ ] **go-i18n: Extend irregular verb coverage** — Audit against common dev/ops vocabulary. Missing forms cause silent fallback to regular rules which may produce wrong output (e.g. "builded" instead of "built").
- [ ] **go-i18n: Add benchmarks** — `grammar_test.go` and `reversal/tokeniser_test.go` need `Benchmark*` functions. The engine will run in hot paths (TIM, Poindexter) — need baseline numbers.
### Phase 2: Reference Distribution + 1B Classification Pipeline
#### 2a: 1B Pre-Classification (based on LEK-1B benchmarks)
- [ ] **go-i18n: Classification benchmark suite** — `classify_bench_test.go` with 200+ domain-tagged sentences. Categories: {technical, creative, ethical, casual}. Ground truth for calibrating 1B pre-tags.
- [ ] **go-i18n: 1B pre-sort pipeline tool** — CLI/func that reads JSONL corpus, classifies via LEK-Gemma3-1B, writes back with `domain_1b` field. Target: ~5K sentences/sec on M3.
- [ ] **go-i18n: 1B vs 27B calibration check** — Sample 500 sentences, classify with both, measure agreement. 75% baseline from benchmarks, technical↔creative is known weak spot.
- [x] **go-i18n: Article/irregular validator** — `validate.go` + `validate_test.go` (14 tests). `ValidateArticle()`, `ValidateIrregular()`, batch variants. Commit `3c55d91`.
#### 2b: Reference Distributions
- [ ] **go-i18n: Reference distribution builder** — Process 88K scored seeds through tokeniser + imprint. Pre-sort by `domain_1b` tag. Output: per-category reference distributions as JSON.
- [ ] **go-i18n: Imprint comparator** — Distance metrics (cosine, KL divergence, Mahalanobis) against reference distributions. Classification signal with confidence. Poindexter integration point.
- [ ] **go-i18n: Cross-domain anomaly detection** — Flag texts where 1B domain tag disagrees with imprint classification. Training signal or genuine cross-domain text — both valuable.
### Phase 3: Multi-Language
- [ ] **go-i18n: Grammar table format spec** — Document the exact JSON schema for `gram.*` keys so new languages can be added. Currently only inferred from `en.json`.
- [ ] **go-i18n: French grammar tables** — First non-English language. French has gendered nouns, complex verb conjugation, elision rules. Good stress test for the grammar engine's language-agnostic design.
---
## core/go (this repo)
- [ ] **core/go: Remove pkg/i18n dependency** — Once core/cli imports go-i18n directly, remove `pkg/i18n/` from this repo. The locale files are bad data and shouldn't be migrated.
- [ ] **core/go: Update go.work** — Add go-i18n to the workspace for local development (`go work use ../go-i18n`).
---
## Workflow
1. Virgil (this session) writes tasks above after research
2. Second GoLand session opens the target repo and works from this TODO
3. When a task is done, mark `[x]` and note the commit/PR
4. If a task reveals new work, add it here
5. Scale to other repos once pattern is proven

457
docs/ecosystem.md Normal file
View file

@ -0,0 +1,457 @@
# Core Go Ecosystem
The Core Go ecosystem is a set of 19 standalone Go modules that form the infrastructure backbone for the host-uk platform and the Lethean network. All modules are hosted under the `forge.lthn.ai/core/` organisation. Each module has its own repository, independent versioning, and a `docs/` directory.
The CLI framework documented in the rest of this site (`forge.lthn.ai/core/cli`) is one node in this graph. The satellite packages listed here are separate repositories that the CLI imports or that stand alone as libraries.
---
## Module Index
| Package | Module Path | Managed By |
|---------|-------------|-----------|
| [go-inference](#go-inference) | `forge.lthn.ai/core/go-inference` | Virgil |
| [go-mlx](#go-mlx) | `forge.lthn.ai/core/go-mlx` | Virgil |
| [go-rocm](#go-rocm) | `forge.lthn.ai/core/go-rocm` | Charon |
| [go-ml](#go-ml) | `forge.lthn.ai/core/go-ml` | Virgil |
| [go-ai](#go-ai) | `forge.lthn.ai/core/go-ai` | Virgil |
| [go-agentic](#go-agentic) | `forge.lthn.ai/core/go-agentic` | Charon |
| [go-rag](#go-rag) | `forge.lthn.ai/core/go-rag` | Charon |
| [go-i18n](#go-i18n) | `forge.lthn.ai/core/go-i18n` | Virgil |
| [go-html](#go-html) | `forge.lthn.ai/core/go-html` | Charon |
| [go-crypt](#go-crypt) | `forge.lthn.ai/core/go-crypt` | Virgil |
| [go-scm](#go-scm) | `forge.lthn.ai/core/go-scm` | Charon |
| [go-p2p](#go-p2p) | `forge.lthn.ai/core/go-p2p` | Charon |
| [go-devops](#go-devops) | `forge.lthn.ai/core/go-devops` | Virgil |
| [go-help](#go-help) | `forge.lthn.ai/core/go-help` | Charon |
| [go-ratelimit](#go-ratelimit) | `forge.lthn.ai/core/go-ratelimit` | Charon |
| [go-session](#go-session) | `forge.lthn.ai/core/go-session` | Charon |
| [go-store](#go-store) | `forge.lthn.ai/core/go-store` | Charon |
| [go-ws](#go-ws) | `forge.lthn.ai/core/go-ws` | Charon |
| [go-webview](#go-webview) | `forge.lthn.ai/core/go-webview` | Charon |
---
## Dependency Graph
The graph below shows import relationships. An arrow `A → B` means A imports B.
```
go-inference (no dependencies — foundation contract)
↑
├── go-mlx (CGO, Apple Silicon Metal GPU)
├── go-rocm (AMD ROCm, llama-server subprocess)
└── go-ml (scoring engine, backends, orchestrator)
↑
└── go-ai (MCP hub, 49 tools)
↑
└── go-agentic (service lifecycle, allowances)
go-rag (Qdrant + Ollama, standalone)
↑
└── go-ai
go-i18n (grammar engine, standalone; Phase 2a imports go-mlx)
go-crypt (standalone)
↑
├── go-p2p (UEPS wire protocol)
└── go-scm (AgentCI dispatch)
go-store (SQLite KV, standalone)
↑
├── go-ratelimit (sliding window limiter)
├── go-session (transcript parser)
└── go-agentic
go-ws (WebSocket hub, standalone)
↑
└── go-ai
go-webview (CDP client, standalone)
↑
└── go-ai
go-html (DOM compositor, standalone)
go-help (help catalogue, standalone)
go-devops (Ansible, build, infrastructure — imports go-scm)
```
The CLI framework (`forge.lthn.ai/core/cli`) has internal equivalents of several of these packages (`pkg/rag`, `pkg/ws`, `pkg/webview`, `pkg/i18n`) that were developed in parallel. The satellite packages are the canonical standalone versions intended for use outside the CLI binary.
---
## Package Descriptions
### go-inference
**Module:** `forge.lthn.ai/core/go-inference`
Zero-dependency interface package that defines the common contract for all inference backends in the ecosystem:
- `TextModel` — the top-level model interface (`Generate`, `Stream`, `Close`)
- `Backend` — hardware/runtime abstraction (Metal, ROCm, CPU, remote)
- `Token` — streaming token type with metadata
No concrete implementations live here. Any package that needs to call inference without depending on a specific hardware library imports `go-inference` and receives an implementation at runtime.
---
### go-mlx
**Module:** `forge.lthn.ai/core/go-mlx`
Native Metal GPU inference for Apple Silicon using CGO bindings to `mlx-c` (the C API for Apple's MLX framework). Implements the `go-inference` interfaces.
Build requirements:
- macOS 13+ (Ventura) on Apple Silicon
- `mlx-c` installed (`brew install mlx`)
- CGO enabled: `CGO_CFLAGS` and `CGO_LDFLAGS` must reference the mlx-c headers and library
Features:
- Loads GGUF and MLX-format models
- Streaming token generation directly on GPU
- Quantised model support (Q4, Q8)
- Phase 4 backend abstraction in progress — will allow hot-swapping backends at runtime
Local path: `/Users/snider/Code/go-mlx`
---
### go-rocm
**Module:** `forge.lthn.ai/core/go-rocm`
AMD ROCm GPU inference for Linux. Rather than using CGO, this package manages a `llama-server` subprocess (from llama.cpp) compiled with ROCm support and communicates over its HTTP API.
Features:
- Subprocess lifecycle management (start, health-check, restart on crash)
- OpenAI-compatible HTTP client wrapping llama-server's API
- Implements `go-inference` interfaces
- Targeted at the homelab RX 7800 XT running Ubuntu 24.04
Managed by Charon (Linux homelab).
---
### go-ml
**Module:** `forge.lthn.ai/core/go-ml`
Scoring engine, backend registry, and agent orchestration layer. The hub that connects models from `go-mlx`, `go-rocm`, and future backends into a unified interface.
Features:
- Backend registry: register multiple inference backends, select by capability
- Scoring pipeline: evaluate model outputs against rubrics
- Agent orchestrator: coordinate multi-step inference tasks
- ~3.5K LOC
---
### go-ai
**Module:** `forge.lthn.ai/core/go-ai`
MCP (Model Context Protocol) server hub with 49 registered tools. Acts as the primary facade for AI capabilities in the ecosystem.
Features:
- 49 MCP tools covering file operations, RAG, metrics, process management, WebSocket, and CDP/webview
- Imports `go-ml`, `go-rag`, `go-mlx`
- Can run as stdio MCP server or TCP MCP server
- AI usage metrics recorded to JSONL
Run the MCP server:
```bash
# stdio (for Claude Desktop / Claude Code)
core mcp serve
# TCP
MCP_ADDR=:9000 core mcp serve
```
---
### go-agentic
**Module:** `forge.lthn.ai/core/go-agentic`
Service lifecycle and allowance management for autonomous agents. Handles:
- Agent session tracking and state persistence
- Allowance system: budget constraints on tool calls, token usage, and wall-clock time
- Integration with `go-store` for persistence
- REST client for the PHP `core-agentic` backend
Managed by Charon.
---
### go-rag
**Module:** `forge.lthn.ai/core/go-rag`
Retrieval-Augmented Generation pipeline using Qdrant for vector storage and Ollama for embeddings.
Features:
- `ChunkMarkdown`: semantic splitting by H2 headers and paragraphs with overlap
- `Ingest`: crawl a directory of Markdown files, embed, and store in Qdrant
- `Query`: semantic search returning ranked `QueryResult` slices
- `FormatResultsContext`: formats results as XML tags for LLM prompt injection
- Clients: `QdrantClient` and `OllamaClient` wrapping their respective Go SDKs
Managed by Charon.
---
### go-i18n
**Module:** `forge.lthn.ai/core/go-i18n`
Grammar engine for natural-language generation. Goes beyond key-value lookup tables to handle pluralisation, verb conjugation, past tense, gerunds, and semantic sentence construction ("Subject verbed object").
Features:
- `T(key, args...)` — main translation function
- `S(noun, value)` — semantic subject with grammatical context
- Language rules defined in JSON; algorithmic fallbacks for irregular verbs
- **GrammarImprint**: a linguistic hash (reversal of the grammar engine) used as a semantic fingerprint — part of the Lethean identity verification stack
- Phase 2a (imports `go-mlx` for language model-assisted reversal) currently blocked on `go-mlx` Phase 4
Local path: `/Users/snider/Code/go-i18n`
---
### go-html
**Module:** `forge.lthn.ai/core/go-html`
HLCRF DOM compositor — a programmatic HTML/DOM construction library targeting both server-side rendering and WASM (browser).
HLCRF stands for Header, Left, Content, Right, Footer — the region layout model used throughout the CLI's terminal UI and web rendering layer.
Features:
- Composable region-based layout (mirrors the terminal `Composite` in `pkg/cli`)
- WASM build target: runs in the browser without JavaScript
- Used by the LEM Chat UI and web SDK generation
Managed by Charon.
---
### go-crypt
**Module:** `forge.lthn.ai/core/go-crypt`
Cryptographic primitives, authentication, and trust policy enforcement.
Features:
- Password hashing (Argon2id with tuned parameters)
- Symmetric encryption (ChaCha20-Poly1305, AES-GCM)
- Key derivation (HKDF, Scrypt)
- OpenPGP challenge-response authentication
- Trust policies: define and evaluate access rules
- Foundation for the UEPS (User-controlled Encryption Policy System) wire protocol in `go-p2p`
---
### go-scm
**Module:** `forge.lthn.ai/core/go-scm`
Source control management and CI integration, including the AgentCI dispatch system.
Features:
- Forgejo and Gitea API clients (typed wrappers)
- GitHub integration via the `gh` CLI
- `AgentCI`: dispatches AI work items to agent runners over SSH using Charm stack libraries (`soft-serve`, `keygen`, `melt`, `wishlist`)
- PR lifecycle management: create, review, merge, label
- JSONL job journal for audit trails
Managed by Charon.
---
### go-p2p
**Module:** `forge.lthn.ai/core/go-p2p`
Peer-to-peer mesh networking implementing the UEPS (User-controlled Encryption Policy System) wire protocol.
Features:
- UEPS: consent-gated TLV frames with Ed25519 consent tokens and an Intent-Broker
- Peer discovery and mesh routing
- Encrypted relay transport
- Integration with `go-crypt` for all cryptographic operations
This is a core component of the Lethean Web3 network layer.
Managed by Charon (Linux homelab).
---
### go-devops
**Module:** `forge.lthn.ai/core/go-devops`
Infrastructure automation, build tooling, and release pipeline utilities, intended as a standalone library form of what the Core CLI provides as commands.
Features:
- Ansible-lite engine (native Go SSH playbook execution)
- LinuxKit image building and VM lifecycle
- Multi-target binary build and release
- Integration with `go-scm` for repository operations
---
### go-help
**Module:** `forge.lthn.ai/core/go-help`
Embedded documentation catalogue with full-text search and an optional HTTP server for serving help content.
Features:
- YAML-frontmatter Markdown topic parsing
- In-memory reverse index with title/heading/body scoring
- Snippet extraction with keyword highlighting
- `HTTP server` mode: serve the catalogue as a documentation site
- Used by the `core pkg search` command and the `pkg/help` package inside the CLI
Managed by Charon.
---
### go-ratelimit
**Module:** `forge.lthn.ai/core/go-ratelimit`
Sliding-window rate limiter with a SQLite persistence backend.
Features:
- Token bucket and sliding-window algorithms
- SQLite backend via `go-store` for durable rate state across restarts
- HTTP middleware helper
- Used by `go-ai` and `go-agentic` to enforce per-agent API quotas
Managed by Charon.
---
### go-session
**Module:** `forge.lthn.ai/core/go-session`
Claude Code JSONL transcript parser and visualisation toolkit (standalone version of `pkg/session` inside the CLI).
Features:
- `ParseTranscript(path)`: reads `.jsonl` session files and reconstructs tool use timelines
- `ListSessions(dir)`: scans a Claude projects directory for session files
- `Search(dir, query)`: full-text search across sessions
- `RenderHTML(sess, path)`: single-file HTML visualisation
- `RenderMP4(sess, path)`: terminal video replay via VHS
Managed by Charon.
---
### go-store
**Module:** `forge.lthn.ai/core/go-store`
SQLite-backed key-value store with reactive change notification.
Features:
- `Get`, `Set`, `Delete`, `List` over typed keys
- `Watch(key, handler)`: register a callback that fires on change
- `OnChange(handler)`: subscribe to all changes
- Used by `go-ratelimit`, `go-session`, and `go-agentic` for lightweight persistence
Managed by Charon.
---
### go-ws
**Module:** `forge.lthn.ai/core/go-ws`
WebSocket hub with channel-based subscriptions and an optional Redis pub/sub bridge for multi-instance deployments.
Features:
- Hub pattern: central registry of connected clients
- Channel routing: `SendToChannel(topic, msg)` delivers only to subscribers
- Redis bridge: publish messages from one instance, receive on all
- HTTP handler: `hub.Handler()` for embedding in any Go HTTP server
- `SendProcessOutput(id, line)`: convenience method for streaming process logs
Managed by Charon.
---
### go-webview
**Module:** `forge.lthn.ai/core/go-webview`
Chrome DevTools Protocol (CDP) client for browser automation, testing, and AI-driven web interaction (standalone version of `pkg/webview` inside the CLI).
Features:
- Navigation, click, type, screenshot
- `Evaluate(script)`: arbitrary JavaScript execution with result capture
- Console capture and filtering
- Angular-aware helpers: `WaitForAngular()`, `GetNgModel(selector)`
- `ActionSequence`: chain interactions into a single call
- Used by `go-ai` to expose browser tools to MCP agents
Managed by Charon.
---
## Forge Repository Paths
All repositories are hosted at `forge.lthn.ai` (Forgejo). SSH access uses port 2223:
```
ssh://git@forge.lthn.ai:2223/core/go-inference.git
ssh://git@forge.lthn.ai:2223/core/go-mlx.git
ssh://git@forge.lthn.ai:2223/core/go-rocm.git
ssh://git@forge.lthn.ai:2223/core/go-ml.git
ssh://git@forge.lthn.ai:2223/core/go-ai.git
ssh://git@forge.lthn.ai:2223/core/go-agentic.git
ssh://git@forge.lthn.ai:2223/core/go-rag.git
ssh://git@forge.lthn.ai:2223/core/go-i18n.git
ssh://git@forge.lthn.ai:2223/core/go-html.git
ssh://git@forge.lthn.ai:2223/core/go-crypt.git
ssh://git@forge.lthn.ai:2223/core/go-scm.git
ssh://git@forge.lthn.ai:2223/core/go-p2p.git
ssh://git@forge.lthn.ai:2223/core/go-devops.git
ssh://git@forge.lthn.ai:2223/core/go-help.git
ssh://git@forge.lthn.ai:2223/core/go-ratelimit.git
ssh://git@forge.lthn.ai:2223/core/go-session.git
ssh://git@forge.lthn.ai:2223/core/go-store.git
ssh://git@forge.lthn.ai:2223/core/go-ws.git
ssh://git@forge.lthn.ai:2223/core/go-webview.git
```
HTTPS authentication is not available on Forge. Always use SSH remotes.
---
## Go Workspace Setup
The satellite packages can be used together in a Go workspace. After cloning the repositories you need:
```bash
go work init
go work use ./go-inference ./go-mlx ./go-rag ./go-ai # add as needed
go work sync
```
The CLI repository already uses a Go workspace that includes `cmd/core-gui`, `cmd/bugseti`, and `cmd/examples/*`.
---
## See Also
- [index.md](index.md) — Main documentation hub
- [getting-started.md](getting-started.md) — CLI installation
- [configuration.md](configuration.md) — `repos.yaml` registry format

241
docs/index.md
View file

@ -1,98 +1,207 @@
# Core CLI
# Core Go Framework — Documentation
Core is a unified CLI for the host-uk ecosystem - build, release, and deploy Go, Wails, PHP, and container workloads.
Core is a Go framework and unified CLI for the host-uk ecosystem. It provides two complementary things: a **dependency injection container** for building Go services and Wails v3 desktop applications, and a **command-line tool** for managing the full development lifecycle across Go, PHP, and container workloads.
## Installation
The `core` binary is the single entry point for all development tasks: testing, building, releasing, multi-repo management, MCP servers, and AI-assisted workflows.
```bash
# Via Go (recommended)
go install forge.lthn.ai/core/cli/cmd/core@latest
---
# Or download binary from releases
curl -Lo core https://forge.lthn.ai/core/cli/releases/latest/download/core-$(go env GOOS)-$(go env GOARCH)
chmod +x core && sudo mv core /usr/local/bin/
## Getting Started
# Verify
core doctor
| Document | Description |
|----------|-------------|
| [Getting Started](getting-started.md) | Install the CLI, run your first build, and set up a multi-repo workspace |
| [User Guide](user-guide.md) | Key concepts and daily workflow patterns |
| [Workflows](workflows.md) | End-to-end task sequences for common scenarios |
| [FAQ](faq.md) | Answers to common questions |
---
## Architecture
| Document | Description |
|----------|-------------|
| [Package Standards](pkg/PACKAGE_STANDARDS.md) | Canonical patterns for creating packages: Service struct, factory, IPC, thread safety |
| [pkg/i18n — Grammar](pkg/i18n/GRAMMAR.md) | Grammar engine internals and language rule format |
| [pkg/i18n — Extending](pkg/i18n/EXTENDING.md) | How to add new locales and translation files |
### Framework Architecture Summary
The Core framework (`pkg/framework`) is a dependency injection container built around three ideas:
**Service registry.** Services are registered via factory functions and retrieved with type-safe generics:
```go
core, _ := framework.New(
framework.WithService(mypackage.NewService(opts)),
)
svc, _ := framework.ServiceFor[*mypackage.Service](core, "mypackage")
```
See [Getting Started](getting-started.md) for all installation options including building from source.
**Lifecycle.** Services implementing `Startable` or `Stoppable` are called automatically during boot and shutdown.
**ACTION bus.** Services communicate by broadcasting typed messages via `core.ACTION(msg)` and registering handlers via `core.RegisterAction()`. This decouples packages without requiring direct imports between them.
---
## Command Reference
See [cmd/](cmd/) for full command documentation.
The `core` CLI is documented command-by-command in `docs/cmd/`:
| Command | Description |
|---------|-------------|
| [go](cmd/go/) | Go development (test, fmt, lint, cov) |
| [php](cmd/php/) | Laravel/PHP development |
| [build](cmd/build/) | Build Go, Wails, Docker, LinuxKit projects |
| [ci](cmd/ci/) | Publish releases (dry-run by default) |
| [sdk](cmd/sdk/) | SDK generation and validation |
| [dev](cmd/dev/) | Multi-repo workflow + dev environment |
| [pkg](cmd/pkg/) | Package search and install |
| [vm](cmd/vm/) | LinuxKit VM management |
| [docs](cmd/docs/) | Documentation management |
| [setup](cmd/setup/) | Clone repos from registry |
| [doctor](cmd/doctor/) | Check development environment |
| [cmd/](cmd/) | Full command index |
| [cmd/go/](cmd/go/) | Go development: test, fmt, lint, coverage, mod, work |
| [cmd/php/](cmd/php/) | Laravel/PHP development: dev server, test, deploy |
| [cmd/build/](cmd/build/) | Build Go, Wails, Docker, LinuxKit projects |
| [cmd/ci/](cmd/ci/) | Publish releases to GitHub, Docker, npm, Homebrew |
| [cmd/sdk/](cmd/sdk/) | SDK generation and OpenAPI validation |
| [cmd/dev/](cmd/dev/) | Multi-repo workflow and sandboxed dev environment |
| [cmd/ai/](cmd/ai/) | AI task management and Claude integration |
| [cmd/pkg/](cmd/pkg/) | Package search and install |
| [cmd/vm/](cmd/vm/) | LinuxKit VM management |
| [cmd/docs/](cmd/docs/) | Documentation sync and management |
| [cmd/setup/](cmd/setup/) | Clone repositories from a registry |
| [cmd/doctor/](cmd/doctor/) | Verify development environment |
| [cmd/test/](cmd/test/) | Run Go tests with coverage reporting |
## Quick Start
---
```bash
# Go development
core go test # Run tests
core go test --coverage # With coverage
core go fmt # Format code
core go lint # Lint code
## Packages
# Build
core build # Auto-detect and build
core build --targets linux/amd64,darwin/arm64
The Core repository contains the following internal packages. Full API analysis for each is available in the batch analysis documents listed under [Reference](#reference).
# Release (dry-run by default)
core ci # Preview release
core ci --we-are-go-for-launch # Actually publish
### Foundation
# Multi-repo workflow
core dev work # Status + commit + push
core dev work --status # Just show status
| Package | Description |
|---------|-------------|
| `pkg/framework` | Dependency injection container; re-exports `pkg/framework/core` |
| `pkg/log` | Structured logger with `Err` error type, operation chains, and log rotation |
| `pkg/config` | 12-factor config management layered over Viper; accepts `io.Medium` |
| `pkg/io` | Filesystem abstraction (`Medium` interface); `NewSandboxed`, `MockMedium` |
| `pkg/crypt` | Opinionated crypto: Argon2id passwords, ChaCha20 encryption, HMAC |
| `pkg/cache` | File-based JSON cache with TTL expiry |
| `pkg/i18n` | Grammar engine with pluralisation, verb conjugation, semantic sentences |
# PHP development
core php dev # Start dev environment
core php test # Run tests
```
### CLI and Interaction
## Configuration
| Package | Description |
|---------|-------------|
| `pkg/cli` | CLI runtime: Cobra wrapping, ANSI styling, prompts, daemon lifecycle |
| `pkg/help` | Embedded documentation catalogue with in-memory full-text search |
| `pkg/session` | Claude Code JSONL transcript parser; HTML and MP4 export |
| `pkg/workspace` | Isolated, PGP-keyed workspace environments with IPC control |
Core uses `.core/` directory for project configuration:
### Build and Release
```
.core/
├── release.yaml # Release targets and settings
├── build.yaml # Build configuration (optional)
└── linuxkit/ # LinuxKit templates
```
| Package | Description |
|---------|-------------|
| `pkg/build` | Project type detection, cross-compilation, archiving, checksums |
| `pkg/release` | Semantic versioning, conventional-commit changelogs, multi-target publishing |
| `pkg/container` | LinuxKit VM lifecycle via QEMU/Hyperkit; template management |
| `pkg/process` | `os/exec` wrapper with ring-buffer output, DAG task runner, ACTION streaming |
| `pkg/jobrunner` | Poll-dispatch automation engine with JSONL audit journal |
And `repos.yaml` in workspace root for multi-repo management.
### Source Control and Hosting
## Guides
| Package | Description |
|---------|-------------|
| `pkg/git` | Multi-repo status, push, pull; concurrent status checks |
| `pkg/repos` | `repos.yaml` registry loader; topological dependency ordering |
| `pkg/gitea` | Gitea API client with PR metadata extraction |
| `pkg/forge` | Forgejo API client with PR metadata extraction |
| `pkg/plugin` | Git-based CLI extension system |
- [Getting Started](getting-started.md) - Installation and first steps
- [Workflows](workflows.md) - Common task sequences
- [Troubleshooting](troubleshooting.md) - When things go wrong
- [Migration](migration.md) - Moving from legacy tools
### AI and Agentic
| Package | Description |
|---------|-------------|
| `pkg/mcp` | MCP server exposing file, process, RAG, and CDP tools to AI agents |
| `pkg/rag` | RAG pipeline: Markdown chunking, Ollama embeddings, Qdrant vector search |
| `pkg/ai` | Facade over RAG and metrics; `QueryRAGForTask` for prompt enrichment |
| `pkg/agentic` | REST client for core-agentic; `AutoCommit`, `CreatePR`, `BuildTaskContext` |
| `pkg/agentci` | Configuration bridge for AgentCI dispatch targets |
| `pkg/collect` | Data collection pipeline from GitHub, forums, market APIs |
### Infrastructure and Networking
| Package | Description |
|---------|-------------|
| `pkg/devops` | LinuxKit dev environment lifecycle; SSH bridging; project auto-detection |
| `pkg/ansible` | Native Go Ansible-lite engine; SSH playbook execution without the CLI |
| `pkg/webview` | Chrome DevTools Protocol client; Angular-aware automation |
| `pkg/ws` | WebSocket hub with channel-based subscriptions |
| `pkg/unifi` | UniFi controller client for network management |
| `pkg/auth` | OpenPGP challenge-response authentication; air-gapped flow |
---
## Workflows
| Document | Description |
|----------|-------------|
| [Workflows](workflows.md) | Go build and release, PHP deploy, multi-repo daily workflow, hotfix |
| [Migration](migration.md) | Migrating from `push-all.sh`, raw `go` commands, `goreleaser`, or manual git |
---
## Reference
- [Configuration](configuration.md) - All config options
- [Glossary](glossary.md) - Term definitions
| Document | Description |
|----------|-------------|
| [Configuration](configuration.md) | `.core/` directory, `release.yaml`, `build.yaml`, `php.yaml`, `repos.yaml`, environment variables |
| [Glossary](glossary.md) | Term definitions: target, workspace, registry, publisher, dry-run |
| [Troubleshooting](troubleshooting.md) | Installation failures, build errors, release issues, multi-repo problems, PHP issues |
| [Claude Code Skill](skill/) | Install the `core` skill to teach Claude Code how to use this CLI |
## Claude Code Skill
### Historical Package Analysis
Install the skill to teach Claude Code how to use the Core CLI:
The following documents were generated by an automated analysis pipeline (Gemini, February 2026) to extract architecture, public API, and test coverage notes from each package. They remain valid as architectural reference.
```bash
curl -fsSL https://raw.githubusercontent.com/host-uk/core/main/.claude/skills/core/install.sh | bash
```
| Document | Packages Covered |
|----------|-----------------|
| [pkg-batch1-analysis.md](pkg-batch1-analysis.md) | `pkg/log`, `pkg/config`, `pkg/io`, `pkg/crypt`, `pkg/auth` |
| [pkg-batch2-analysis.md](pkg-batch2-analysis.md) | `pkg/cli`, `pkg/help`, `pkg/session`, `pkg/workspace` |
| [pkg-batch3-analysis.md](pkg-batch3-analysis.md) | `pkg/build`, `pkg/container`, `pkg/process`, `pkg/jobrunner` |
| [pkg-batch4-analysis.md](pkg-batch4-analysis.md) | `pkg/git`, `pkg/repos`, `pkg/gitea`, `pkg/forge`, `pkg/release` |
| [pkg-batch5-analysis.md](pkg-batch5-analysis.md) | `pkg/agentci`, `pkg/agentic`, `pkg/ai`, `pkg/rag` |
| [pkg-batch6-analysis.md](pkg-batch6-analysis.md) | `pkg/ansible`, `pkg/devops`, `pkg/framework`, `pkg/mcp`, `pkg/plugin`, `pkg/unifi`, `pkg/webview`, `pkg/ws`, `pkg/collect`, `pkg/i18n`, `pkg/cache` |
See [skill/](skill/) for details.
### Design Plans
| Document | Description |
|----------|-------------|
| [plans/2026-02-05-core-ide-job-runner-design.md](plans/2026-02-05-core-ide-job-runner-design.md) | Autonomous job runner design for core-ide: poller, dispatcher, MCP handler registry, JSONL training data |
| [plans/2026-02-05-core-ide-job-runner-plan.md](plans/2026-02-05-core-ide-job-runner-plan.md) | Implementation plan for the job runner |
| [plans/2026-02-05-mcp-integration.md](plans/2026-02-05-mcp-integration.md) | MCP integration design notes |
| [plans/2026-02-17-lem-chat-design.md](plans/2026-02-17-lem-chat-design.md) | LEM Chat Web Components design: streaming SSE, zero-dependency vanilla UI |
---
## Satellite Packages
The Core ecosystem extends across 19 standalone Go modules, all hosted under `forge.lthn.ai/core/`. Each has its own repository and `docs/` directory.
See [ecosystem.md](ecosystem.md) for the full map, module paths, and dependency graph.
| Package | Purpose |
|---------|---------|
| [go-inference](ecosystem.md#go-inference) | Shared `TextModel`/`Backend`/`Token` interfaces — the common contract |
| [go-mlx](ecosystem.md#go-mlx) | Native Metal GPU inference via CGO/mlx-c (Apple Silicon) |
| [go-rocm](ecosystem.md#go-rocm) | AMD ROCm GPU inference via llama-server subprocess |
| [go-ml](ecosystem.md#go-ml) | Scoring engine, backends, agent orchestrator |
| [go-ai](ecosystem.md#go-ai) | MCP hub with 49 registered tools |
| [go-agentic](ecosystem.md#go-agentic) | Service lifecycle and allowance management for agents |
| [go-rag](ecosystem.md#go-rag) | Qdrant vector search and Ollama embeddings |
| [go-i18n](ecosystem.md#go-i18n) | Grammar engine, reversal, GrammarImprint |
| [go-html](ecosystem.md#go-html) | HLCRF DOM compositor and WASM target |
| [go-crypt](ecosystem.md#go-crypt) | Cryptographic primitives, auth, trust policies |
| [go-scm](ecosystem.md#go-scm) | SCM/CI integration and AgentCI dispatch |
| [go-p2p](ecosystem.md#go-p2p) | P2P mesh networking and UEPS wire protocol |
| [go-devops](ecosystem.md#go-devops) | Ansible automation, build tooling, infrastructure, release |
| [go-help](ecosystem.md#go-help) | YAML help catalogue with full-text search and HTTP server |
| [go-ratelimit](ecosystem.md#go-ratelimit) | Sliding-window rate limiter with SQLite backend |
| [go-session](ecosystem.md#go-session) | Claude Code JSONL transcript parser |
| [go-store](ecosystem.md#go-store) | SQLite key-value store with `Watch`/`OnChange` |
| [go-ws](ecosystem.md#go-ws) | WebSocket hub with Redis bridge |
| [go-webview](ecosystem.md#go-webview) | Chrome DevTools Protocol automation client |