No description
Find a file
Snider bfa5353205
Add streaming API to pkg/io and optimize agentic context gathering (#313)
* feat(io): add streaming API to Medium interface and optimize agentic context

- Added ReadStream and WriteStream to io.Medium interface.
- Implemented streaming methods in local and mock mediums.
- Updated pkg/agentic/context.go to use streaming I/O with LimitReader.
- Added 5000-byte truncation limit for all AI context file reads to reduce memory usage.
- Documented when to use streaming vs full-file APIs in io.Medium.

* feat(io): optimize streaming API and fix PR feedback

- Fixed resource leak in agentic context by using defer for closing file streams.
- Improved truncation logic in agentic context to handle multibyte characters correctly by checking byte length before string conversion.
- Added comprehensive documentation to ReadStream and WriteStream in local medium.
- Added unit tests for ReadStream and WriteStream in local medium.
- Applied formatting and fixed auto-merge CI configuration.

* feat(io): add streaming API and fix CI failures (syntax fix)

- Introduced ReadStream and WriteStream to io.Medium interface.
- Implemented streaming methods in local and mock mediums.
- Optimized agentic context with streaming reads and truncation logic.
- Fixed syntax error in local client tests by overwriting the file.
- Fixed auto-merge CI by adding checkout and repository context.
- Applied formatting fixes.
2026-02-05 11:00:49 +00:00
.claude feat(i18n): expand CLI translations and fix noun form detection 2026-02-01 07:39:49 +00:00
.core Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
.gemini feat(mcp): add workspace root validation to prevent path traversal (#100) 2026-02-01 21:59:34 +00:00
.github Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
docs Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
internal Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
pkg Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
tasks/plans refactor: move plans to tasks/, framework docs to core-gui 2026-01-29 15:01:46 +00:00
tools/rag feat: wire release command, add tar.xz support, unified installers (#277) 2026-02-04 00:49:57 +00:00
.coderabbit.yaml ci(workflows): use host-uk/build@dev for releases (#264) 2026-02-04 08:54:24 +00:00
.gitattributes feat: git command, build improvements, and go fmt git-aware (#74) 2026-02-01 10:48:44 +00:00
.gitignore feat(jobrunner): add automated PR workflow system (#329) 2026-02-05 10:36:21 +00:00
AUDIT-DEPENDENCIES.md docs(audit): add dependency security audit report (#248) 2026-02-02 08:04:26 +00:00
CLAUDE.md Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
diff_dev_jules.txt Implement Background Goroutines for Long-Running Operations (#309) 2026-02-05 10:26:45 +00:00
diff_jules_dev.txt Implement Background Goroutines for Long-Running Operations (#309) 2026-02-05 10:26:45 +00:00
GEMINI.md test: increase coverage to 63.8% across packages 2026-01-29 14:28:23 +00:00
go.mod Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
go.sum feat(jobrunner): add automated PR workflow system (#329) 2026-02-05 10:36:21 +00:00
install.bat feat: wire release command, add tar.xz support, unified installers (#277) 2026-02-04 00:49:57 +00:00
install.sh feat: wire release command, add tar.xz support, unified installers (#277) 2026-02-04 00:49:57 +00:00
ISSUES_TRIAGE.md ci(workflows): use host-uk/build@dev for releases (#264) 2026-02-04 08:54:24 +00:00
LICENSE.txt Add initial project structure with core functionality and basic files 2025-10-24 04:55:10 +01:00
local.test Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
main.go refactor(cli): add Fatal() and simplify main entry point 2026-01-30 22:16:34 +00:00
Makefile removes interfaces for pkg's moved out of core. 2025-11-24 23:48:24 +00:00
mkdocs.yml Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
README.md Add streaming API to pkg/io and optimize agentic context gathering (#313) 2026-02-05 11:00:49 +00:00
Taskfile.yml Add configuration documentation to README (#304) 2026-02-05 10:56:49 +00:00

Core

codecov Go Test Coverage Code Scanning Go Version License

Core is a Web3 Framework, written in Go using Wails.io to replace Electron and the bloat of browsers that, at their core, still live in their mum's basement.

Vision

Core is an opinionated Web3 desktop application framework providing:

  1. Service-Oriented Architecture - Pluggable services with dependency injection
  2. Encrypted Workspaces - Each workspace gets its own PGP keypair, files are obfuscated
  3. Cross-Platform Storage - Abstract storage backends (local, SFTP, WebDAV) behind a Medium interface
  4. Multi-Brand Support - Same codebase powers different "hub" apps (AdminHub, ServerHub, GatewayHub, DeveloperHub, ClientHub)
  5. Built-in Crypto - PGP encryption/signing, hashing, checksums as first-class citizens

Mental model: A secure, encrypted workspace manager where each "workspace" is a cryptographically isolated environment. The framework handles windows, menus, trays, config, and i18n.

Quick Start

import core "github.com/host-uk/core"

app := core.New(
  core.WithServiceLock(),
)

Prerequisites

Development Workflow (TDD)

task test-gen    # 1. Generate test stubs
task test        # 2. Run tests (watch them fail)
# 3. Implement your feature
task test        # 4. Run tests (watch them pass)
task review      # 5. CodeRabbit review

Building & Running

# GUI (Wails)
task gui:dev      # Development with hot-reload
task gui:build    # Production build

# CLI
task cli:build    # Build to cmd/core/bin/core
task cli:run      # Build and run

Configuration

Core uses a layered configuration system where values are resolved in the following priority:

  1. Command-line flags (if applicable)
  2. Environment variables
  3. Configuration file
  4. Default values

Configuration File

The default configuration file is located at ~/.core/config.yaml.

Format

The file uses YAML format and supports nested structures.

# ~/.core/config.yaml
dev:
  editor: vim
  debug: true

log:
  level: info

Environment Variables

Layered Configuration Mapping

Any configuration value can be overridden using environment variables with the CORE_CONFIG_ prefix. After stripping the CORE_CONFIG_ prefix, the remaining variable name is converted to lowercase and underscores are replaced with dots to map to the configuration hierarchy.

Examples:

  • CORE_CONFIG_DEV_EDITOR=nano maps to dev.editor: nano
  • CORE_CONFIG_LOG_LEVEL=debug maps to log.level: debug

Common Environment Variables

Variable Description
CORE_DAEMON Set to 1 to run the application in daemon mode.
NO_COLOR If set (to any value), disables ANSI color output.
MCP_ADDR Address for the MCP TCP server (e.g., localhost:9100). If not set, MCP uses Stdio.
COOLIFY_TOKEN API token for Coolify deployments.
AGENTIC_TOKEN API token for Agentic services.
UNIFI_URL URL of the UniFi controller (e.g., https://192.168.1.1).
UNIFI_INSECURE Set to 1 or true to skip UniFi TLS verification.

All Tasks

Task Description
task test Run all Go tests
task test-gen Generate test stubs for public API
task check go mod tidy + tests + review
task review CodeRabbit review
task cov Generate coverage.txt
task cov-view Open HTML coverage report
task sync Update public API Go files

Architecture

Project Structure

.
├── core.go              # Facade re-exporting pkg/core
├── pkg/
│   ├── core/            # Service container, DI, Runtime[T]
│   ├── config/          # JSON persistence, XDG paths
│   ├── display/         # Windows, tray, menus (Wails)
│   ├── crypt/           # Hashing, checksums, PGP
│   │   └── openpgp/     # Full PGP implementation
│   ├── io/              # Medium interface + backends
│   ├── workspace/       # Encrypted workspace management
│   ├── help/            # In-app documentation
│   └── i18n/            # Internationalization
├── cmd/
│   ├── core/            # CLI application
│   └── core-gui/        # Wails GUI application
└── go.work              # Links root, cmd/core, cmd/core-gui

Service Pattern (Dual-Constructor DI)

Every service follows this pattern:

// Static DI - standalone use/testing (no core.Runtime)
func New() (*Service, error)

// Dynamic DI - for core.WithService() registration
func Register(c *core.Core) (any, error)

Services embed *core.Runtime[Options] for access to Core() and Config().

IPC/Action System

Services implement HandleIPCEvents(c *core.Core, msg core.Message) error - auto-discovered via reflection. Handles typed actions like core.ActionServiceStartup.


Wails v3 Frontend Bindings

Core uses Wails v3 to expose Go methods to a WebView2 browser runtime. Wails automatically generates TypeScript bindings for registered services.

Documentation: Wails v3 Method Bindings

How It Works

  1. Go services with exported methods are registered with Wails
  2. Run wails3 generate bindings (or wails3 dev / wails3 build)
  3. TypeScript SDK is generated in frontend/bindings/
  4. Frontend calls Go methods with full type safety, no HTTP overhead

Current Binding Architecture

// cmd/core-gui/main.go
app.RegisterService(application.NewService(coreService))  // Only Core is registered

Problem: Only Core is registered with Wails. Sub-services (crypt, workspace, display, etc.) are internal to Core's service map - their methods aren't directly exposed to JS.

Currently exposed (see cmd/core-gui/public/bindings/):

// From frontend:
import { ACTION, Config, Service } from './bindings/github.com/host-uk/core/pkg/core'

ACTION(msg)              // Broadcast IPC message
Config()                 // Get config service reference
Service("workspace")     // Get service by name (returns any)

NOT exposed: Direct calls like workspace.CreateWorkspace() or crypt.Hash().

The IPC Bridge Pattern (Chosen Architecture)

Sub-services are accessed via Core's IPC/ACTION system, not direct Wails bindings:

// Frontend calls Core.ACTION() with typed messages
import { ACTION } from './bindings/github.com/host-uk/core/pkg/core'

// Open a window
ACTION({ action: "display.open_window", name: "settings", options: { Title: "Settings", Width: 800 } })

// Switch workspace
ACTION({ action: "workspace.switch_workspace", name: "myworkspace" })

Each service implements HandleIPCEvents(c *core.Core, msg core.Message) to process these messages:

// pkg/display/display.go
func (s *Service) HandleIPCEvents(c *core.Core, msg core.Message) error {
    switch m := msg.(type) {
    case map[string]any:
        if action, ok := m["action"].(string); ok && action == "display.open_window" {
            return s.handleOpenWindowAction(m)
        }
    }
    return nil
}

Why this pattern:

  • Single Wails service (Core) = simpler binding generation
  • Services remain decoupled from Wails
  • Centralized message routing via ACTION()
  • Services can communicate internally using same pattern

Current gap: Not all service methods have IPC handlers yet. See HandleIPCEvents in each service to understand what's wired up.

Generating Bindings

cd cmd/core-gui
wails3 generate bindings    # Regenerate after Go changes

Bindings output to cmd/core-gui/public/bindings/github.com/host-uk/core/ mirroring Go package structure.


Service Interfaces (pkg/core/interfaces.go)

type Config interface {
    Get(key string, out any) error
    Set(key string, v any) error
}

type Display interface {
    OpenWindow(opts ...WindowOption) error
}

type Workspace interface {
    CreateWorkspace(identifier, password string) (string, error)
    SwitchWorkspace(name string) error
    WorkspaceFileGet(filename string) (string, error)
    WorkspaceFileSet(filename, content string) error
}

type Crypt interface {
    EncryptPGP(writer io.Writer, recipientPath, data string, ...) (string, error)
    DecryptPGP(recipientPath, message, passphrase string, ...) (string, error)
}

Current State (Prototype)

Working

Package Notes
pkg/core Service container, DI, thread-safe - solid
pkg/config JSON persistence, XDG paths - solid
pkg/crypt Hashing, checksums, PGP - solid, well-tested
pkg/help Embedded docs, Show/ShowAt - solid
pkg/i18n Multi-language with go-i18n - solid
pkg/io Medium interface + local backend - solid
pkg/workspace Workspace creation, switching, file ops - functional

Partial

Package Issues
pkg/display Window creation works; menu/tray handlers are TODOs

Priority Work Items

1. IMPLEMENT: System Tray Brand Support

pkg/display/tray.go:52-63 - Commented brand-specific menu items need implementation.

2. ADD: Integration Tests

Package Notes
pkg/display Integration tests requiring Wails runtime (27% unit coverage)

Package Deep Dives

pkg/workspace - The Core Feature

Each workspace is:

  1. Identified by LTHN hash of user identifier
  2. Has directory structure: config/, log/, data/, files/, keys/
  3. Gets a PGP keypair generated on creation
  4. Files accessed via obfuscated paths

The workspaceList maps workspace IDs to public keys.

pkg/crypt/openpgp

Full PGP using github.com/ProtonMail/go-crypto:

  • CreateKeyPair(name, passphrase) - RSA-4096 with revocation cert
  • EncryptPGP() - Encrypt + optional signing
  • DecryptPGP() - Decrypt + optional signature verification

pkg/io - Storage Abstraction

type Medium interface {
    Read(path string) (string, error)
    Write(path, content string) error
    EnsureDir(path string) error
    IsFile(path string) bool
    FileGet(path string) (string, error)
    FileSet(path, content string) error
}

Implementations: local/, sftp/, webdav/


Future Work

Phase 1: Core Stability

  • Fix workspace medium injection (critical blocker)
  • Initialize io.Local global
  • Clean up dead code (orphaned vars, broken wrappers)
  • Wire up IPC handlers for all services (config, crypt, display, help, i18n, workspace)
  • Complete display menu handlers (New/List workspace)
  • Tray icon setup with asset embedding
  • Test coverage for io packages
  • System tray brand-specific menus

Phase 2: Multi-Brand Support

  • Define brand configuration system (config? build flags?)
  • Implement brand-specific tray menus (AdminHub, ServerHub, GatewayHub, DeveloperHub, ClientHub)
  • Brand-specific theming/assets
  • Per-brand default workspace configurations

Phase 3: Remote Storage

  • Complete SFTP backend (pkg/io/sftp/)
  • Complete WebDAV backend (pkg/io/webdav/)
  • Workspace sync across storage backends
  • Conflict resolution for multi-device access

Phase 4: Enhanced Crypto

  • Key management UI (import/export, key rotation)
  • Multi-recipient encryption
  • Hardware key support (YubiKey, etc.)
  • Encrypted workspace backup/restore

Phase 5: Developer Experience

  • TypeScript types for IPC messages (codegen from Go structs)
  • Hot-reload for service registration
  • Plugin system for third-party services
  • CLI tooling for workspace management

Phase 6: Distribution

  • Auto-update mechanism
  • Platform installers (DMG, MSI, AppImage)
  • Signing and notarization
  • Crash reporting integration

For New Contributors

  1. Run task test to verify all tests pass
  2. Follow TDD: task test-gen creates stubs, implement to pass
  3. The dual-constructor pattern is intentional: New(deps) for tests, Register() for runtime
  4. See cmd/core-gui/main.go for how services wire together
  5. IPC handlers in each service's HandleIPCEvents() are the frontend bridge