go/docs/configuration.md

381 lines
7.8 KiB
Markdown
Raw Normal View History

# Configuration
Core uses `.core/` directory for project configuration.
## Directory Structure
```
.core/
├── release.yaml # Release configuration
├── build.yaml # Build configuration (optional)
├── php.yaml # PHP configuration (optional)
└── linuxkit/ # LinuxKit templates
├── server.yml
└── dev.yml
```
## release.yaml
Full release configuration reference:
```yaml
version: 1
project:
name: myapp
repository: myorg/myapp
build:
targets:
- os: linux
arch: amd64
- os: linux
arch: arm64
- os: darwin
arch: amd64
- os: darwin
arch: arm64
- os: windows
arch: amd64
publishers:
# GitHub Releases (required - others reference these artifacts)
- type: github
prerelease: false
draft: false
# npm binary wrapper
- type: npm
package: "@myorg/myapp"
access: public # or "restricted"
# Homebrew formula
- type: homebrew
tap: myorg/homebrew-tap
formula: myapp
official:
enabled: false
output: dist/homebrew
# Scoop manifest (Windows)
- type: scoop
bucket: myorg/scoop-bucket
official:
enabled: false
output: dist/scoop
# AUR (Arch Linux)
- type: aur
maintainer: "Name <email>"
# Chocolatey (Windows)
- type: chocolatey
push: false # true to publish
# Docker multi-arch
- type: docker
registry: ghcr.io
image: myorg/myapp
dockerfile: Dockerfile
platforms:
- linux/amd64
- linux/arm64
tags:
- latest
- "{{.Version}}"
build_args:
VERSION: "{{.Version}}"
# LinuxKit images
- type: linuxkit
config: .core/linuxkit/server.yml
formats:
- iso
- qcow2
- docker
platforms:
- linux/amd64
- linux/arm64
changelog:
include:
- feat
- fix
- perf
- refactor
exclude:
- chore
- docs
- style
- test
- ci
```
## build.yaml
Optional build configuration:
```yaml
version: 1
project:
name: myapp
binary: myapp
build:
main: ./cmd/myapp
env:
CGO_ENABLED: "0"
flags:
- -trimpath
ldflags:
- -s -w
- -X main.version={{.Version}}
- -X main.commit={{.Commit}}
targets:
- os: linux
arch: amd64
- os: darwin
arch: arm64
```
## php.yaml
PHP/Laravel configuration:
```yaml
version: 1
dev:
domain: myapp.test
ssl: true
port: 8000
services:
- frankenphp
- vite
- horizon
- reverb
- redis
test:
parallel: true
feat: BugSETI app, WebSocket hub, browser automation, and MCP tools (#336) * feat: add security logging and fix framework regressions This commit implements comprehensive security event logging and resolves critical regressions in the core framework. Security Logging: - Enhanced `pkg/log` with a `Security` level and helper. - Added `log.Username()` to consistently identify the executing user. - Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs. - Added `SecurityStyle` to the CLI for consistent visual representation of security events. UniFi Security (CodeQL): - Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert. - Added a `--verify-tls` flag and configuration option to control TLS verification. - Updated command handlers to support the new verification parameter. Framework Fixes: - Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge. - Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures. - These fixes resolve project-wide compilation errors caused by the signature mismatches. I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability. * feat(mcp): add RAG tools (query, ingest, collections) Add vector database tools to the MCP server for RAG operations: - rag_query: Search for relevant documentation using semantic similarity - rag_ingest: Ingest files or directories into the vector database - rag_collections: List available collections Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile) and pkg/rag for Qdrant client access. Default collection is "hostuk-docs" with topK=5 for queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add metrics tools (record, query) Add MCP tools for recording and querying AI/security metrics events. The metrics_record tool writes events to daily JSONL files, and the metrics_query tool provides aggregated statistics by type, repo, and agent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add 'core mcp serve' command Add CLI command to start the MCP server for AI tool integration. - Create internal/cmd/mcpcmd package with serve subcommand - Support --workspace flag for directory restriction - Handle SIGINT/SIGTERM for clean shutdown - Register in full.go build variant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ws): add WebSocket hub package for real-time streaming Add pkg/ws package implementing a hub pattern for WebSocket connections: - Hub manages client connections, broadcasts, and channel subscriptions - Client struct represents connected WebSocket clients - Message types: process_output, process_status, event, error, ping/pong - Channel-based subscription system (subscribe/unsubscribe) - SendProcessOutput and SendProcessStatus for process streaming integration - Full test coverage including concurrency tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add process management and WebSocket MCP tools Add MCP tools for process management: - process_start: Start a new external process - process_stop: Gracefully stop a running process - process_kill: Force kill a process - process_list: List all managed processes - process_output: Get captured process output - process_input: Send input to process stdin Add MCP tools for WebSocket: - ws_start: Start WebSocket server for real-time streaming - ws_info: Get hub statistics (clients, channels) Update Service struct with optional process.Service and ws.Hub fields, new WithProcessService and WithWSHub options, getter methods, and Shutdown method for cleanup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(webview): add browser automation package via Chrome DevTools Protocol Add pkg/webview package for browser automation: - webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate - cdp.go: Chrome DevTools Protocol WebSocket client implementation - actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder - console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher - angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability Add MCP tools for webview: - webview_connect/disconnect: Connection management - webview_navigate: Page navigation - webview_click/type/query/wait: DOM interaction - webview_console: Console output capture - webview_eval: JavaScript execution - webview_screenshot: Screenshot capture Add documentation: - docs/mcp/angular-testing.md: Guide for Angular application testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: document new packages and BugSETI application - Update CLAUDE.md with documentation for: - pkg/ws (WebSocket hub for real-time streaming) - pkg/webview (Browser automation via CDP) - pkg/mcp (MCP server tools: process, ws, webview) - BugSETI application overview - Add comprehensive README for BugSETI with: - Installation and configuration guide - Usage workflow documentation - Architecture overview - Contributing guidelines Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(bugseti): add BugSETI system tray app with auto-update BugSETI - Distributed Bug Fixing like SETI@home but for code Features: - System tray app with Wails v3 - GitHub issue fetching with label filters - Issue queue with priority management - AI context seeding via seed-agent-developer skill - Automated PR submission flow - Stats tracking and leaderboard - Cross-platform notifications - Self-updating with stable/beta/nightly channels Includes: - cmd/bugseti: Main application with Angular frontend - internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify) - internal/bugseti/updater: Auto-update system (checker, downloader, installer) - .github/workflows/bugseti-release.yml: CI/CD for all platforms Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve import cycle and code duplication - Remove pkg/log import from pkg/io/local to break import cycle (pkg/log/rotation.go imports pkg/io, creating circular dependency) - Use stderr logging for security events in sandbox escape detection - Remove unused sync/atomic import from core.go - Fix duplicate LogSecurity function declarations in cli/log.go - Update workspace/service.go Crypt() call to match interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update tests for new function signatures and format code - Update core_test.go: Config(), Display() now panic instead of returning error - Update runtime_pkg_test.go: sr.Config() now panics instead of returning error - Update MustServiceFor tests to use assert.Panics - Format BugSETI, MCP tools, and webview packages with gofmt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Snider <631881+Snider@users.noreply.github.com> Co-authored-by: Claude <developers@lethean.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 17:22:05 +00:00
coverage: true
thresholds:
statements: 40
branches: 35
deploy:
coolify:
server: https://coolify.example.com
project: my-project
environment: production
```
## LinuxKit Templates
LinuxKit YAML configuration:
```yaml
kernel:
image: linuxkit/kernel:6.6
cmdline: "console=tty0 console=ttyS0"
init:
- linuxkit/init:latest
- linuxkit/runc:latest
- linuxkit/containerd:latest
- linuxkit/ca-certificates:latest
onboot:
- name: sysctl
image: linuxkit/sysctl:latest
services:
- name: dhcpcd
image: linuxkit/dhcpcd:latest
- name: sshd
image: linuxkit/sshd:latest
- name: myapp
image: myorg/myapp:latest
capabilities:
- CAP_NET_BIND_SERVICE
files:
- path: /etc/myapp/config.yaml
contents: |
server:
port: 8080
```
## repos.yaml
Package registry for multi-repo workspaces:
```yaml
# Organisation name (used for GitHub URLs)
org: host-uk
# Base path for cloning (default: current directory)
base_path: .
# Default settings for all repos
defaults:
ci: github
license: EUPL-1.2
branch: main
# Repository definitions
repos:
# Foundation packages (no dependencies)
core-php:
type: foundation
description: Foundation framework
core-devops:
type: foundation
description: Development environment
clone: false # Skip during setup (already exists)
# Module packages (depend on foundation)
core-tenant:
type: module
depends_on: [core-php]
description: Multi-tenancy module
core-admin:
type: module
depends_on: [core-php, core-tenant]
description: Admin panel
core-api:
type: module
depends_on: [core-php]
description: REST API framework
# Product packages (user-facing applications)
core-bio:
type: product
depends_on: [core-php, core-tenant]
description: Link-in-bio product
domain: bio.host.uk.com
core-social:
type: product
depends_on: [core-php, core-tenant]
description: Social scheduling
domain: social.host.uk.com
# Templates
core-template:
type: template
description: Starter template for new projects
```
### repos.yaml Fields
| Field | Required | Description |
|-------|----------|-------------|
| `org` | Yes | GitHub organisation name |
| `base_path` | No | Directory for cloning (default: `.`) |
| `defaults` | No | Default settings applied to all repos |
| `repos` | Yes | Map of repository definitions |
### Repository Fields
| Field | Required | Description |
|-------|----------|-------------|
| `type` | Yes | `foundation`, `module`, `product`, or `template` |
| `description` | No | Human-readable description |
| `depends_on` | No | List of package dependencies |
| `clone` | No | Set `false` to skip during setup |
| `domain` | No | Production domain (for products) |
| `branch` | No | Override default branch |
### Package Types
| Type | Description | Dependencies |
|------|-------------|--------------|
| `foundation` | Core framework packages | None |
| `module` | Reusable modules | Foundation packages |
| `product` | User-facing applications | Foundation + modules |
| `template` | Starter templates | Any |
---
## Environment Variables
Complete reference of environment variables used by Core CLI.
### Authentication
| Variable | Used By | Description |
|----------|---------|-------------|
| `GITHUB_TOKEN` | `core ci`, `core dev` | GitHub API authentication |
| `ANTHROPIC_API_KEY` | `core ai`, `core dev claude` | Claude API key |
| `AGENTIC_TOKEN` | `core ai task*` | Agentic API authentication |
| `AGENTIC_BASE_URL` | `core ai task*` | Agentic API endpoint |
### Publishing
| Variable | Used By | Description |
|----------|---------|-------------|
| `NPM_TOKEN` | `core ci` (npm publisher) | npm registry auth token |
| `CHOCOLATEY_API_KEY` | `core ci` (chocolatey publisher) | Chocolatey API key |
| `DOCKER_USERNAME` | `core ci` (docker publisher) | Docker registry username |
| `DOCKER_PASSWORD` | `core ci` (docker publisher) | Docker registry password |
### Deployment
| Variable | Used By | Description |
|----------|---------|-------------|
| `COOLIFY_URL` | `core php deploy` | Coolify server URL |
| `COOLIFY_TOKEN` | `core php deploy` | Coolify API token |
| `COOLIFY_APP_ID` | `core php deploy` | Production application ID |
| `COOLIFY_STAGING_APP_ID` | `core php deploy --staging` | Staging application ID |
### Build
| Variable | Used By | Description |
|----------|---------|-------------|
| `CGO_ENABLED` | `core build`, `core go *` | Enable/disable CGO (default: 0) |
| `GOOS` | `core build` | Target operating system |
| `GOARCH` | `core build` | Target architecture |
### Configuration Paths
| Variable | Description |
|----------|-------------|
| `CORE_CONFIG` | Override config directory (default: `~/.core/`) |
| `CORE_REGISTRY` | Override repos.yaml path |
---
## Defaults
If no configuration exists, sensible defaults are used:
- **Targets**: linux/amd64, linux/arm64, darwin/amd64, darwin/arm64, windows/amd64
- **Publishers**: GitHub only
- **Changelog**: feat, fix, perf, refactor included
Implement log retention policy (#306) * Implement log retention policy - Added Append method to io.Medium interface and implementations. - Defined RotationOptions and updated log.Options to support log rotation. - Implemented RotatingWriter in pkg/log/rotation.go with size and age-based retention. - Updated Logger to use RotatingWriter when configured. - Added comprehensive tests for log rotation and retention. - Documented the log retention policy in docs/pkg/log.md and docs/configuration.md. - Fixed MockMedium to return current time for Stat to avoid premature cleanup in tests. * Fix formatting issues in pkg/io/local/client.go The CI failed due to formatting issues. This commit fixes them and ensures all modified files are properly formatted. * Fix auto-merge workflow CI failure Inlined the auto-merge logic and added actions/checkout and --repo flag to gh command to provide the necessary git context. This resolves the 'fatal: not a git repository' error in CI. * Address feedback on log retention policy - Made cleanup synchronous in RotatingWriter for better reliability. - Improved rotation error handling with recovery logic. - Fixed size tracking to only increment on successful writes. - Updated MockMedium to support and preserve ModTimes for age-based testing. - Added TestRotatingWriter_AgeRetention and TestLogger_RotationIntegration. - Implemented negative MaxAge to disable age-based retention. - Updated documentation for clarity on Output priority and MaxAge behavior. - Fixed typo in test comments. - Fixed CI failure in auto-merge workflow. --------- Co-authored-by: Claude <developers@lethean.io>
2026-02-05 10:26:32 +00:00
## Logging
Logging can be configured to rotate and retain logs automatically.
Default retention policy:
- **Max Size**: 100 MB
- **Max Backups**: 5
- **Max Age**: 28 days
Example configuration:
```yaml
level: info
rotation:
filename: "app.log"
max_size: 100 # megabytes
max_backups: 5 # number of old log files to retain
max_age: 28 # days to keep old log files
```