From 36f790ed227cd4346aa2a2a42a518d4fff735a2a Mon Sep 17 00:00:00 2001 From: Clotho Date: Mon, 16 Feb 2026 17:16:48 +0000 Subject: [PATCH] docs: add comprehensive CLI documentation for issue #4 Added comprehensive documentation for core/cli including: 1. Complete CLI Reference (cli-reference.md) - All 30 commands with detailed descriptions - Subcommands, flags, and usage examples - Organized by category (Development, AI/ML, DevOps, etc.) 2. Build Variants System (build-variants.md) - How to create custom builds (minimal, dev, devops, ai/ml) - Three methods: import commenting, multiple main.go, build tags - External variant repos (core/php, core/ci) - Size optimization and compression tips 3. MCP Integration (mcp-integration.md) - Model Context Protocol server setup - All available tools (file ops, RAG, metrics, process mgmt, WebView/CDP) - Transport modes (stdio, TCP) - Security considerations and workspace restriction - Use cases and troubleshooting 4. Environment Variables (environment-variables.md) - All supported environment variables - Configuration file locations (user, project, registry) - CORE_CONFIG_* mapping system - Service-specific vars (MCP, RAG, ML, Git, etc.) - Best practices and troubleshooting 5. Updated Navigation - Enhanced cmd/index.md with categories and quick links - Updated mkdocs.yml with new documentation pages Fixes #4 Co-Authored-By: Claude Sonnet 4.5 --- docs/build-variants.md | 367 +++++++++++ docs/cli-reference.md | 1073 +++++++++++++++++++++++++++++++++ docs/cmd/index.md | 107 +++- docs/environment-variables.md | 735 ++++++++++++++++++++++ docs/mcp-integration.md | 658 ++++++++++++++++++++ mkdocs.yml | 31 +- 6 files changed, 2946 insertions(+), 25 deletions(-) create mode 100644 docs/build-variants.md create mode 100644 docs/cli-reference.md create mode 100644 docs/environment-variables.md create mode 100644 docs/mcp-integration.md diff --git a/docs/build-variants.md b/docs/build-variants.md new file mode 100644 index 00000000..264fffa9 --- /dev/null +++ b/docs/build-variants.md @@ -0,0 +1,367 @@ +# Build Variants + +The Core CLI supports a modular build system that allows you to create different binary variants with different feature sets. This is useful for creating minimal builds, specialized builds, or builds tailored to specific use cases. + +## Overview + +The variant system works through Go's module system and selective import statements in `main.go`. By commenting out or including specific import statements, you can control which commands are included in the final binary. + +## Available Variants + +### Full Build (Default) + +The default build includes all commands and features. This is the standard build used for development and general-purpose use. + +**Size:** ~50-100 MB (depending on platform) + +**Included:** All commands documented in [CLI Reference](cli-reference.md) + +### Minimal Build + +A minimal build includes only essential commands, suitable for resource-constrained environments or when you only need basic functionality. + +**Recommended commands for minimal build:** +- `config` - Configuration management +- `help` - Help system +- `doctor` - Environment checks +- `test` - Basic testing + +### Specialized Builds + +You can create specialized builds for specific use cases: + +#### Developer Build +Focus on development workflow commands: +- `dev` - Development workflow +- `git` - Git operations +- `go` - Go development +- `test` - Testing +- `qa` - Quality assurance +- `docs` - Documentation + +#### DevOps Build +Focus on infrastructure and deployment: +- `deploy` - Deployment management +- `prod` - Production infrastructure +- `vm` - Virtual machine management +- `unifi` - Network management +- `monitor` - Security monitoring + +#### AI/ML Build +Focus on AI and machine learning operations: +- `ai` - AI task management +- `ml` - ML pipeline +- `rag` - RAG system +- `mcp` - MCP server +- `collect` - Data collection + +## How to Create a Variant + +### Method 1: Comment Out Imports in main.go + +Edit `main.go` and comment out the commands you don't want: + +```go +package main + +import ( + "forge.lthn.ai/core/go/pkg/cli" + + // Essential commands + _ "forge.lthn.ai/core/cli/cmd/config" + _ "forge.lthn.ai/core/cli/cmd/help" + _ "forge.lthn.ai/core/cli/cmd/doctor" + + // Development commands + _ "forge.lthn.ai/core/cli/cmd/dev" + _ "forge.lthn.ai/core/cli/cmd/go" + _ "forge.lthn.ai/core/cli/cmd/test" + + // Comment out to exclude: + // _ "forge.lthn.ai/core/cli/cmd/ai" + // _ "forge.lthn.ai/core/cli/cmd/ml" + // _ "forge.lthn.ai/core/cli/cmd/deploy" + // etc. +) + +func main() { + cli.Main() +} +``` + +Then build: + +```bash +task cli:build +``` + +### Method 2: Create Multiple main.go Files + +Create separate entry points for different variants: + +```bash +# Directory structure +cmd/ + core/ # Full build + main.go + core-minimal/ # Minimal build + main.go + core-dev/ # Developer build + main.go + core-devops/ # DevOps build + main.go +``` + +Each `main.go` includes only the relevant imports for that variant. + +Build specific variant: + +```bash +cd cmd/core-minimal +go build -o ../../bin/core-minimal . +``` + +### Method 3: Build Tags + +Use Go build tags to conditionally include commands: + +```go +//go:build !minimal +// +build !minimal + +package ai + +// Command registration... +``` + +Build with tags: + +```bash +# Full build +go build -o bin/core . + +# Minimal build +go build -tags minimal -o bin/core-minimal . +``` + +## External Variant Repositories + +The Core CLI architecture supports external command repositories that can be optionally included. These are commented out in `main.go` by default: + +```go +// Variant repos (optional — comment out to exclude) +// _ "forge.lthn.ai/core/php" +// _ "forge.lthn.ai/core/ci" +``` + +### Currently Available External Repos + +#### core/php (Archived) +PHP and Laravel development commands. **Note:** This has been moved to its own repository and is no longer included by default. + +#### core/ci (Archived) +CI/CD pipeline commands. **Note:** This has been moved to its own repository and is no longer included by default. + +To include these in your build: + +1. Add the module to your `go.mod`: + ```bash + go get forge.lthn.ai/core/php@latest + ``` + +2. Uncomment the import in `main.go`: + ```go + _ "forge.lthn.ai/core/php" + ``` + +3. Rebuild: + ```bash + task cli:build + ``` + +## Build Optimization + +### Size Optimization + +For smaller binaries, use the release build with stripped symbols: + +```bash +task cli:build:release +``` + +This uses the following ldflags: +``` +-s -w # Strip debug info and symbol table +``` + +### Compression + +Further reduce binary size with UPX: + +```bash +# Install UPX +# macOS: brew install upx +# Linux: apt-get install upx-ucl + +# Compress binary +upx --best --lzma bin/core +``` + +**Note:** Compressed binaries may trigger antivirus false positives and won't work with code signing on macOS. + +## Version Information + +All builds include embedded version information via ldflags: + +```bash +# View version +core --version + +# Build with custom version +go build -ldflags "-X forge.lthn.ai/core/go/pkg/cli.AppVersion=1.2.3" . +``` + +The Taskfile automatically sets version info from git tags: + +```yaml +LDFLAGS_BASE: >- + -X {{.PKG}}.AppVersion={{.SEMVER_VERSION}} + -X {{.PKG}}.BuildCommit={{.SEMVER_COMMIT}} + -X {{.PKG}}.BuildDate={{.SEMVER_DATE}} + -X {{.PKG}}.BuildPreRelease={{.SEMVER_PRERELEASE}} +``` + +## Examples + +### Create a Minimal Developer Build + +1. Edit `main.go`: +```go +package main + +import ( + "forge.lthn.ai/core/go/pkg/cli" + + _ "forge.lthn.ai/core/cli/cmd/config" + _ "forge.lthn.ai/core/cli/cmd/dev" + _ "forge.lthn.ai/core/cli/cmd/doctor" + _ "forge.lthn.ai/core/cli/cmd/git" + _ "forge.lthn.ai/core/cli/cmd/go" + _ "forge.lthn.ai/core/cli/cmd/help" + _ "forge.lthn.ai/core/cli/cmd/test" +) + +func main() { + cli.Main() +} +``` + +2. Build: +```bash +task cli:build:release +``` + +3. Result: ~20-30 MB binary with only development commands + +### Create an AI/ML Specialist Build + +1. Edit `main.go`: +```go +package main + +import ( + "forge.lthn.ai/core/go/pkg/cli" + + _ "forge.lthn.ai/core/cli/cmd/ai" + _ "forge.lthn.ai/core/cli/cmd/collect" + _ "forge.lthn.ai/core/cli/cmd/config" + _ "forge.lthn.ai/core/cli/cmd/daemon" + _ "forge.lthn.ai/core/cli/cmd/help" + _ "forge.lthn.ai/core/cli/cmd/mcp" + _ "forge.lthn.ai/core/cli/cmd/ml" + _ "forge.lthn.ai/core/cli/cmd/rag" + _ "forge.lthn.ai/core/cli/cmd/session" +) + +func main() { + cli.Main() +} +``` + +2. Build: +```bash +task cli:build:release +``` + +3. Result: Binary optimized for AI/ML workflows + +## Testing Variants + +Test your variant to ensure all needed commands are present: + +```bash +# Build variant +task cli:build + +# Test available commands +./bin/core help + +# Verify specific command +./bin/core dev --help +``` + +## Distribution + +When distributing variant builds, use clear naming: + +```bash +# Good naming examples +core-full-v1.2.3-linux-amd64 +core-minimal-v1.2.3-darwin-arm64 +core-dev-v1.2.3-windows-amd64.exe +``` + +Include a README describing what's included in each variant. + +## Best Practices + +1. **Document your variant** - Keep a record of which commands are included +2. **Test thoroughly** - Ensure all needed functionality is present +3. **Version consistently** - Use the same version numbering scheme +4. **Consider dependencies** - Some commands depend on others (e.g., `dev` uses `git`) +5. **Keep it simple** - Don't create too many variants unless necessary +6. **Use release builds** - Always use `-s -w` ldflags for distribution + +## Troubleshooting + +### Command Not Found + +If a command is missing after building a variant: + +1. Check that the import is uncommented in `main.go` +2. Verify the module is in `go.mod` +3. Run `go mod tidy` +4. Rebuild with `task cli:build` + +### Build Errors + +If you get import errors: + +1. Ensure all required modules are in `go.mod` +2. Run `go mod download` +3. Check for version conflicts with `go mod why` + +### Large Binary Size + +If your minimal build is still too large: + +1. Use release build flags: `task cli:build:release` +2. Remove unused commands from imports +3. Consider using UPX compression (with caveats noted above) +4. Check for large embedded assets + +## See Also + +- [CLI Reference](cli-reference.md) - Complete command documentation +- [Configuration](configuration.md) - Configuration system +- [Getting Started](getting-started.md) - Installation and setup diff --git a/docs/cli-reference.md b/docs/cli-reference.md new file mode 100644 index 00000000..25a83219 --- /dev/null +++ b/docs/cli-reference.md @@ -0,0 +1,1073 @@ +# CLI Reference + +This document provides a comprehensive reference for all available commands in the Core CLI. + +## Overview + +The Core CLI is a unified development tool that provides commands for: +- AI/ML operations and agent management +- Multi-repository development workflows +- Infrastructure and deployment management +- Security monitoring and analysis +- Data collection and RAG systems +- Cryptographic utilities +- Testing and quality assurance + +## Quick Command Index + +| Command | Category | Description | +|---------|----------|-------------| +| [ai](#ai) | AI/ML | Agentic task management and agent configuration | +| [collect](#collect) | Data | Collect data from external sources | +| [config](#config) | Config | Configuration management | +| [crypt](#crypt) | Security | Cryptographic utilities | +| [daemon](#daemon) | Service | Background service management | +| [deploy](#deploy) | DevOps | Coolify deployment management | +| [dev](#dev) | Development | Multi-repo development workflow | +| [docs](#docs) | Documentation | Documentation management | +| [doctor](#doctor) | Utilities | Environment verification | +| [forge](#forge) | Git | Forgejo instance management | +| [git](#git) | Git | Root-level git workflow commands | +| [gitea](#gitea) | Git | Gitea instance management | +| [go](#go) | Development | Go development commands | +| [help](#help) | Utilities | Help documentation | +| [lab](#lab) | Monitoring | Homelab monitoring dashboard | +| [mcp](#mcp) | AI/ML | Model Context Protocol server | +| [ml](#ml) | AI/ML | ML inference and training pipeline | +| [monitor](#monitor) | Security | Security monitoring | +| [plugin](#plugin) | Utilities | Plugin management | +| [prod](#prod) | DevOps | Production infrastructure management | +| [qa](#qa) | Development | Quality assurance workflow | +| [rag](#rag) | AI/ML | RAG system for embeddings and semantic search | +| [security](#security) | Security | Security management | +| [session](#session) | Utilities | Session recording and replay | +| [setup](#setup) | Utilities | Workspace setup and bootstrap | +| [test](#test) | Development | Test running | +| [unifi](#unifi) | Networking | UniFi network management | +| [update](#update) | Utilities | CLI self-update | +| [vm](#vm) | DevOps | LinuxKit VM management | +| [workspace](#workspace) | Config | Workspace configuration | + +## Detailed Command Reference + +### ai + +**Agentic task management and agent configuration** + +Manage AI agents, tasks, and integrations with external services. + +**Subcommands:** + +- `agent` - Manage AgentCI dispatch targets + - `add` - Add a new dispatch target + - `list` - List all configured targets + - `status` - Show target status + - `logs` - View target logs + - `setup` - Initial target setup + - `remove` - Remove a target +- `tasks` - Agentic task management + - `list` - List all tasks + - `view` - View task details + - `update` - Update task status + - `complete` - Mark task as complete +- `git` - Git integration for tasks + - `commit` - Create commits for tasks + - `PR` - Create pull requests +- `metrics` - Task metrics tracking +- `commands` - AI command management +- `updates` - Task updates +- `ratelimits` - Rate limiting configuration +- `dispatch` - Task dispatch management + +**Examples:** + +```bash +# List all AI tasks +core ai tasks list + +# View task details +core ai tasks view TASK-123 + +# Add dispatch target +core ai agent add --name my-agent --url https://api.example.com +``` + +--- + +### collect + +**Collect data from various sources** + +Data collection tools for gathering information from external sources like GitHub, forums, and research papers. + +**Persistent Flags:** + +- `--output`, `-o` - Output directory +- `--verbose`, `-v` - Verbose output +- `--dry-run` - Show what would happen without executing + +**Subcommands:** + +- `github` - Collect data from GitHub repositories +- `bitcointalk` - Collect from BitcoinTalk forum +- `market` - Collect market data +- `papers` - Collect research papers +- `excavate` - Deep excavation of sources +- `process` - Process collected data +- `dispatch` - Dispatch collection jobs + +**Examples:** + +```bash +# Collect GitHub data +core collect github --org myorg --output ./data + +# Collect research papers +core collect papers --query "machine learning" --output ./papers +``` + +--- + +### config + +**Manage configuration** + +Manage Core CLI configuration values stored in `~/.core/config.yaml`. + +**Subcommands:** + +- `get` - Get a config value +- `set` - Set a config value +- `list` - List all config values +- `path` - Show config file path + +**Examples:** + +```bash +# Get a config value +core config get dev.editor + +# Set a config value +core config set dev.editor vim + +# List all config +core config list + +# Show config file location +core config path +``` + +**See also:** [Configuration Reference](configuration.md) + +--- + +### crypt + +**Cryptographic utilities** + +Encrypt, decrypt, hash, and checksum files and data. + +**Subcommands:** + +- `hash` - Hash files or data +- `encrypt` - Encrypt files or data +- `keygen` - Generate cryptographic keys +- `checksum` - Generate and verify checksums + +**Examples:** + +```bash +# Hash a file +core crypt hash --file myfile.txt + +# Generate a PGP key +core crypt keygen --name "John Doe" --email john@example.com + +# Encrypt a file +core crypt encrypt --file secret.txt --recipient john@example.com + +# Generate checksums +core crypt checksum --file release.tar.gz +``` + +--- + +### daemon + +**Start the core daemon** + +Start Core daemon for long-running services like MCP. + +**Flags:** + +- `--mcp-transport`, `-t` - MCP transport (stdio, tcp, socket) +- `--mcp-addr`, `-a` - Listen address for TCP/socket +- `--health-addr` - Health check endpoint address +- `--pid-file` - PID file path + +**Environment Variables:** + +- `CORE_MCP_TRANSPORT` - MCP transport type +- `CORE_MCP_ADDR` - MCP listen address +- `CORE_HEALTH_ADDR` - Health check address +- `CORE_PID_FILE` - PID file location + +**Examples:** + +```bash +# Start daemon with MCP on stdio +core daemon + +# Start daemon with TCP MCP server +core daemon --mcp-transport tcp --mcp-addr :9100 + +# Start with health endpoint +core daemon --health-addr :8080 +``` + +--- + +### deploy + +**Deployment infrastructure management** + +Manage Coolify deployments and infrastructure. + +**Flags:** + +- `--url` - Coolify instance URL +- `--token` - API token +- `--json` - JSON output format + +**Subcommands:** + +- `servers` - List Coolify servers +- `projects` - List projects +- `apps` - List applications +- `databases` / `dbs` / `db` - List databases +- `services` - List services +- `team` - Show team info +- `call` - Call arbitrary Coolify API operations + +**Examples:** + +```bash +# List all servers +core deploy servers + +# List applications +core deploy apps + +# Call custom API endpoint +core deploy call --method GET --path /api/v1/status +``` + +--- + +### dev + +**Development workflow commands** + +Multi-repository development workflows with health checks, commits, and CI monitoring. + +**Subcommands:** + +- `work` - Combined status, commit, and push workflow +- `health` - Quick health check across repos +- `commit` - Claude-assisted commit message generation +- `push` - Push repos with unpushed commits +- `pull` - Pull repos behind remote +- `issues` - List open issues +- `reviews` - List PRs needing review +- `ci` - Check GitHub Actions CI status +- `impact` - Analyze dependency impact +- `workflow` - CI/workflow management +- `api` - API synchronization +- `vm` - VM management + - `install`, `boot`, `stop`, `status`, `shell`, `serve`, `test`, `claude`, `update` +- `file-sync` - Sync files across repos (safe for AI agents) +- `apply` - Run commands across repos (safe for AI agents) + +**Examples:** + +```bash +# Full workflow: status, commit, push +core dev work + +# Health check all repos +core dev health + +# Claude-assisted commit +core dev commit + +# List open issues +core dev issues + +# Check CI status +core dev ci +``` + +--- + +### docs + +**Documentation management** + +Manage project documentation. + +**Subcommands:** + +- `sync` - Sync documentation +- `list` - List documentation +- `commands` - Command documentation +- `scan` - Scan for documentation + +**Examples:** + +```bash +# Sync documentation +core docs sync + +# List available docs +core docs list +``` + +--- + +### doctor + +**Check development environment** + +Verify required tools and dependencies are installed and configured. + +**Flags:** + +- `--verbose` - Show detailed check results + +**Checks:** + +- Required tools validation (Go, Git, Task, etc.) +- Optional tools validation +- GitHub SSH access +- GitHub CLI authentication +- Workspace validation + +**Examples:** + +```bash +# Quick environment check +core doctor + +# Verbose check with details +core doctor --verbose +``` + +--- + +### forge + +**Forgejo instance management** + +Manage Forgejo repositories, issues, and pull requests. + +**Subcommands:** + +- `config` - Configure Forgejo connection +- `status` - Show instance status and version +- `repos` - List repositories +- `issues` - List and create issues +- `prs` - List pull requests +- `migrate` - Migrate repos from external services +- `sync` - Sync GitHub repos to upstream branches +- `orgs` - List organizations +- `labels` - List and create labels + +**Examples:** + +```bash +# Configure Forgejo +core forge config --url https://forge.example.com --token YOUR_TOKEN + +# List repositories +core forge repos + +# List issues +core forge issues + +# Create an issue +core forge issues create --title "Bug fix" --body "Description" +``` + +--- + +### git + +**Root-level git workflow commands** + +Git operations wrapper (imports commands from dev package). + +**Subcommands:** + +- `health` - Show repo status +- `commit` - Claude-assisted commits +- `push` - Push repositories +- `pull` - Pull repositories +- `work` - Combined workflow +- `file-sync` - Safe file synchronization +- `apply` - Safe command execution + +**Examples:** + +```bash +# Check repo health +core git health + +# Claude-assisted commit +core git commit +``` + +--- + +### gitea + +**Gitea instance management** + +Manage Gitea repositories and integrations. + +**Subcommands:** + +- `config` - Configure Gitea connection +- `repos` - List repositories +- `issues` - List and create issues +- `prs` - List pull requests +- `mirror` - Create GitHub-to-Gitea mirrors +- `sync` - Sync GitHub repos to upstream branches + +**Examples:** + +```bash +# Configure Gitea +core gitea config --url https://gitea.example.com + +# Create GitHub mirror +core gitea mirror --repo owner/repo +``` + +--- + +### go + +**Go development utilities** + +Go-specific development tools for testing, coverage, and quality assurance. + +**Subcommands:** + +- `qa` - Quality assurance checks +- `test` - Run tests +- `cov` - Code coverage analysis +- `fmt` - Code formatting +- `lint` - Linting +- `install` - Install Go tools +- `mod` - Module management +- `work` - Workspace management +- `fuzz` - Fuzz testing + +**Examples:** + +```bash +# Run QA checks +core go qa + +# Run tests with coverage +core go cov + +# Format code +core go fmt + +# Run specific test +core go test --run TestMyFunction +``` + +--- + +### help + +**Display help documentation** + +Access built-in help documentation. + +**Flags:** + +- `--search`, `-s` - Search help topics + +**Examples:** + +```bash +# Show help +core help + +# Search help topics +core help --search "configuration" +``` + +--- + +### lab + +**Homelab monitoring dashboard** + +Real-time monitoring of machines, training runs, models, and services. + +**Subcommands:** + +- `serve` - Start lab dashboard web server + - `--bind` - HTTP listen address + +**Examples:** + +```bash +# Start dashboard on default port +core lab serve + +# Start on custom port +core lab serve --bind :8080 +``` + +--- + +### mcp + +**Model Context Protocol server** + +MCP server providing file operations, RAG, and metrics tools for AI assistants. + +**Subcommands:** + +- `serve` - Start the MCP server + - `--workspace`, `-w` - Restrict file operations to directory + +**Environment Variables:** + +- `MCP_ADDR` - TCP address (e.g., `localhost:9999`). If not set, uses stdio. + +**Transport:** + +- **stdio** (default) - For Claude Code integration +- **TCP** - Set via `MCP_ADDR` environment variable + +**Examples:** + +```bash +# Start with stdio (for Claude Code) +core mcp serve + +# Start with workspace restriction +core mcp serve --workspace /path/to/project + +# Start TCP server +MCP_ADDR=localhost:9999 core mcp serve +``` + +**See also:** [MCP Integration](mcp-integration.md) + +--- + +### ml + +**ML inference, scoring, and training pipeline** + +Commands for model scoring, probe evaluation, data export, and format conversion. + +**Persistent Flags:** + +- `--api-url` - OpenAI-compatible API URL +- `--judge-url` - Judge model API URL (Ollama) +- `--judge-model` - Judge model name +- `--influx` - InfluxDB URL +- `--influx-db` - InfluxDB database +- `--db` - DuckDB database path +- `--model` - Model name for API + +**Subcommands:** + +- `score` - Score responses with heuristic and LLM judges +- `probe` - Run capability and content probes against a model +- `export` - Export golden set to training formats +- `expand` - Generate expansion responses +- `status` - Show training and generation progress +- `gguf` - Convert MLX LoRA adapter to GGUF format +- `convert` - Convert MLX LoRA adapter to PEFT format +- `agent` - Run the scoring agent daemon +- `worker` - Run a distributed worker node +- `serve` - Start OpenAI-compatible inference server +- `inventory` - Show DuckDB table inventory +- `query` - Run ad-hoc SQL against DuckDB +- `metrics` - Push golden set stats to InfluxDB +- `ingest` - Ingest benchmark scores to InfluxDB +- `normalize` - Deduplicate seeds into expansion prompts +- `seed-influx` - Migrate golden set from DuckDB to InfluxDB +- `consolidate` - Pull and merge response JSONL files +- `import-all` - Import all LEM data into DuckDB +- `approve` - Filter scored expansions into training JSONL +- `publish` - Upload Parquet dataset to HuggingFace Hub +- `coverage` - Analyze seed coverage by region and domain + +**Examples:** + +```bash +# Score model responses +core ml score --model gpt-4 + +# Run capability probes +core ml probe --api-url http://localhost:8000 + +# Export training data +core ml export --format jsonl +``` + +--- + +### monitor + +**Security finding aggregation** + +Aggregate GitHub code scanning, Dependabot, and secret scanning alerts. + +**Flags:** + +- `--repo` - Specific repository +- `--all` - All repositories +- `--severity` - Filter by severity +- `--json` - JSON output + +**Examples:** + +```bash +# Monitor all repos +core monitor --all + +# Monitor specific repo with high severity +core monitor --repo owner/repo --severity high +``` + +--- + +### plugin + +**Manage plugins** + +Install and manage CLI plugins. + +**Subcommands:** + +- `install` - Install a plugin from GitHub +- `list` - List installed plugins +- `info` - Show detailed plugin information +- `update` - Update a plugin or all plugins +- `remove` - Remove an installed plugin + +**Examples:** + +```bash +# Install plugin +core plugin install owner/repo + +# List installed +core plugin list + +# Update all +core plugin update +``` + +--- + +### prod + +**Production infrastructure management** + +Manage Host UK production infrastructure. + +**Flags:** + +- `--config` - Path to infra.yaml + +**Subcommands:** + +- `status` - Show infrastructure health +- `setup` - Phase 1: discover topology, create LB, configure DNS +- `dns` - Manage DNS records via CloudNS +- `lb` - Manage Hetzner load balancer +- `ssh` - SSH into a production host + +**Examples:** + +```bash +# Check infrastructure status +core prod status + +# Setup infrastructure +core prod setup + +# SSH to host +core prod ssh web1 +``` + +--- + +### qa + +**Quality assurance workflow** + +Verify work (CI status, reviews, issues) - complements 'dev' which is about doing work. + +**Subcommands:** + +- `watch` - Monitor GitHub Actions after push +- `review` - PR review status with actionable steps +- `health` - Aggregate CI health across repos +- `issues` - Intelligent issue triage +- `docblock` - Documentation block validation + +**Examples:** + +```bash +# Watch CI after push +core qa watch + +# Check PR review status +core qa review + +# Health check across repos +core qa health +``` + +--- + +### rag + +**RAG system for embeddings and semantic search** + +Retrieval-Augmented Generation system using Qdrant and Ollama. + +**Persistent Flags:** + +- `--qdrant-host` - Qdrant host (default: localhost) +- `--qdrant-port` - Qdrant port (default: 6334) +- `--ollama-host` - Ollama host (default: localhost) +- `--ollama-port` - Ollama port (default: 11434) +- `--model` - Embedding model (default: nomic-embed-text) +- `--verbose`, `-v` - Verbose output + +**Subcommands:** + +- `ingest` - Ingest documents into collections +- `query` - Query collections +- `collections` - Manage collections (list, stats, delete) + +**Examples:** + +```bash +# Ingest documents +core rag ingest --collection docs --path ./documentation + +# Query collection +core rag query --collection docs --query "how to install" + +# List collections +core rag collections list +``` + +--- + +### security + +**Security management** + +Security operations and scanning. + +**Subcommands:** + +- `jobs` - Security job management +- `secrets` - Secrets management +- `alerts` - Security alerts +- `scan` - Security scanning + +**Examples:** + +```bash +# List security alerts +core security alerts + +# Run security scan +core security scan +``` + +--- + +### session + +**Session recording and replay** + +Record and replay CLI sessions with timeline generation. + +**Subcommands:** + +- `list` - List recent sessions +- `replay` - Generate HTML timeline (and optional MP4) from session + - `--mp4` - Generate MP4 video + - `--output` - Output directory +- `search` - Search across session transcripts + +**Examples:** + +```bash +# List sessions +core session list + +# Replay session as HTML +core session replay SESSION_ID + +# Replay with MP4 +core session replay SESSION_ID --mp4 +``` + +--- + +### setup + +**Workspace setup and bootstrap** + +Set up development workspace from registry. + +**Flags:** + +- `--registry` - Registry path +- `--only` - Specific setup targets +- `--dry-run` - Show what would happen +- `--all` - Setup all targets +- `--name` - Target name +- `--build` - Build after setup + +**Subcommands:** + +- `wizard` - Interactive setup wizard +- `github` - GitHub setup +- `registry` - Registry setup +- `ci` - CI setup +- `repo` - Repository setup +- `bootstrap` - Bootstrap new workspace + +**Examples:** + +```bash +# Interactive setup +core setup wizard + +# Setup from registry +core setup --registry repos.yaml --all + +# Bootstrap workspace +core setup bootstrap +``` + +--- + +### test + +**Run tests** + +Execute Go tests with various options. + +**Flags:** + +- `--verbose` - Verbose output +- `--coverage` - Generate coverage report +- `--short` - Run short tests only +- `--pkg` - Specific package +- `--run` - Test filter pattern +- `--race` - Enable race detector +- `--json` - JSON output + +**Examples:** + +```bash +# Run all tests +core test + +# Run with coverage +core test --coverage + +# Run specific test +core test --run TestMyFunction + +# Race detection +core test --race +``` + +--- + +### unifi + +**UniFi network management** + +Manage UniFi network devices and configuration. + +**Subcommands:** + +- `config` - Configure UniFi connection +- `clients` - List connected clients +- `devices` - List infrastructure devices +- `sites` - List controller sites +- `networks` - List network segments and VLANs +- `routes` - List gateway routing table + +**Examples:** + +```bash +# Configure UniFi +core unifi config --url https://192.168.1.1 + +# List clients +core unifi clients + +# List devices +core unifi devices +``` + +--- + +### update + +**Update core CLI to latest version** + +Self-update the Core CLI binary. + +**Flags:** + +- `--channel` - Release channel (stable, beta, alpha, dev) +- `--force` - Force update even if on latest +- `--check` - Check without applying + +**Subcommands:** + +- `check` - Check for available updates + +**Channels:** + +- `stable` - Stable releases +- `beta` - Beta releases +- `alpha` - Alpha releases +- `dev` - Rolling development releases + +**Examples:** + +```bash +# Update to latest stable +core update + +# Check for updates +core update check + +# Update to beta channel +core update --channel beta +``` + +--- + +### vm + +**Virtual machine management** + +Manage LinuxKit VMs for testing and development. + +**Subcommands:** + +- `run` - Run a VM +- `ps` - List running VMs +- `stop` - Stop a VM +- `logs` - View VM logs +- `exec` - Execute command in VM +- `templates` - Manage VM templates + +**Examples:** + +```bash +# Run VM +core vm run my-vm + +# List running VMs +core vm ps + +# Execute command +core vm exec my-vm -- ls -la +``` + +--- + +### workspace + +**Manage workspace configuration** + +Configure workspace-specific settings. + +**Subcommands:** + +- `active` - Show or set active package +- `task` - Workspace task management +- `agent` - Workspace agent configuration + +**Examples:** + +```bash +# Show active workspace +core workspace active + +# Set active package +core workspace active --set ./pkg/mypackage +``` + +--- + +## Common Patterns + +### Flags and Options + +**Common Persistent Flags:** + +- `--verbose`, `-v` - Verbose output +- `--dry-run` - Show what would happen without executing +- `--output`, `-o` - Output file or directory +- `--json` - JSON output format +- `--help`, `-h` - Show help for command + +### Environment Variables + +Many commands respect environment variables. See [Environment Variables](environment-variables.md) for a complete list. + +### Configuration Files + +Commands load configuration from: + +1. Command-line flags (highest priority) +2. Environment variables +3. Configuration file (`~/.core/config.yaml`) +4. Default values (lowest priority) + +See [Configuration Reference](configuration.md) for details. + +## Getting Help + +For command-specific help: + +```bash +core --help +core --help +``` + +For general help: + +```bash +core help +core --help +``` diff --git a/docs/cmd/index.md b/docs/cmd/index.md index de2c061f..1f5cdf53 100644 --- a/docs/cmd/index.md +++ b/docs/cmd/index.md @@ -1,24 +1,66 @@ # Core CLI -Unified interface for Go/PHP development, multi-repo management, and deployment. +Unified interface for development, multi-repo management, deployment, AI/ML operations, and infrastructure management. -## Commands +## Quick Links +- **[Complete CLI Reference](../cli-reference.md)** - Comprehensive documentation of all commands +- **[Build Variants](../build-variants.md)** - Create custom builds with different feature sets +- **[MCP Integration](../mcp-integration.md)** - Model Context Protocol server for AI assistants +- **[Environment Variables](../environment-variables.md)** - Configuration via environment + +## Command Categories + +### Development & Testing +| Command | Description | +|---------|-------------| +| [dev](dev/) | Multi-repo workflow + dev environment | +| [go](go/) | Go development tools | +| [test](test/) | Run Go tests with coverage | +| [qa](../cli-reference.md#qa) | Quality assurance workflow | +| [doctor](doctor/) | Check environment | + +### AI & Machine Learning | Command | Description | |---------|-------------| | [ai](ai/) | AI agent task management and Claude integration | -| [go](go/) | Go development tools | -| [php](php/) | Laravel/PHP development tools | -| [build](build/) | Build projects | -| [ci](ci/) | Publish releases | -| [sdk](sdk/) | SDK validation and compatibility | -| [dev](dev/) | Multi-repo workflow + dev environment | -| [pkg](pkg/) | Package management | +| AI ml | ML inference and training pipeline | +| AI mcp | Model Context Protocol server | +| AI rag | RAG system for semantic search | +| AI collect | Data collection from external sources | + +### Infrastructure & Deployment +| Command | Description | +|---------|-------------| +| AI deploy | Coolify deployment management | +| AI prod | Production infrastructure management | | [vm](vm/) | LinuxKit VM management | +| AI unifi | UniFi network management | +| AI monitor | Security monitoring | + +### Git & Forge +| Command | Description | +|---------|-------------| +| AI git | Root-level git workflow | +| AI forge | Forgejo instance management | +| AI gitea | Gitea instance management | + +### Configuration & Utilities +| Command | Description | +|---------|-------------| +| AI config | Configuration management | +| AI crypt | Cryptographic utilities | | [docs](docs/) | Documentation management | -| [setup](setup/) | Clone repos from registry | -| [doctor](doctor/) | Check environment | -| [test](test/) | Run Go tests with coverage | +| AI help | Help documentation | +| AI plugin | Plugin management | +| AI workspace | Workspace configuration | + +### Services +| Command | Description | +|---------|-------------| +| AI daemon | Background service management | +| AI session | Session recording and replay | +| AI lab | Homelab monitoring dashboard | ## Installation @@ -29,3 +71,44 @@ go install forge.lthn.ai/core/cli/cmd/core@latest Verify: `core doctor` See [Getting Started](../getting-started.md) for all installation options. + +## Build Variants + +The Core CLI supports modular builds. You can create specialized variants: + +- **Full** - All commands (default) +- **Minimal** - Essential commands only +- **Developer** - Development-focused +- **DevOps** - Infrastructure and deployment +- **AI/ML** - AI and machine learning operations + +See [Build Variants](../build-variants.md) for details. + +## Quick Start + +```bash +# Check environment +core doctor + +# Multi-repo workflow +core dev work + +# Start MCP server for AI assistants +core mcp serve --workspace /path/to/project + +# Run tests +core test + +# Get help +core help +core --help +``` + +## Documentation + +- [CLI Reference](../cli-reference.md) - Complete command documentation +- [Build Variants](../build-variants.md) - Custom builds +- [MCP Integration](../mcp-integration.md) - MCP server setup +- [Environment Variables](../environment-variables.md) - Configuration options +- [Configuration](../configuration.md) - Config file reference +- [Getting Started](../getting-started.md) - Installation guide diff --git a/docs/environment-variables.md b/docs/environment-variables.md new file mode 100644 index 00000000..3ea7dd3e --- /dev/null +++ b/docs/environment-variables.md @@ -0,0 +1,735 @@ +# Environment Variables + +The Core CLI uses environment variables for configuration, authentication, and runtime behavior. This document lists all supported environment variables and their purposes. + +## Configuration System + +Environment variables are part of Core's layered configuration system: + +1. **Command-line flags** (highest priority) +2. **Environment variables** ← This document +3. **Configuration file** (`~/.core/config.yaml`) +4. **Default values** (lowest priority) + +## Configuration File Locations + +### User Configuration + +| File | Path | Purpose | +|------|------|---------| +| Main config | `~/.core/config.yaml` | User preferences and framework settings | +| Agentic config | `~/.core/agentic.yaml` | AI agent service configuration | +| MCP config | `~/.claude/mcp_config.json` | Claude Code MCP server settings | +| Keybindings | `~/.claude/keybindings.json` | Claude Code keyboard shortcuts | + +### Project Configuration + +Located in `.core/` directory of project root: + +| File | Path | Purpose | +|------|------|---------| +| Build config | `.core/build.yaml` | Build targets and flags | +| Release config | `.core/release.yaml` | Release automation settings | +| CI config | `.core/ci.yaml` | CI pipeline configuration | + +### Registry Configuration + +Searched in order: + +1. Current directory: `./repos.yaml` +2. Parent directories (walking up) +3. Home Code directory: `~/Code/host-uk/repos.yaml` +4. Config directory: `~/.config/core/repos.yaml` + +## General Configuration + +### CORE_CONFIG_* + +Map configuration values to the YAML hierarchy using dot notation. + +**Format:** `CORE_CONFIG_=` + +After stripping the `CORE_CONFIG_` prefix, the remaining variable name is converted to lowercase and underscores are replaced with dots. + +**Examples:** + +```bash +# dev.editor: vim +export CORE_CONFIG_DEV_EDITOR=vim + +# log.level: debug +export CORE_CONFIG_LOG_LEVEL=debug + +# ai.model: gpt-4 +export CORE_CONFIG_AI_MODEL=gpt-4 +``` + +### NO_COLOR + +Disable ANSI color output. + +**Values:** Any value (presence disables colors) + +**Example:** +```bash +export NO_COLOR=1 +core doctor # No colored output +``` + +### CORE_DAEMON + +Run application in daemon mode. + +**Values:** `1` or `true` + +**Example:** +```bash +export CORE_DAEMON=1 +core mcp serve # Runs as background daemon +``` + +## MCP Server + +### MCP_ADDR + +TCP address for MCP server. If not set, MCP uses stdio transport. + +**Format:** `host:port` or `:port` + +**Default:** (stdio mode) + +**Examples:** +```bash +# Listen on localhost:9999 +export MCP_ADDR=localhost:9999 +core mcp serve + +# Listen on all interfaces (⚠️ not recommended) +export MCP_ADDR=:9999 +core mcp serve +``` + +### CORE_MCP_TRANSPORT + +MCP transport mode. + +**Values:** `stdio`, `tcp`, `socket` + +**Default:** `stdio` + +**Example:** +```bash +export CORE_MCP_TRANSPORT=tcp +export CORE_MCP_ADDR=:9100 +core daemon +``` + +### CORE_MCP_ADDR + +Alternative to `MCP_ADDR` for daemon mode. + +**Format:** Same as `MCP_ADDR` + +### CORE_HEALTH_ADDR + +Health check endpoint address for daemon mode. + +**Format:** `host:port` or `:port` + +**Default:** None (disabled) + +**Example:** +```bash +export CORE_HEALTH_ADDR=:8080 +core daemon +# Health check: curl http://localhost:8080/health +``` + +### CORE_PID_FILE + +PID file location for daemon mode. + +**Default:** (platform-specific temp directory) + +**Example:** +```bash +export CORE_PID_FILE=/var/run/core-mcp.pid +core daemon +``` + +## Service APIs + +### COOLIFY_TOKEN + +API token for Coolify deployments. + +**Example:** +```bash +export COOLIFY_TOKEN=your-api-token +core deploy servers +``` + +### AGENTIC_TOKEN + +API token for Agentic AI services. + +**Example:** +```bash +export AGENTIC_TOKEN=your-token +core ai tasks list +``` + +## Networking + +### UNIFI_URL + +UniFi controller URL. + +**Format:** `https://host` or `https://host:port` + +**Example:** +```bash +export UNIFI_URL=https://192.168.1.1 +core unifi clients +``` + +### UNIFI_INSECURE + +Skip TLS certificate verification for UniFi controller. + +**Values:** `1`, `true`, `yes` + +**Example:** +```bash +export UNIFI_INSECURE=1 +export UNIFI_URL=https://192.168.1.1 +core unifi devices +``` + +## Git/Forge Integration + +### GITHUB_TOKEN + +GitHub API token for operations requiring authentication. + +**Note:** Usually set by GitHub CLI (`gh auth login`) + +**Example:** +```bash +export GITHUB_TOKEN=ghp_xxxxxxxxxxxxx +core dev issues +``` + +### FORGEJO_URL + +Forgejo instance URL. + +**Example:** +```bash +export FORGEJO_URL=https://forge.example.com +core forge repos +``` + +### FORGEJO_TOKEN + +Forgejo API token. + +**Example:** +```bash +export FORGEJO_TOKEN=your-token +core forge issues +``` + +### GITEA_URL + +Gitea instance URL. + +**Example:** +```bash +export GITEA_URL=https://gitea.example.com +core gitea repos +``` + +### GITEA_TOKEN + +Gitea API token. + +**Example:** +```bash +export GITEA_TOKEN=your-token +core gitea issues +``` + +## RAG System + +### QDRANT_HOST + +Qdrant vector database host. + +**Default:** `localhost` + +**Example:** +```bash +export QDRANT_HOST=localhost +export QDRANT_PORT=6334 +core rag collections list +``` + +### QDRANT_PORT + +Qdrant gRPC port. + +**Default:** `6334` + +### OLLAMA_HOST + +Ollama API host for embeddings. + +**Default:** `localhost` + +**Example:** +```bash +export OLLAMA_HOST=localhost +export OLLAMA_PORT=11434 +core rag ingest --collection docs --path ./documentation +``` + +### OLLAMA_PORT + +Ollama API port. + +**Default:** `11434` + +## ML Pipeline + +### ML_API_URL + +OpenAI-compatible API endpoint for ML operations. + +**Example:** +```bash +export ML_API_URL=http://localhost:8000/v1 +core ml score +``` + +### ML_JUDGE_URL + +Judge model API URL (typically Ollama). + +**Example:** +```bash +export ML_JUDGE_URL=http://localhost:11434 +core ml score --judge-model llama2 +``` + +### ML_JUDGE_MODEL + +Judge model name for scoring. + +**Default:** (varies by command) + +**Example:** +```bash +export ML_JUDGE_MODEL=llama2:latest +core ml score +``` + +### INFLUX_URL + +InfluxDB URL for metrics storage. + +**Example:** +```bash +export INFLUX_URL=http://localhost:8086 +export INFLUX_DB=metrics +core ml metrics +``` + +### INFLUX_DB + +InfluxDB database name. + +**Example:** +```bash +export INFLUX_DB=ml_metrics +core ml ingest +``` + +## Build System + +### CGO_ENABLED + +Enable or disable CGO for builds. + +**Values:** `0` (disabled), `1` (enabled) + +**Default:** `0` (Go default) + +**Example:** +```bash +export CGO_ENABLED=1 +task cli:build +``` + +### GOOS + +Target operating system for cross-compilation. + +**Values:** `linux`, `darwin`, `windows`, `freebsd`, etc. + +**Example:** +```bash +export GOOS=linux +export GOARCH=amd64 +task cli:build +``` + +### GOARCH + +Target architecture for cross-compilation. + +**Values:** `amd64`, `arm64`, `386`, `arm`, etc. + +**Example:** +```bash +export GOOS=darwin +export GOARCH=arm64 +task cli:build # Build for Apple Silicon +``` + +## Testing + +### TEST_VERBOSE + +Enable verbose test output. + +**Values:** `1`, `true` + +**Example:** +```bash +export TEST_VERBOSE=1 +core test +``` + +### TEST_SHORT + +Run short tests only. + +**Values:** `1`, `true` + +**Example:** +```bash +export TEST_SHORT=1 +core test +``` + +### TEST_RACE + +Enable race detector. + +**Values:** `1`, `true` + +**Example:** +```bash +export TEST_RACE=1 +core test +``` + +## Infrastructure + +### HETZNER_TOKEN + +Hetzner Cloud API token. + +**Example:** +```bash +export HETZNER_TOKEN=your-token +core prod status +``` + +### CLOUDNS_AUTH_ID + +CloudNS authentication ID for DNS management. + +**Example:** +```bash +export CLOUDNS_AUTH_ID=your-id +export CLOUDNS_AUTH_PASSWORD=your-password +core prod dns +``` + +### CLOUDNS_AUTH_PASSWORD + +CloudNS authentication password. + +## Virtual Machines + +### LINUXKIT_PATH + +Path to LinuxKit binary. + +**Default:** `linuxkit` (from PATH) + +**Example:** +```bash +export LINUXKIT_PATH=/usr/local/bin/linuxkit +core vm run +``` + +### VM_MEMORY + +Default memory allocation for VMs. + +**Format:** Integer (MB) or string with suffix (e.g., "2G") + +**Default:** 2048 (2GB) + +**Example:** +```bash +export VM_MEMORY=4096 +core vm run my-vm +``` + +### VM_CPUS + +Default CPU count for VMs. + +**Default:** 2 + +**Example:** +```bash +export VM_CPUS=4 +core vm run my-vm +``` + +## Development + +### DEBUG + +Enable debug logging. + +**Values:** `1`, `true` + +**Example:** +```bash +export DEBUG=1 +core dev work +``` + +### LOG_LEVEL + +Global log level. + +**Values:** `debug`, `info`, `warn`, `error` + +**Default:** `info` + +**Example:** +```bash +export LOG_LEVEL=debug +core mcp serve +``` + +### EDITOR + +Default text editor for interactive commands. + +**Default:** `vim` or system default + +**Example:** +```bash +export EDITOR=nvim +core dev commit # Uses neovim for editing +``` + +### PAGER + +Default pager for command output. + +**Default:** `less` or system default + +**Example:** +```bash +export PAGER=bat +core help +``` + +## Best Practices + +### 1. Use Config File for Persistent Settings + +Environment variables are temporary. For persistent configuration, use `~/.core/config.yaml`: + +```yaml +dev: + editor: nvim +log: + level: info +ai: + model: gpt-4 +``` + +Set via CLI: +```bash +core config set dev.editor nvim +``` + +### 2. Use .env Files for Project-Specific Settings + +Create `.env` in project root: + +```bash +# .env +MCP_ADDR=localhost:9100 +QDRANT_HOST=localhost +OLLAMA_HOST=localhost +``` + +Load with: +```bash +source .env +core mcp serve +``` + +Or use `direnv` for automatic loading. + +### 3. Secure Sensitive Tokens + +Never commit tokens to version control: + +```bash +# .gitignore +.env +.env.local +*.token +``` + +Use environment-specific files: +```bash +.env.development +.env.staging +.env.production +``` + +### 4. Override in CI/CD + +Set environment variables in CI/CD pipelines: + +```yaml +# .github/workflows/test.yml +env: + CORE_CONFIG_LOG_LEVEL: debug + NO_COLOR: 1 +``` + +### 5. Document Project Requirements + +Create `.env.example` with required variables: + +```bash +# .env.example +MCP_ADDR=localhost:9100 +QDRANT_HOST=localhost +QDRANT_PORT=6334 +OLLAMA_HOST=localhost +# FORGEJO_TOKEN=your-token-here +``` + +## Checking Configuration + +### View Current Config + +```bash +# All config values +core config list + +# Specific value +core config get dev.editor +``` + +### View Config File Location + +```bash +core config path +``` + +### Debug Environment Variables + +```bash +# Show all CORE_* variables +env | grep CORE_ + +# Show all MCP-related variables +env | grep MCP +``` + +### Test Configuration + +```bash +# Verify environment setup +core doctor --verbose +``` + +## Troubleshooting + +### Variable Not Taking Effect + +1. **Check priority:** Command flags override environment variables + ```bash + # This flag takes priority + core mcp serve --workspace /path + ``` + +2. **Verify variable is set:** + ```bash + echo $MCP_ADDR + ``` + +3. **Check for typos:** + ```bash + # Wrong + export CORE_CONFIG_dev_editor=vim + + # Correct + export CORE_CONFIG_DEV_EDITOR=vim + ``` + +4. **Restart shell or reload:** + ```bash + source ~/.bashrc # or ~/.zshrc + ``` + +### Config File vs Environment Variables + +If config file and environment variable conflict, environment variable wins: + +```yaml +# ~/.core/config.yaml +log: + level: info +``` + +```bash +# This overrides the config file +export CORE_CONFIG_LOG_LEVEL=debug +core mcp serve # Uses debug level +``` + +### Case Sensitivity + +Variable names are case-sensitive: + +```bash +# Wrong +export mcp_addr=:9100 + +# Correct +export MCP_ADDR=:9100 +``` + +## See Also + +- [Configuration Reference](configuration.md) - Complete config file documentation +- [CLI Reference](cli-reference.md) - Command documentation +- [MCP Integration](mcp-integration.md) - MCP server setup +- [Getting Started](getting-started.md) - Installation and setup diff --git a/docs/mcp-integration.md b/docs/mcp-integration.md new file mode 100644 index 00000000..718c9f59 --- /dev/null +++ b/docs/mcp-integration.md @@ -0,0 +1,658 @@ +# MCP Integration + +The Core CLI includes a Model Context Protocol (MCP) server that provides file operations, RAG (Retrieval-Augmented Generation), metrics tracking, and process management tools for AI assistants like Claude Code. + +## Overview + +MCP is a protocol that allows AI assistants to interact with external tools and services. The Core CLI's MCP server exposes a rich set of tools for: + +- **File Operations** - Read, write, edit, delete files and directories +- **RAG System** - Semantic search using Qdrant + Ollama embeddings +- **Metrics** - Record and query application metrics +- **Process Management** - Start, stop, and monitor processes +- **Language Detection** - Detect programming languages +- **WebSocket** - Real-time communication +- **WebView/CDP** - Browser automation via Chrome DevTools Protocol + +## Quick Start + +### Start MCP Server + +**Stdio mode (for Claude Code):** +```bash +core mcp serve +``` + +**TCP mode:** +```bash +MCP_ADDR=localhost:9999 core mcp serve +``` + +**With workspace restriction:** +```bash +core mcp serve --workspace /path/to/project +``` + +### Configure in Claude Code + +Add to your Claude Code MCP configuration (`~/.claude/mcp_config.json`): + +```json +{ + "mcpServers": { + "core-mcp": { + "command": "core", + "args": ["mcp", "serve"], + "env": {} + } + } +} +``` + +With workspace restriction: + +```json +{ + "mcpServers": { + "core-mcp": { + "command": "core", + "args": ["mcp", "serve", "--workspace", "/home/user/projects/myapp"], + "env": {} + } + } +} +``` + +## Transport Modes + +### Stdio (Default) + +Best for integration with Claude Code and other AI assistants that communicate via standard input/output. + +```bash +core mcp serve +``` + +**Pros:** +- Simple setup +- Secure (no network exposure) +- Works with Claude Code out of the box + +**Cons:** +- Single client only +- Process-coupled (client must manage server lifecycle) + +### TCP + +Best for network-based integrations or when you need multiple clients. + +```bash +MCP_ADDR=localhost:9999 core mcp serve +``` + +**Pros:** +- Multiple clients can connect +- Server runs independently +- Can connect from remote machines (if exposed) + +**Cons:** +- Requires network configuration +- No built-in authentication (use firewall/SSH tunnel) +- More complex setup + +## Available Tools + +### File Operations + +#### file_read +Read contents of a file. + +**Parameters:** +- `path` (string, required) - File path to read + +**Example:** +```json +{ + "name": "file_read", + "arguments": { + "path": "/path/to/file.txt" + } +} +``` + +#### file_write +Write content to a file (creates or overwrites). + +**Parameters:** +- `path` (string, required) - File path to write +- `content` (string, required) - Content to write + +**Example:** +```json +{ + "name": "file_write", + "arguments": { + "path": "/path/to/file.txt", + "content": "Hello, world!" + } +} +``` + +#### file_edit +Edit a file by replacing old content with new content. + +**Parameters:** +- `path` (string, required) - File path to edit +- `old_content` (string, required) - Content to find +- `new_content` (string, required) - Replacement content + +**Example:** +```json +{ + "name": "file_edit", + "arguments": { + "path": "/path/to/file.txt", + "old_content": "old text", + "new_content": "new text" + } +} +``` + +#### file_delete +Delete a file. + +**Parameters:** +- `path` (string, required) - File path to delete + +#### file_rename +Rename or move a file. + +**Parameters:** +- `old_path` (string, required) - Current file path +- `new_path` (string, required) - New file path + +#### file_exists +Check if a file or directory exists. + +**Parameters:** +- `path` (string, required) - Path to check + +**Returns:** +- `exists` (boolean) +- `is_dir` (boolean) + +#### dir_list +List contents of a directory. + +**Parameters:** +- `path` (string, required) - Directory path +- `recursive` (boolean, optional) - Recursively list subdirectories + +#### dir_create +Create a directory (including parent directories). + +**Parameters:** +- `path` (string, required) - Directory path to create + +### RAG (Retrieval-Augmented Generation) + +#### rag_ingest +Ingest documents into a collection for semantic search. + +**Parameters:** +- `collection` (string, required) - Collection name +- `documents` (array, required) - Documents to ingest + - `id` (string) - Document ID + - `text` (string) - Document content + - `metadata` (object, optional) - Additional metadata + +**Example:** +```json +{ + "name": "rag_ingest", + "arguments": { + "collection": "documentation", + "documents": [ + { + "id": "doc1", + "text": "This is a documentation page about MCP", + "metadata": {"type": "docs", "topic": "mcp"} + } + ] + } +} +``` + +#### rag_query +Query a collection using semantic search. + +**Parameters:** +- `collection` (string, required) - Collection name +- `query` (string, required) - Search query +- `limit` (integer, optional) - Maximum results (default: 5) + +**Returns:** +Array of matching documents with scores. + +#### rag_collections +List all RAG collections. + +**Returns:** +Array of collection names and their document counts. + +### Metrics + +#### metrics_record +Record a metric value. + +**Parameters:** +- `name` (string, required) - Metric name +- `value` (number, required) - Metric value +- `tags` (object, optional) - Metric tags/labels +- `timestamp` (string, optional) - ISO 8601 timestamp + +**Example:** +```json +{ + "name": "metrics_record", + "arguments": { + "name": "api.requests", + "value": 1, + "tags": {"endpoint": "/api/users", "status": "200"} + } +} +``` + +#### metrics_query +Query recorded metrics. + +**Parameters:** +- `name` (string, optional) - Filter by metric name +- `tags` (object, optional) - Filter by tags +- `from` (string, optional) - Start time (ISO 8601) +- `to` (string, optional) - End time (ISO 8601) + +### Language Detection + +#### lang_detect +Detect programming language from code or filename. + +**Parameters:** +- `content` (string, optional) - Code content +- `filename` (string, optional) - Filename + +**Returns:** +- `language` (string) - Detected language +- `confidence` (number) - Detection confidence (0-1) + +#### lang_list +List all supported languages. + +**Returns:** +Array of supported language identifiers. + +### Process Management + +#### process_start +Start a new process. + +**Parameters:** +- `id` (string, required) - Process identifier +- `command` (string, required) - Command to execute +- `args` (array, optional) - Command arguments +- `env` (object, optional) - Environment variables +- `cwd` (string, optional) - Working directory + +**Example:** +```json +{ + "name": "process_start", + "arguments": { + "id": "build-process", + "command": "go", + "args": ["build", "-o", "bin/app"], + "cwd": "/path/to/project" + } +} +``` + +#### process_stop +Stop a running process (SIGTERM). + +**Parameters:** +- `id` (string, required) - Process identifier + +#### process_kill +Force kill a process (SIGKILL). + +**Parameters:** +- `id` (string, required) - Process identifier + +#### process_list +List all managed processes. + +**Returns:** +Array of process info (ID, command, status, PID). + +#### process_output +Get output from a process. + +**Parameters:** +- `id` (string, required) - Process identifier +- `tail` (integer, optional) - Number of recent lines + +#### process_input +Send input to a process. + +**Parameters:** +- `id` (string, required) - Process identifier +- `input` (string, required) - Input to send + +### WebSocket + +#### ws_start +Start a WebSocket hub server. + +**Parameters:** +- `addr` (string, optional) - Listen address (default: ":8080") + +**Returns:** +- `url` (string) - WebSocket server URL + +#### ws_info +Get WebSocket server information. + +**Returns:** +- `running` (boolean) +- `url` (string) +- `connections` (integer) + +### WebView/CDP (Chrome DevTools Protocol) + +#### webview_connect +Connect to Chrome/Chromium via CDP. + +**Parameters:** +- `url` (string, required) - Debug URL (e.g., "http://localhost:9222") + +#### webview_navigate +Navigate to a URL. + +**Parameters:** +- `url` (string, required) - URL to navigate to + +#### webview_click +Click an element. + +**Parameters:** +- `selector` (string, required) - CSS selector + +#### webview_type +Type text into an element. + +**Parameters:** +- `selector` (string, required) - CSS selector +- `text` (string, required) - Text to type + +#### webview_query +Query DOM for elements. + +**Parameters:** +- `selector` (string, required) - CSS selector + +**Returns:** +Array of matching elements. + +#### webview_console +Get console logs. + +**Returns:** +Array of console messages. + +#### webview_eval +Evaluate JavaScript. + +**Parameters:** +- `script` (string, required) - JavaScript code + +**Returns:** +Evaluation result. + +#### webview_screenshot +Take a screenshot. + +**Parameters:** +- `full_page` (boolean, optional) - Full page screenshot + +**Returns:** +- `data` (string) - Base64-encoded PNG + +#### webview_wait +Wait for condition. + +**Parameters:** +- `type` (string, required) - Condition type ("selector", "navigation", "timeout") +- `value` (string/number, required) - Condition value + +#### webview_disconnect +Disconnect from CDP. + +## Security Considerations + +### Workspace Restriction + +**Always** use `--workspace` flag when exposing MCP server to AI assistants: + +```bash +core mcp serve --workspace /path/to/safe/directory +``` + +This restricts file operations to the specified directory and its subdirectories, preventing the AI from accessing sensitive files outside the project. + +### File Operations + +Without workspace restriction, the MCP server has full filesystem access. This is powerful but dangerous: + +❌ **Never** run unrestricted in production: +```bash +core mcp serve # Full filesystem access! +``` + +✅ **Always** restrict to project directory: +```bash +core mcp serve --workspace /home/user/projects/myapp +``` + +### Network Exposure + +When using TCP mode: + +- **Localhost only** by default: `MCP_ADDR=localhost:9999` +- **Never** expose to public internet without authentication +- Use SSH tunnels for remote access +- Consider firewall rules to restrict access + +### Process Management + +Processes started via `process_start` run with the same permissions as the MCP server. Be cautious when: + +- Running as root/administrator +- Executing untrusted commands +- Exposing process management to external clients + +## Running as Daemon + +Use the `daemon` command to run MCP server as a background service: + +```bash +core daemon --mcp-transport tcp --mcp-addr :9100 +``` + +With health endpoint: + +```bash +core daemon --mcp-transport tcp --mcp-addr :9100 --health-addr :8080 +``` + +Check health: + +```bash +curl http://localhost:8080/health +``` + +## Environment Variables + +| Variable | Description | Default | +|----------|-------------|---------| +| `MCP_ADDR` | TCP listen address | (stdio) | +| `CORE_MCP_TRANSPORT` | Transport mode (stdio, tcp) | stdio | +| `CORE_MCP_ADDR` | Alternative to MCP_ADDR | - | +| `CORE_HEALTH_ADDR` | Health check endpoint | - | + +## Use Cases + +### 1. Claude Code Integration + +Enable Claude Code to read and modify files in your project: + +```json +{ + "mcpServers": { + "core": { + "command": "core", + "args": ["mcp", "serve", "--workspace", "${workspaceFolder}"] + } + } +} +``` + +### 2. Semantic Code Search + +Ingest codebase into RAG system for semantic search: + +```bash +# Start MCP server +core mcp serve --workspace /path/to/project + +# Ingest code (via AI assistant) +# Use: rag_ingest with collection="code" and documents from source files + +# Query code +# Use: rag_query with query="authentication logic" +``` + +### 3. Build Automation + +Monitor build processes with real-time output: + +```bash +# Start MCP with WebSocket +# Use: ws_start +# Use: process_start with command="go build" +# Subscribe to process output via WebSocket +``` + +### 4. Web Automation Testing + +Automate browser testing: + +```bash +# Start Chrome with remote debugging: +# google-chrome --remote-debugging-port=9222 + +# Use CDP tools: +# webview_connect url="http://localhost:9222" +# webview_navigate url="https://example.com" +# webview_click selector="#login-button" +# webview_screenshot +``` + +## Troubleshooting + +### Server Won't Start + +**Error:** Port already in use + +```bash +# Check what's using the port +lsof -i :9999 + +# Kill the process or use different port +MCP_ADDR=:9100 core mcp serve +``` + +**Error:** Permission denied + +```bash +# Don't use privileged ports (<1024) without sudo +# Use high ports instead: +MCP_ADDR=:9999 core mcp serve +``` + +### File Operations Fail + +**Error:** File outside workspace + +Make sure file paths are within the workspace: + +```bash +# Workspace: /home/user/project +# ✅ OK: /home/user/project/src/main.go +# ❌ Fail: /etc/passwd +``` + +**Error:** Permission denied + +Check file/directory permissions and MCP server user. + +### RAG Not Working + +**Error:** Qdrant connection failed + +Ensure Qdrant is running: + +```bash +# Start Qdrant (Docker) +docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant +``` + +**Error:** Ollama not available + +Ensure Ollama is running: + +```bash +# Start Ollama +ollama serve + +# Pull embedding model +ollama pull nomic-embed-text +``` + +### Process Management Issues + +**Error:** Process not found + +List processes to verify ID: + +```json +{"name": "process_list"} +``` + +**Error:** Process already exists + +Use unique process IDs or stop existing process first. + +## Performance Tips + +1. **Use workspace restriction** - Reduces filesystem traversal overhead +2. **Batch RAG ingestion** - Ingest multiple documents in one call +3. **Limit query results** - Use `limit` parameter in rag_query +4. **Process output streaming** - Use WebSocket for real-time output +5. **Reuse connections** - Keep WebView CDP connection open for multiple operations + +## See Also + +- [CLI Reference](cli-reference.md) - Complete command documentation +- [Daemon Mode](daemon-mode.md) - Running MCP as a service +- [RAG System](rag-system.md) - Detailed RAG documentation +- [Environment Variables](environment-variables.md) - Configuration options diff --git a/mkdocs.yml b/mkdocs.yml index 09585dea..cb4a3214 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -50,19 +50,24 @@ nav: - Workflows: workflows.md - CLI Reference: - Overview: cmd/index.md - - AI: cmd/ai/index.md - - Build: cmd/build/index.md - - CI: cmd/ci/index.md - - Dev: cmd/dev/index.md - - Go: cmd/go/index.md - - PHP: cmd/php/index.md - - SDK: cmd/sdk/index.md - - Setup: cmd/setup/index.md - - Doctor: cmd/doctor/index.md - - Test: cmd/test/index.md - - VM: cmd/vm/index.md - - Pkg: cmd/pkg/index.md - - Docs: cmd/docs/index.md + - Complete Reference: cli-reference.md + - Build Variants: build-variants.md + - MCP Integration: mcp-integration.md + - Environment Variables: environment-variables.md + - Commands: + - AI: cmd/ai/index.md + - Build: cmd/build/index.md + - CI: cmd/ci/index.md + - Dev: cmd/dev/index.md + - Go: cmd/go/index.md + - PHP: cmd/php/index.md + - SDK: cmd/sdk/index.md + - Setup: cmd/setup/index.md + - Doctor: cmd/doctor/index.md + - Test: cmd/test/index.md + - VM: cmd/vm/index.md + - Pkg: cmd/pkg/index.md + - Docs: cmd/docs/index.md - Getting Started: - Installation: getting-started/installation.md - Quick Start: getting-started/quickstart.md