cli/pkg/ansible/ssh.go

452 lines
11 KiB
Go
Raw Normal View History

feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
package ansible
import (
"bytes"
"context"
"fmt"
"io"
"net"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/host-uk/core/pkg/log"
"golang.org/x/crypto/ssh"
"golang.org/x/crypto/ssh/knownhosts"
)
// SSHClient handles SSH connections to remote hosts.
type SSHClient struct {
host string
port int
user string
password string
keyFile string
client *ssh.Client
mu sync.Mutex
become bool
becomeUser string
becomePass string
timeout time.Duration
}
// SSHConfig holds SSH connection configuration.
type SSHConfig struct {
Host string
Port int
User string
Password string
KeyFile string
Become bool
BecomeUser string
BecomePass string
Timeout time.Duration
}
// NewSSHClient creates a new SSH client.
func NewSSHClient(cfg SSHConfig) (*SSHClient, error) {
if cfg.Port == 0 {
cfg.Port = 22
}
if cfg.User == "" {
cfg.User = "root"
}
if cfg.Timeout == 0 {
cfg.Timeout = 30 * time.Second
}
client := &SSHClient{
host: cfg.Host,
port: cfg.Port,
user: cfg.User,
password: cfg.Password,
keyFile: cfg.KeyFile,
become: cfg.Become,
becomeUser: cfg.BecomeUser,
becomePass: cfg.BecomePass,
timeout: cfg.Timeout,
}
return client, nil
}
// Connect establishes the SSH connection.
func (c *SSHClient) Connect(ctx context.Context) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.client != nil {
return nil
}
var authMethods []ssh.AuthMethod
// Try key-based auth first
if c.keyFile != "" {
keyPath := c.keyFile
if strings.HasPrefix(keyPath, "~") {
home, _ := os.UserHomeDir()
keyPath = filepath.Join(home, keyPath[1:])
}
if key, err := os.ReadFile(keyPath); err == nil {
if signer, err := ssh.ParsePrivateKey(key); err == nil {
authMethods = append(authMethods, ssh.PublicKeys(signer))
}
}
}
// Try default SSH keys
if len(authMethods) == 0 {
home, _ := os.UserHomeDir()
defaultKeys := []string{
filepath.Join(home, ".ssh", "id_ed25519"),
filepath.Join(home, ".ssh", "id_rsa"),
}
for _, keyPath := range defaultKeys {
if key, err := os.ReadFile(keyPath); err == nil {
if signer, err := ssh.ParsePrivateKey(key); err == nil {
authMethods = append(authMethods, ssh.PublicKeys(signer))
break
}
}
}
}
// Fall back to password auth
if c.password != "" {
authMethods = append(authMethods, ssh.Password(c.password))
authMethods = append(authMethods, ssh.KeyboardInteractive(func(user, instruction string, questions []string, echos []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range questions {
answers[i] = c.password
}
return answers, nil
}))
}
if len(authMethods) == 0 {
return log.E("ssh.Connect", "no authentication method available", nil)
}
// Host key verification
var hostKeyCallback ssh.HostKeyCallback
feat: BugSETI app, WebSocket hub, browser automation, and MCP tools (#336) * feat: add security logging and fix framework regressions This commit implements comprehensive security event logging and resolves critical regressions in the core framework. Security Logging: - Enhanced `pkg/log` with a `Security` level and helper. - Added `log.Username()` to consistently identify the executing user. - Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs. - Added `SecurityStyle` to the CLI for consistent visual representation of security events. UniFi Security (CodeQL): - Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert. - Added a `--verify-tls` flag and configuration option to control TLS verification. - Updated command handlers to support the new verification parameter. Framework Fixes: - Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge. - Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures. - These fixes resolve project-wide compilation errors caused by the signature mismatches. I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability. * feat(mcp): add RAG tools (query, ingest, collections) Add vector database tools to the MCP server for RAG operations: - rag_query: Search for relevant documentation using semantic similarity - rag_ingest: Ingest files or directories into the vector database - rag_collections: List available collections Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile) and pkg/rag for Qdrant client access. Default collection is "hostuk-docs" with topK=5 for queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add metrics tools (record, query) Add MCP tools for recording and querying AI/security metrics events. The metrics_record tool writes events to daily JSONL files, and the metrics_query tool provides aggregated statistics by type, repo, and agent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add 'core mcp serve' command Add CLI command to start the MCP server for AI tool integration. - Create internal/cmd/mcpcmd package with serve subcommand - Support --workspace flag for directory restriction - Handle SIGINT/SIGTERM for clean shutdown - Register in full.go build variant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ws): add WebSocket hub package for real-time streaming Add pkg/ws package implementing a hub pattern for WebSocket connections: - Hub manages client connections, broadcasts, and channel subscriptions - Client struct represents connected WebSocket clients - Message types: process_output, process_status, event, error, ping/pong - Channel-based subscription system (subscribe/unsubscribe) - SendProcessOutput and SendProcessStatus for process streaming integration - Full test coverage including concurrency tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add process management and WebSocket MCP tools Add MCP tools for process management: - process_start: Start a new external process - process_stop: Gracefully stop a running process - process_kill: Force kill a process - process_list: List all managed processes - process_output: Get captured process output - process_input: Send input to process stdin Add MCP tools for WebSocket: - ws_start: Start WebSocket server for real-time streaming - ws_info: Get hub statistics (clients, channels) Update Service struct with optional process.Service and ws.Hub fields, new WithProcessService and WithWSHub options, getter methods, and Shutdown method for cleanup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(webview): add browser automation package via Chrome DevTools Protocol Add pkg/webview package for browser automation: - webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate - cdp.go: Chrome DevTools Protocol WebSocket client implementation - actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder - console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher - angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability Add MCP tools for webview: - webview_connect/disconnect: Connection management - webview_navigate: Page navigation - webview_click/type/query/wait: DOM interaction - webview_console: Console output capture - webview_eval: JavaScript execution - webview_screenshot: Screenshot capture Add documentation: - docs/mcp/angular-testing.md: Guide for Angular application testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: document new packages and BugSETI application - Update CLAUDE.md with documentation for: - pkg/ws (WebSocket hub for real-time streaming) - pkg/webview (Browser automation via CDP) - pkg/mcp (MCP server tools: process, ws, webview) - BugSETI application overview - Add comprehensive README for BugSETI with: - Installation and configuration guide - Usage workflow documentation - Architecture overview - Contributing guidelines Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(bugseti): add BugSETI system tray app with auto-update BugSETI - Distributed Bug Fixing like SETI@home but for code Features: - System tray app with Wails v3 - GitHub issue fetching with label filters - Issue queue with priority management - AI context seeding via seed-agent-developer skill - Automated PR submission flow - Stats tracking and leaderboard - Cross-platform notifications - Self-updating with stable/beta/nightly channels Includes: - cmd/bugseti: Main application with Angular frontend - internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify) - internal/bugseti/updater: Auto-update system (checker, downloader, installer) - .github/workflows/bugseti-release.yml: CI/CD for all platforms Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve import cycle and code duplication - Remove pkg/log import from pkg/io/local to break import cycle (pkg/log/rotation.go imports pkg/io, creating circular dependency) - Use stderr logging for security events in sandbox escape detection - Remove unused sync/atomic import from core.go - Fix duplicate LogSecurity function declarations in cli/log.go - Update workspace/service.go Crypt() call to match interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update tests for new function signatures and format code - Update core_test.go: Config(), Display() now panic instead of returning error - Update runtime_pkg_test.go: sr.Config() now panics instead of returning error - Update MustServiceFor tests to use assert.Panics - Format BugSETI, MCP tools, and webview packages with gofmt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Snider <631881+Snider@users.noreply.github.com> Co-authored-by: Claude <developers@lethean.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 17:22:05 +00:00
home, err := os.UserHomeDir()
if err != nil {
return log.E("ssh.Connect", "failed to get user home dir", err)
}
knownHostsPath := filepath.Join(home, ".ssh", "known_hosts")
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
feat: BugSETI app, WebSocket hub, browser automation, and MCP tools (#336) * feat: add security logging and fix framework regressions This commit implements comprehensive security event logging and resolves critical regressions in the core framework. Security Logging: - Enhanced `pkg/log` with a `Security` level and helper. - Added `log.Username()` to consistently identify the executing user. - Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs. - Added `SecurityStyle` to the CLI for consistent visual representation of security events. UniFi Security (CodeQL): - Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert. - Added a `--verify-tls` flag and configuration option to control TLS verification. - Updated command handlers to support the new verification parameter. Framework Fixes: - Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge. - Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures. - These fixes resolve project-wide compilation errors caused by the signature mismatches. I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability. * feat(mcp): add RAG tools (query, ingest, collections) Add vector database tools to the MCP server for RAG operations: - rag_query: Search for relevant documentation using semantic similarity - rag_ingest: Ingest files or directories into the vector database - rag_collections: List available collections Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile) and pkg/rag for Qdrant client access. Default collection is "hostuk-docs" with topK=5 for queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add metrics tools (record, query) Add MCP tools for recording and querying AI/security metrics events. The metrics_record tool writes events to daily JSONL files, and the metrics_query tool provides aggregated statistics by type, repo, and agent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add 'core mcp serve' command Add CLI command to start the MCP server for AI tool integration. - Create internal/cmd/mcpcmd package with serve subcommand - Support --workspace flag for directory restriction - Handle SIGINT/SIGTERM for clean shutdown - Register in full.go build variant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ws): add WebSocket hub package for real-time streaming Add pkg/ws package implementing a hub pattern for WebSocket connections: - Hub manages client connections, broadcasts, and channel subscriptions - Client struct represents connected WebSocket clients - Message types: process_output, process_status, event, error, ping/pong - Channel-based subscription system (subscribe/unsubscribe) - SendProcessOutput and SendProcessStatus for process streaming integration - Full test coverage including concurrency tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add process management and WebSocket MCP tools Add MCP tools for process management: - process_start: Start a new external process - process_stop: Gracefully stop a running process - process_kill: Force kill a process - process_list: List all managed processes - process_output: Get captured process output - process_input: Send input to process stdin Add MCP tools for WebSocket: - ws_start: Start WebSocket server for real-time streaming - ws_info: Get hub statistics (clients, channels) Update Service struct with optional process.Service and ws.Hub fields, new WithProcessService and WithWSHub options, getter methods, and Shutdown method for cleanup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(webview): add browser automation package via Chrome DevTools Protocol Add pkg/webview package for browser automation: - webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate - cdp.go: Chrome DevTools Protocol WebSocket client implementation - actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder - console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher - angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability Add MCP tools for webview: - webview_connect/disconnect: Connection management - webview_navigate: Page navigation - webview_click/type/query/wait: DOM interaction - webview_console: Console output capture - webview_eval: JavaScript execution - webview_screenshot: Screenshot capture Add documentation: - docs/mcp/angular-testing.md: Guide for Angular application testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: document new packages and BugSETI application - Update CLAUDE.md with documentation for: - pkg/ws (WebSocket hub for real-time streaming) - pkg/webview (Browser automation via CDP) - pkg/mcp (MCP server tools: process, ws, webview) - BugSETI application overview - Add comprehensive README for BugSETI with: - Installation and configuration guide - Usage workflow documentation - Architecture overview - Contributing guidelines Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(bugseti): add BugSETI system tray app with auto-update BugSETI - Distributed Bug Fixing like SETI@home but for code Features: - System tray app with Wails v3 - GitHub issue fetching with label filters - Issue queue with priority management - AI context seeding via seed-agent-developer skill - Automated PR submission flow - Stats tracking and leaderboard - Cross-platform notifications - Self-updating with stable/beta/nightly channels Includes: - cmd/bugseti: Main application with Angular frontend - internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify) - internal/bugseti/updater: Auto-update system (checker, downloader, installer) - .github/workflows/bugseti-release.yml: CI/CD for all platforms Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve import cycle and code duplication - Remove pkg/log import from pkg/io/local to break import cycle (pkg/log/rotation.go imports pkg/io, creating circular dependency) - Use stderr logging for security events in sandbox escape detection - Remove unused sync/atomic import from core.go - Fix duplicate LogSecurity function declarations in cli/log.go - Update workspace/service.go Crypt() call to match interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update tests for new function signatures and format code - Update core_test.go: Config(), Display() now panic instead of returning error - Update runtime_pkg_test.go: sr.Config() now panics instead of returning error - Update MustServiceFor tests to use assert.Panics - Format BugSETI, MCP tools, and webview packages with gofmt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Snider <631881+Snider@users.noreply.github.com> Co-authored-by: Claude <developers@lethean.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 17:22:05 +00:00
// Ensure known_hosts file exists
if _, err := os.Stat(knownHostsPath); os.IsNotExist(err) {
if err := os.MkdirAll(filepath.Dir(knownHostsPath), 0700); err != nil {
return log.E("ssh.Connect", "failed to create .ssh dir", err)
}
if err := os.WriteFile(knownHostsPath, nil, 0600); err != nil {
return log.E("ssh.Connect", "failed to create known_hosts file", err)
}
Remove StrictHostKeyChecking=no from SSH commands (#315) * Remove StrictHostKeyChecking=no and implement proper host key verification This commit addresses security concerns from the OWASP audit by enforcing strict host key verification for all SSH and SCP commands. Key changes: - Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and pkg/devops. - Removed insecure host key verification from pkg/ansible SSH client. - Implemented a synchronous host key discovery mechanism during VM boot using ssh-keyscan to populate ~/.core/known_hosts. - Updated the devops Boot lifecycle to wait until the host key is verified. - Ensured pkg/ansible correctly handles missing known_hosts files. - Refactored hardcoded SSH port 2222 to a package constant DefaultSSHPort. - Added CORE_SKIP_SSH_SCAN environment variable for test environments. * Remove StrictHostKeyChecking=no and implement proper host key verification Addresses security concerns from OWASP audit by enforcing strict host key verification. Changes: - Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and devops. - Removed insecure host key verification from pkg/ansible. - Added synchronous host key discovery using ssh-keyscan during VM boot. - Updated Boot lifecycle to wait for host key verification. - Handled missing known_hosts file in pkg/ansible. - Refactored hardcoded SSH port to DefaultSSHPort constant. - Fixed formatting issues identified by QA check. * Secure SSH commands and fix auto-merge CI failure Addresses OWASP security audit by enforcing strict host key verification and fixes a CI failure in the auto-merge workflow. Key changes: - Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and pkg/devops. - Removed insecure host key verification from pkg/ansible. - Implemented synchronous host key discovery using ssh-keyscan during VM boot. - Handled missing known_hosts file in pkg/ansible. - Refactored hardcoded SSH port to DefaultSSHPort constant. - Added pkg/ansible/ssh_test.go to verify SSH client initialization. - Fixed formatting in pkg/io/local/client.go. - Fixed auto-merge.yml by inlining the script and providing repository context to 'gh' command, resolving the "not a git repository" error in CI. * Secure SSH, fix CI auto-merge, and resolve merge conflicts This commit addresses the OWASP security audit by enforcing strict host key verification and resolves persistent CI issues. Security Changes: - Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and devops. - Removed insecure host key verification from pkg/ansible. - Implemented synchronous host key discovery using ssh-keyscan during VM boot. - Updated Boot lifecycle to wait for host key verification. - Handled missing known_hosts file in pkg/ansible. - Refactored hardcoded SSH port to DefaultSSHPort constant. CI and Maintenance: - Fixed auto-merge.yml by inlining the script and adding repository context to 'gh' command, resolving the "not a git repository" error in CI. - Resolved merge conflicts in .github/workflows/auto-merge.yml with dev branch. - Added pkg/ansible/ssh_test.go for SSH client verification. - Fixed formatting in pkg/io/local/client.go to pass QA checks. * Secure SSH and TLS connections, and fix CI issues Addresses security concerns from OWASP audit and CodeQL by enforcing strict host key verification and TLS certificate verification. Security Changes: - Enforced strict SSH host key checking in pkg/container and devops. - Removed insecure SSH host key verification from pkg/ansible. - Added synchronous host key discovery during VM boot using ssh-keyscan. - Updated UniFi client to enforce TLS certificate verification by default. - Added --insecure flag and config option for UniFi to allow opt-in to skipping TLS verification for self-signed certificates. CI and Maintenance: - Fixed auto-merge workflow by providing repository context to 'gh' command. - Resolved merge conflicts in .github/workflows/auto-merge.yml. - Added unit tests for secured Ansible SSH client. - Fixed formatting issues identified by QA checks. * fix: gofmt alignment in cmd_config.go Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Secure connections, fix CI auto-merge, and resolve formatting Addresses OWASP security audit and CodeQL security alerts by enforcing secure defaults for SSH and TLS connections. Key changes: - Enforced strict SSH host key checking (StrictHostKeyChecking=yes). - Implemented synchronous host key verification during VM boot using ssh-keyscan. - Updated UniFi client to enforce TLS certificate verification by default. - Added --insecure flag and config option for UniFi to allow opt-in to skipping TLS verification. - Fixed auto-merge workflow by providing repository context to 'gh' command. - Resolved merge conflicts in .github/workflows/auto-merge.yml. - Fixed formatting in internal/cmd/unifi/cmd_config.go and pkg/io/local/client.go. - Added unit tests for secured Ansible SSH client. --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Claude <developers@lethean.io>
2026-02-05 07:09:34 +00:00
}
feat: BugSETI app, WebSocket hub, browser automation, and MCP tools (#336) * feat: add security logging and fix framework regressions This commit implements comprehensive security event logging and resolves critical regressions in the core framework. Security Logging: - Enhanced `pkg/log` with a `Security` level and helper. - Added `log.Username()` to consistently identify the executing user. - Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs. - Added `SecurityStyle` to the CLI for consistent visual representation of security events. UniFi Security (CodeQL): - Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert. - Added a `--verify-tls` flag and configuration option to control TLS verification. - Updated command handlers to support the new verification parameter. Framework Fixes: - Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge. - Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures. - These fixes resolve project-wide compilation errors caused by the signature mismatches. I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability. * feat(mcp): add RAG tools (query, ingest, collections) Add vector database tools to the MCP server for RAG operations: - rag_query: Search for relevant documentation using semantic similarity - rag_ingest: Ingest files or directories into the vector database - rag_collections: List available collections Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile) and pkg/rag for Qdrant client access. Default collection is "hostuk-docs" with topK=5 for queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add metrics tools (record, query) Add MCP tools for recording and querying AI/security metrics events. The metrics_record tool writes events to daily JSONL files, and the metrics_query tool provides aggregated statistics by type, repo, and agent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add 'core mcp serve' command Add CLI command to start the MCP server for AI tool integration. - Create internal/cmd/mcpcmd package with serve subcommand - Support --workspace flag for directory restriction - Handle SIGINT/SIGTERM for clean shutdown - Register in full.go build variant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ws): add WebSocket hub package for real-time streaming Add pkg/ws package implementing a hub pattern for WebSocket connections: - Hub manages client connections, broadcasts, and channel subscriptions - Client struct represents connected WebSocket clients - Message types: process_output, process_status, event, error, ping/pong - Channel-based subscription system (subscribe/unsubscribe) - SendProcessOutput and SendProcessStatus for process streaming integration - Full test coverage including concurrency tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add process management and WebSocket MCP tools Add MCP tools for process management: - process_start: Start a new external process - process_stop: Gracefully stop a running process - process_kill: Force kill a process - process_list: List all managed processes - process_output: Get captured process output - process_input: Send input to process stdin Add MCP tools for WebSocket: - ws_start: Start WebSocket server for real-time streaming - ws_info: Get hub statistics (clients, channels) Update Service struct with optional process.Service and ws.Hub fields, new WithProcessService and WithWSHub options, getter methods, and Shutdown method for cleanup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(webview): add browser automation package via Chrome DevTools Protocol Add pkg/webview package for browser automation: - webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate - cdp.go: Chrome DevTools Protocol WebSocket client implementation - actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder - console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher - angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability Add MCP tools for webview: - webview_connect/disconnect: Connection management - webview_navigate: Page navigation - webview_click/type/query/wait: DOM interaction - webview_console: Console output capture - webview_eval: JavaScript execution - webview_screenshot: Screenshot capture Add documentation: - docs/mcp/angular-testing.md: Guide for Angular application testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: document new packages and BugSETI application - Update CLAUDE.md with documentation for: - pkg/ws (WebSocket hub for real-time streaming) - pkg/webview (Browser automation via CDP) - pkg/mcp (MCP server tools: process, ws, webview) - BugSETI application overview - Add comprehensive README for BugSETI with: - Installation and configuration guide - Usage workflow documentation - Architecture overview - Contributing guidelines Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(bugseti): add BugSETI system tray app with auto-update BugSETI - Distributed Bug Fixing like SETI@home but for code Features: - System tray app with Wails v3 - GitHub issue fetching with label filters - Issue queue with priority management - AI context seeding via seed-agent-developer skill - Automated PR submission flow - Stats tracking and leaderboard - Cross-platform notifications - Self-updating with stable/beta/nightly channels Includes: - cmd/bugseti: Main application with Angular frontend - internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify) - internal/bugseti/updater: Auto-update system (checker, downloader, installer) - .github/workflows/bugseti-release.yml: CI/CD for all platforms Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve import cycle and code duplication - Remove pkg/log import from pkg/io/local to break import cycle (pkg/log/rotation.go imports pkg/io, creating circular dependency) - Use stderr logging for security events in sandbox escape detection - Remove unused sync/atomic import from core.go - Fix duplicate LogSecurity function declarations in cli/log.go - Update workspace/service.go Crypt() call to match interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update tests for new function signatures and format code - Update core_test.go: Config(), Display() now panic instead of returning error - Update runtime_pkg_test.go: sr.Config() now panics instead of returning error - Update MustServiceFor tests to use assert.Panics - Format BugSETI, MCP tools, and webview packages with gofmt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Snider <631881+Snider@users.noreply.github.com> Co-authored-by: Claude <developers@lethean.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 17:22:05 +00:00
cb, err := knownhosts.New(knownHostsPath)
if err != nil {
return log.E("ssh.Connect", "failed to load known_hosts", err)
}
hostKeyCallback = cb
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
config := &ssh.ClientConfig{
User: c.user,
Auth: authMethods,
HostKeyCallback: hostKeyCallback,
Timeout: c.timeout,
}
addr := fmt.Sprintf("%s:%d", c.host, c.port)
// Connect with context timeout
var d net.Dialer
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return log.E("ssh.Connect", fmt.Sprintf("dial %s", addr), err)
}
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
if err != nil {
// conn is closed by NewClientConn on error
return log.E("ssh.Connect", fmt.Sprintf("ssh connect %s", addr), err)
}
c.client = ssh.NewClient(sshConn, chans, reqs)
return nil
}
// Close closes the SSH connection.
func (c *SSHClient) Close() error {
c.mu.Lock()
defer c.mu.Unlock()
if c.client != nil {
err := c.client.Close()
c.client = nil
return err
}
return nil
}
// Run executes a command on the remote host.
func (c *SSHClient) Run(ctx context.Context, cmd string) (stdout, stderr string, exitCode int, err error) {
if err := c.Connect(ctx); err != nil {
return "", "", -1, err
}
session, err := c.client.NewSession()
if err != nil {
return "", "", -1, log.E("ssh.Run", "new session", err)
}
defer func() { _ = session.Close() }()
var stdoutBuf, stderrBuf bytes.Buffer
session.Stdout = &stdoutBuf
session.Stderr = &stderrBuf
// Apply become if needed
if c.become {
becomeUser := c.becomeUser
if becomeUser == "" {
becomeUser = "root"
}
// Escape single quotes in the command
escapedCmd := strings.ReplaceAll(cmd, "'", "'\\''")
if c.becomePass != "" {
// Use sudo with password via stdin (-S flag)
// We launch a goroutine to write the password to stdin
cmd = fmt.Sprintf("sudo -S -u %s bash -c '%s'", becomeUser, escapedCmd)
stdin, err := session.StdinPipe()
if err != nil {
return "", "", -1, log.E("ssh.Run", "stdin pipe", err)
}
go func() {
feat: infrastructure packages and lint cleanup (#281) * ci: consolidate duplicate workflows and merge CodeQL configs Remove 17 duplicate workflow files that were split copies of the combined originals. Each family (CI, CodeQL, Coverage, PR Build, Alpha Release) had the same job duplicated across separate push/pull_request/schedule/manual trigger files. Merge codeql.yml and codescan.yml into a single codeql.yml with a language matrix covering go, javascript-typescript, python, and actions — matching the previous default setup coverage. Remaining workflows (one per family): - ci.yml (push + PR + manual) - codeql.yml (push + PR + schedule, all languages) - coverage.yml (push + PR + manual) - alpha-release.yml (push + manual) - pr-build.yml (PR + manual) - release.yml (tag push) - agent-verify.yml, auto-label.yml, auto-project.yml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add collect, config, crypt, plugin packages and fix all lint issues Add four new infrastructure packages with CLI commands: - pkg/config: layered configuration (defaults → file → env → flags) - pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums) - pkg/plugin: plugin system with GitHub-based install/update/remove - pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate) Fix all golangci-lint issues across the entire codebase (~100 errcheck, staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that `core go qa` passes with 0 issues. Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:34:43 +00:00
defer func() { _ = stdin.Close() }()
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
_, _ = io.WriteString(stdin, c.becomePass+"\n")
}()
} else if c.password != "" {
// Try using connection password for sudo
cmd = fmt.Sprintf("sudo -S -u %s bash -c '%s'", becomeUser, escapedCmd)
stdin, err := session.StdinPipe()
if err != nil {
return "", "", -1, log.E("ssh.Run", "stdin pipe", err)
}
go func() {
feat: infrastructure packages and lint cleanup (#281) * ci: consolidate duplicate workflows and merge CodeQL configs Remove 17 duplicate workflow files that were split copies of the combined originals. Each family (CI, CodeQL, Coverage, PR Build, Alpha Release) had the same job duplicated across separate push/pull_request/schedule/manual trigger files. Merge codeql.yml and codescan.yml into a single codeql.yml with a language matrix covering go, javascript-typescript, python, and actions — matching the previous default setup coverage. Remaining workflows (one per family): - ci.yml (push + PR + manual) - codeql.yml (push + PR + schedule, all languages) - coverage.yml (push + PR + manual) - alpha-release.yml (push + manual) - pr-build.yml (PR + manual) - release.yml (tag push) - agent-verify.yml, auto-label.yml, auto-project.yml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add collect, config, crypt, plugin packages and fix all lint issues Add four new infrastructure packages with CLI commands: - pkg/config: layered configuration (defaults → file → env → flags) - pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums) - pkg/plugin: plugin system with GitHub-based install/update/remove - pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate) Fix all golangci-lint issues across the entire codebase (~100 errcheck, staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that `core go qa` passes with 0 issues. Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:34:43 +00:00
defer func() { _ = stdin.Close() }()
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
_, _ = io.WriteString(stdin, c.password+"\n")
}()
} else {
// Try passwordless sudo
cmd = fmt.Sprintf("sudo -n -u %s bash -c '%s'", becomeUser, escapedCmd)
}
}
// Run with context
done := make(chan error, 1)
go func() {
done <- session.Run(cmd)
}()
select {
case <-ctx.Done():
_ = session.Signal(ssh.SIGKILL)
return "", "", -1, ctx.Err()
case err := <-done:
exitCode = 0
if err != nil {
if exitErr, ok := err.(*ssh.ExitError); ok {
exitCode = exitErr.ExitStatus()
} else {
return stdoutBuf.String(), stderrBuf.String(), -1, err
}
}
return stdoutBuf.String(), stderrBuf.String(), exitCode, nil
}
}
// RunScript runs a script on the remote host.
func (c *SSHClient) RunScript(ctx context.Context, script string) (stdout, stderr string, exitCode int, err error) {
// Escape the script for heredoc
cmd := fmt.Sprintf("bash <<'ANSIBLE_SCRIPT_EOF'\n%s\nANSIBLE_SCRIPT_EOF", script)
return c.Run(ctx, cmd)
}
// Upload copies a file to the remote host.
func (c *SSHClient) Upload(ctx context.Context, local io.Reader, remote string, mode os.FileMode) error {
if err := c.Connect(ctx); err != nil {
return err
}
// Read content
content, err := io.ReadAll(local)
if err != nil {
return log.E("ssh.Upload", "read content", err)
}
// Create parent directory
dir := filepath.Dir(remote)
dirCmd := fmt.Sprintf("mkdir -p %q", dir)
if c.become {
dirCmd = fmt.Sprintf("sudo mkdir -p %q", dir)
}
if _, _, _, err := c.Run(ctx, dirCmd); err != nil {
return log.E("ssh.Upload", "create parent dir", err)
}
// Use cat to write the file (simpler than SCP)
writeCmd := fmt.Sprintf("cat > %q && chmod %o %q", remote, mode, remote)
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
// If become is needed, we construct a command that reads password then content from stdin
// But we need to be careful with handling stdin for sudo + cat.
// We'll use a session with piped stdin.
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
session2, err := c.client.NewSession()
if err != nil {
return log.E("ssh.Upload", "new session for write", err)
}
defer func() { _ = session2.Close() }()
stdin, err := session2.StdinPipe()
if err != nil {
return log.E("ssh.Upload", "stdin pipe", err)
}
var stderrBuf bytes.Buffer
session2.Stderr = &stderrBuf
if c.become {
becomeUser := c.becomeUser
if becomeUser == "" {
becomeUser = "root"
}
pass := c.becomePass
if pass == "" {
pass = c.password
}
if pass != "" {
// Use sudo -S with password from stdin
writeCmd = fmt.Sprintf("sudo -S -u %s bash -c 'cat > %q && chmod %o %q'",
becomeUser, remote, mode, remote)
} else {
// Use passwordless sudo (sudo -n) to avoid consuming file content as password
writeCmd = fmt.Sprintf("sudo -n -u %s bash -c 'cat > %q && chmod %o %q'",
becomeUser, remote, mode, remote)
}
if err := session2.Start(writeCmd); err != nil {
return log.E("ssh.Upload", "start write", err)
}
go func() {
feat: infrastructure packages and lint cleanup (#281) * ci: consolidate duplicate workflows and merge CodeQL configs Remove 17 duplicate workflow files that were split copies of the combined originals. Each family (CI, CodeQL, Coverage, PR Build, Alpha Release) had the same job duplicated across separate push/pull_request/schedule/manual trigger files. Merge codeql.yml and codescan.yml into a single codeql.yml with a language matrix covering go, javascript-typescript, python, and actions — matching the previous default setup coverage. Remaining workflows (one per family): - ci.yml (push + PR + manual) - codeql.yml (push + PR + schedule, all languages) - coverage.yml (push + PR + manual) - alpha-release.yml (push + manual) - pr-build.yml (PR + manual) - release.yml (tag push) - agent-verify.yml, auto-label.yml, auto-project.yml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add collect, config, crypt, plugin packages and fix all lint issues Add four new infrastructure packages with CLI commands: - pkg/config: layered configuration (defaults → file → env → flags) - pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums) - pkg/plugin: plugin system with GitHub-based install/update/remove - pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate) Fix all golangci-lint issues across the entire codebase (~100 errcheck, staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that `core go qa` passes with 0 issues. Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:34:43 +00:00
defer func() { _ = stdin.Close() }()
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
if pass != "" {
_, _ = io.WriteString(stdin, pass+"\n")
}
_, _ = stdin.Write(content)
}()
} else {
// Normal write
if err := session2.Start(writeCmd); err != nil {
return log.E("ssh.Upload", "start write", err)
}
go func() {
feat: infrastructure packages and lint cleanup (#281) * ci: consolidate duplicate workflows and merge CodeQL configs Remove 17 duplicate workflow files that were split copies of the combined originals. Each family (CI, CodeQL, Coverage, PR Build, Alpha Release) had the same job duplicated across separate push/pull_request/schedule/manual trigger files. Merge codeql.yml and codescan.yml into a single codeql.yml with a language matrix covering go, javascript-typescript, python, and actions — matching the previous default setup coverage. Remaining workflows (one per family): - ci.yml (push + PR + manual) - codeql.yml (push + PR + schedule, all languages) - coverage.yml (push + PR + manual) - alpha-release.yml (push + manual) - pr-build.yml (PR + manual) - release.yml (tag push) - agent-verify.yml, auto-label.yml, auto-project.yml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add collect, config, crypt, plugin packages and fix all lint issues Add four new infrastructure packages with CLI commands: - pkg/config: layered configuration (defaults → file → env → flags) - pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums) - pkg/plugin: plugin system with GitHub-based install/update/remove - pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate) Fix all golangci-lint issues across the entire codebase (~100 errcheck, staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that `core go qa` passes with 0 issues. Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:34:43 +00:00
defer func() { _ = stdin.Close() }()
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
_, _ = stdin.Write(content)
}()
}
if err := session2.Wait(); err != nil {
return log.E("ssh.Upload", fmt.Sprintf("write failed (stderr: %s)", stderrBuf.String()), err)
}
return nil
}
// Download copies a file from the remote host.
func (c *SSHClient) Download(ctx context.Context, remote string) ([]byte, error) {
if err := c.Connect(ctx); err != nil {
return nil, err
}
cmd := fmt.Sprintf("cat %q", remote)
stdout, stderr, exitCode, err := c.Run(ctx, cmd)
if err != nil {
return nil, err
}
if exitCode != 0 {
return nil, log.E("ssh.Download", fmt.Sprintf("cat failed: %s", stderr), nil)
}
return []byte(stdout), nil
}
// FileExists checks if a file exists on the remote host.
func (c *SSHClient) FileExists(ctx context.Context, path string) (bool, error) {
cmd := fmt.Sprintf("test -e %q && echo yes || echo no", path)
stdout, _, exitCode, err := c.Run(ctx, cmd)
if err != nil {
return false, err
}
if exitCode != 0 {
// test command failed but didn't error - file doesn't exist
return false, nil
}
return strings.TrimSpace(stdout) == "yes", nil
}
// Stat returns file info from the remote host.
func (c *SSHClient) Stat(ctx context.Context, path string) (map[string]any, error) {
// Simple approach - get basic file info
cmd := fmt.Sprintf(`
if [ -e %q ]; then
if [ -d %q ]; then
echo "exists=true isdir=true"
else
echo "exists=true isdir=false"
fi
else
echo "exists=false"
fi
`, path, path)
stdout, _, _, err := c.Run(ctx, cmd)
if err != nil {
return nil, err
}
result := make(map[string]any)
parts := strings.Fields(strings.TrimSpace(stdout))
for _, part := range parts {
kv := strings.SplitN(part, "=", 2)
if len(kv) == 2 {
result[kv[0]] = kv[1] == "true"
}
}
return result, nil
}
// SetBecome enables privilege escalation.
func (c *SSHClient) SetBecome(become bool, user, password string) {
c.mu.Lock()
defer c.mu.Unlock()
c.become = become
if user != "" {
c.becomeUser = user
}
if password != "" {
c.becomePass = password
}
}