cli/internal/cmd/deploy/cmd_ansible.go

313 lines
8.7 KiB
Go
Raw Normal View History

feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
package deploy
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"forge.lthn.ai/core/go/pkg/ansible"
"forge.lthn.ai/core/go/pkg/cli"
feat: wire release command, add tar.xz support, unified installers (#277) * feat(cli): wire release command and add installer scripts - Wire up `core build release` subcommand (was orphaned) - Wire up `core monitor` command (missing import in full variant) - Add installer scripts for Unix (.sh) and Windows (.bat) - setup: Interactive with variant selection - ci: Minimal for CI/CD environments - dev: Full development variant - go/php/agent: Targeted development variants - All scripts include security hardening: - Secure temp directories (mktemp -d) - Architecture validation - Version validation after GitHub API call - Proper cleanup on exit - PowerShell PATH updates on Windows (avoids setx truncation) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(build): add tar.xz support and unified installer scripts - Add tar.xz archive support using Borg's compress package - ArchiveXZ() and ArchiveWithFormat() for configurable compression - Better compression ratio than gzip for release artifacts - Consolidate 12 installer scripts into 2 unified scripts - install.sh and install.bat with BunnyCDN edge variable support - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc. - MODE and VARIANT transformed at edge based on subdomain - Installers prefer tar.xz with automatic fallback to tar.gz - Fixed CodeRabbit issues: HTTP status patterns, tar error handling, verify_install params, VARIANT validation, CI PATH persistence Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: add build and release config files - .core/build.yaml - cross-platform build configuration - .core/release.yaml - release workflow configuration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: move plans from docs/ to tasks/ Consolidate planning documents in tasks/plans/ directory. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(install): address CodeRabbit review feedback - Add curl timeout (--max-time) to prevent hanging on slow networks - Rename TMPDIR to WORK_DIR to avoid clobbering system env var - Add chmod +x to ensure binary has execute permissions - Add error propagation after subroutine calls in batch file - Remove System32 install attempt in CI mode (use consistent INSTALL_DIR) - Fix HTTP status regex for HTTP/2 compatibility Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(rag): add Go RAG implementation with Qdrant + Ollama Add RAG (Retrieval Augmented Generation) tools for storing documentation in Qdrant vector database and querying with semantic search. This replaces the Python tools/rag implementation with a native Go solution. New commands: - core rag ingest [directory] - Ingest markdown files into Qdrant - core rag query [question] - Query vector database with semantic search - core rag collections - List and manage Qdrant collections Features: - Markdown chunking by sections and paragraphs with overlap - UTF-8 safe text handling for international content - Automatic category detection from file paths - Multiple output formats: text, JSON, LLM context injection - Environment variable support for host configuration Dependencies: - github.com/qdrant/go-client (gRPC client) - github.com/ollama/ollama/api (embeddings API) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(deploy): add pure-Go Ansible executor and Coolify API integration Implement infrastructure deployment system with: - pkg/ansible: Pure Go Ansible executor - Playbook/inventory parsing (types.go, parser.go) - Full execution engine with variable templating, loops, blocks, conditionals, handlers, and fact gathering (executor.go) - SSH client with key/password auth and privilege escalation (ssh.go) - 35+ module implementations: shell, command, copy, template, file, apt, service, systemd, user, group, git, docker_compose, etc. (modules.go) - pkg/deploy/coolify: Coolify API client wrapping Python swagger client - List/get servers, projects, applications, databases, services - Generic Call() for any OpenAPI operation - pkg/deploy/python: Embedded Python runtime for swagger client integration - internal/cmd/deploy: CLI commands - core deploy servers/projects/apps/databases/services/team - core deploy call <operation> [params-json] This enables Docker-free infrastructure deployment with Ansible-compatible playbooks executed natively in Go. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): address linter warnings and build errors - Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command) - Fix errcheck warnings by explicitly ignoring best-effort operations - Fix ineffassign warning in cmd_ansible.go All golangci-lint checks now pass for deploy packages. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style(deploy): fix gofmt formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deploy): use known_hosts for SSH host key verification Address CodeQL security alert by using the user's known_hosts file for SSH host key verification when available. Falls back to accepting any key only when known_hosts doesn't exist (common in containerized or ephemeral environments). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app Wire up AI infrastructure with unified pkg/ai package (metrics JSONL, RAG integration), move RAG under `core ai rag`, add `core ai metrics` command, and enrich task context with Qdrant documentation. Add `--target` flag to all security commands for external repo scanning, `core security jobs` for distributing findings as GitHub Issues, and consistent error logging across scan/deps/alerts/secrets commands. Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge (loopback-only HTTP server), WebSocket hub, and Claude Code bridge. Production-ready with Lethean CIC branding, macOS code signing support, and security hardening (origin validation, body size limits, URL scheme checks, memory leak prevention, XSS mitigation). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: address PR review comments from CodeRabbit, Copilot, and Gemini Fixes across 25 files addressing 46+ review comments: - pkg/ai/metrics.go: handle error from Close() on writable file handle - pkg/ansible: restore loop vars after loop, restore become settings, fix Upload with become=true and no password (use sudo -n), honour SSH timeout config, use E() helper for contextual errors, quote git refs in checkout commands - pkg/rag: validate chunk config, guard negative-to-uint64 conversion, use E() helper for errors, add context timeout to Ollama HTTP calls - pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError), handle os.UserHomeDir() error - pkg/build/buildcmd: use cmd.Context() instead of context.Background() for proper Ctrl+C cancellation - install.bat: add curl timeouts, CRLF line endings, use --connect-timeout for archive downloads - install.sh: use absolute path for version check in CI mode - tools/rag: fix broken ingest.py function def, escape HTML in query.py, pin qdrant-client version, add markdown code block languages - internal/cmd/rag: add chunk size validation, env override handling Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(build): make release dry-run by default and remove darwin/amd64 target Replace --dry-run (default false) with --we-are-go-for-launch (default false) so `core build release` is safe by default. Remove darwin/amd64 from default build targets (arm64 only for macOS). Fix cmd_project.go to use command context instead of context.Background(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
"github.com/spf13/cobra"
)
var (
ansibleInventory string
ansibleLimit string
ansibleTags string
ansibleSkipTags string
ansibleVars []string
ansibleVerbose int
ansibleCheck bool
)
var ansibleCmd = &cobra.Command{
Use: "ansible <playbook>",
Short: "Run Ansible playbooks natively (no Python required)",
Long: `Execute Ansible playbooks using a pure Go implementation.
This command parses Ansible YAML playbooks and executes them natively,
without requiring Python or ansible-playbook to be installed.
Supported modules:
- shell, command, raw, script
- copy, template, file, lineinfile, stat, slurp, fetch, get_url
- apt, apt_key, apt_repository, package, pip
- service, systemd
- user, group
- uri, wait_for, git, unarchive
- debug, fail, assert, set_fact, pause
Examples:
core deploy ansible playbooks/coolify/create.yml -i inventory/
core deploy ansible site.yml -l production
core deploy ansible deploy.yml -e "version=1.2.3" -e "env=prod"`,
Args: cobra.ExactArgs(1),
RunE: runAnsible,
}
var ansibleTestCmd = &cobra.Command{
Use: "test <host>",
Short: "Test SSH connectivity to a host",
Long: `Test SSH connection and gather facts from a host.
Examples:
core deploy ansible test linux.snider.dev -u claude -p claude
core deploy ansible test server.example.com -i ~/.ssh/id_rsa`,
Args: cobra.ExactArgs(1),
RunE: runAnsibleTest,
}
var (
testUser string
testPassword string
testKeyFile string
testPort int
)
func init() {
// ansible command flags
ansibleCmd.Flags().StringVarP(&ansibleInventory, "inventory", "i", "", "Inventory file or directory")
ansibleCmd.Flags().StringVarP(&ansibleLimit, "limit", "l", "", "Limit to specific hosts")
ansibleCmd.Flags().StringVarP(&ansibleTags, "tags", "t", "", "Only run plays and tasks tagged with these values")
ansibleCmd.Flags().StringVar(&ansibleSkipTags, "skip-tags", "", "Skip plays and tasks tagged with these values")
ansibleCmd.Flags().StringArrayVarP(&ansibleVars, "extra-vars", "e", nil, "Set additional variables (key=value)")
ansibleCmd.Flags().CountVarP(&ansibleVerbose, "verbose", "v", "Increase verbosity")
ansibleCmd.Flags().BoolVar(&ansibleCheck, "check", false, "Don't make any changes (dry run)")
// test command flags
ansibleTestCmd.Flags().StringVarP(&testUser, "user", "u", "root", "SSH user")
ansibleTestCmd.Flags().StringVarP(&testPassword, "password", "p", "", "SSH password")
ansibleTestCmd.Flags().StringVarP(&testKeyFile, "key", "i", "", "SSH private key file")
ansibleTestCmd.Flags().IntVar(&testPort, "port", 22, "SSH port")
// Add subcommands
ansibleCmd.AddCommand(ansibleTestCmd)
Cmd.AddCommand(ansibleCmd)
}
func runAnsible(cmd *cobra.Command, args []string) error {
playbookPath := args[0]
// Resolve playbook path
if !filepath.IsAbs(playbookPath) {
cwd, _ := os.Getwd()
playbookPath = filepath.Join(cwd, playbookPath)
}
if _, err := os.Stat(playbookPath); os.IsNotExist(err) {
return fmt.Errorf("playbook not found: %s", playbookPath)
}
// Create executor
basePath := filepath.Dir(playbookPath)
executor := ansible.NewExecutor(basePath)
defer executor.Close()
// Set options
executor.Limit = ansibleLimit
executor.CheckMode = ansibleCheck
executor.Verbose = ansibleVerbose
if ansibleTags != "" {
executor.Tags = strings.Split(ansibleTags, ",")
}
if ansibleSkipTags != "" {
executor.SkipTags = strings.Split(ansibleSkipTags, ",")
}
// Parse extra vars
for _, v := range ansibleVars {
parts := strings.SplitN(v, "=", 2)
if len(parts) == 2 {
executor.SetVar(parts[0], parts[1])
}
}
// Load inventory
if ansibleInventory != "" {
invPath := ansibleInventory
if !filepath.IsAbs(invPath) {
cwd, _ := os.Getwd()
invPath = filepath.Join(cwd, invPath)
}
// Check if it's a directory
info, err := os.Stat(invPath)
if err != nil {
return fmt.Errorf("inventory not found: %s", invPath)
}
if info.IsDir() {
// Look for inventory.yml or hosts.yml
for _, name := range []string{"inventory.yml", "hosts.yml", "inventory.yaml", "hosts.yaml"} {
p := filepath.Join(invPath, name)
if _, err := os.Stat(p); err == nil {
invPath = p
break
}
}
}
if err := executor.SetInventory(invPath); err != nil {
return fmt.Errorf("load inventory: %w", err)
}
}
// Set up callbacks
executor.OnPlayStart = func(play *ansible.Play) {
fmt.Printf("\n%s %s\n", cli.TitleStyle.Render("PLAY"), cli.BoldStyle.Render("["+play.Name+"]"))
fmt.Println(strings.Repeat("*", 70))
}
executor.OnTaskStart = func(host string, task *ansible.Task) {
taskName := task.Name
if taskName == "" {
taskName = task.Module
}
fmt.Printf("\n%s %s\n", cli.TitleStyle.Render("TASK"), cli.BoldStyle.Render("["+taskName+"]"))
if ansibleVerbose > 0 {
fmt.Printf("%s\n", cli.DimStyle.Render("host: "+host))
}
}
executor.OnTaskEnd = func(host string, task *ansible.Task, result *ansible.TaskResult) {
status := "ok"
style := cli.SuccessStyle
if result.Failed {
status = "failed"
style = cli.ErrorStyle
} else if result.Skipped {
status = "skipping"
style = cli.DimStyle
} else if result.Changed {
status = "changed"
style = cli.WarningStyle
}
fmt.Printf("%s: [%s]", style.Render(status), host)
if result.Msg != "" && ansibleVerbose > 0 {
fmt.Printf(" => %s", result.Msg)
}
if result.Duration > 0 && ansibleVerbose > 1 {
fmt.Printf(" (%s)", result.Duration.Round(time.Millisecond))
}
fmt.Println()
if result.Failed && result.Stderr != "" {
fmt.Printf("%s\n", cli.ErrorStyle.Render(result.Stderr))
}
if ansibleVerbose > 1 {
if result.Stdout != "" {
fmt.Printf("stdout: %s\n", strings.TrimSpace(result.Stdout))
}
}
}
executor.OnPlayEnd = func(play *ansible.Play) {
fmt.Println()
}
// Run playbook
ctx := context.Background()
start := time.Now()
fmt.Printf("%s Running playbook: %s\n", cli.BoldStyle.Render("▶"), playbookPath)
if err := executor.Run(ctx, playbookPath); err != nil {
return fmt.Errorf("playbook failed: %w", err)
}
fmt.Printf("\n%s Playbook completed in %s\n",
cli.SuccessStyle.Render("✓"),
time.Since(start).Round(time.Millisecond))
return nil
}
func runAnsibleTest(cmd *cobra.Command, args []string) error {
host := args[0]
fmt.Printf("Testing SSH connection to %s...\n", cli.BoldStyle.Render(host))
cfg := ansible.SSHConfig{
Host: host,
Port: testPort,
User: testUser,
Password: testPassword,
KeyFile: testKeyFile,
Timeout: 30 * time.Second,
}
client, err := ansible.NewSSHClient(cfg)
if err != nil {
return fmt.Errorf("create client: %w", err)
}
defer func() { _ = client.Close() }()
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
// Test connection
start := time.Now()
if err := client.Connect(ctx); err != nil {
return fmt.Errorf("connect failed: %w", err)
}
connectTime := time.Since(start)
fmt.Printf("%s Connected in %s\n", cli.SuccessStyle.Render("✓"), connectTime.Round(time.Millisecond))
// Gather facts
fmt.Println("\nGathering facts...")
// Hostname
stdout, _, _, _ := client.Run(ctx, "hostname -f 2>/dev/null || hostname")
fmt.Printf(" Hostname: %s\n", cli.BoldStyle.Render(strings.TrimSpace(stdout)))
// OS
stdout, _, _, _ = client.Run(ctx, "cat /etc/os-release 2>/dev/null | grep PRETTY_NAME | cut -d'\"' -f2")
if stdout != "" {
fmt.Printf(" OS: %s\n", strings.TrimSpace(stdout))
}
// Kernel
stdout, _, _, _ = client.Run(ctx, "uname -r")
fmt.Printf(" Kernel: %s\n", strings.TrimSpace(stdout))
// Architecture
stdout, _, _, _ = client.Run(ctx, "uname -m")
fmt.Printf(" Architecture: %s\n", strings.TrimSpace(stdout))
// Memory
stdout, _, _, _ = client.Run(ctx, "free -h | grep Mem | awk '{print $2}'")
fmt.Printf(" Memory: %s\n", strings.TrimSpace(stdout))
// Disk
stdout, _, _, _ = client.Run(ctx, "df -h / | tail -1 | awk '{print $2 \" total, \" $4 \" available\"}'")
fmt.Printf(" Disk: %s\n", strings.TrimSpace(stdout))
// Docker
stdout, _, _, err = client.Run(ctx, "docker --version 2>/dev/null")
if err == nil {
fmt.Printf(" Docker: %s\n", cli.SuccessStyle.Render(strings.TrimSpace(stdout)))
} else {
fmt.Printf(" Docker: %s\n", cli.DimStyle.Render("not installed"))
}
// Check if Coolify is running
stdout, _, _, _ = client.Run(ctx, "docker ps 2>/dev/null | grep -q coolify && echo 'running' || echo 'not running'")
if strings.TrimSpace(stdout) == "running" {
fmt.Printf(" Coolify: %s\n", cli.SuccessStyle.Render("running"))
} else {
fmt.Printf(" Coolify: %s\n", cli.DimStyle.Render("not installed"))
}
fmt.Printf("\n%s SSH test passed\n", cli.SuccessStyle.Render("✓"))
return nil
}