* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
438 lines
10 KiB
Go
438 lines
10 KiB
Go
package ansible
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
|
|
"github.com/host-uk/core/pkg/log"
|
|
"gopkg.in/yaml.v3"
|
|
)
|
|
|
|
// Parser handles Ansible YAML parsing.
|
|
type Parser struct {
|
|
basePath string
|
|
vars map[string]any
|
|
}
|
|
|
|
// NewParser creates a new Ansible parser.
|
|
func NewParser(basePath string) *Parser {
|
|
return &Parser{
|
|
basePath: basePath,
|
|
vars: make(map[string]any),
|
|
}
|
|
}
|
|
|
|
// ParsePlaybook parses an Ansible playbook file.
|
|
func (p *Parser) ParsePlaybook(path string) ([]Play, error) {
|
|
data, err := os.ReadFile(path)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("read playbook: %w", err)
|
|
}
|
|
|
|
var plays []Play
|
|
if err := yaml.Unmarshal(data, &plays); err != nil {
|
|
return nil, fmt.Errorf("parse playbook: %w", err)
|
|
}
|
|
|
|
// Process each play
|
|
for i := range plays {
|
|
if err := p.processPlay(&plays[i]); err != nil {
|
|
return nil, fmt.Errorf("process play %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
return plays, nil
|
|
}
|
|
|
|
// ParseInventory parses an Ansible inventory file.
|
|
func (p *Parser) ParseInventory(path string) (*Inventory, error) {
|
|
data, err := os.ReadFile(path)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("read inventory: %w", err)
|
|
}
|
|
|
|
var inv Inventory
|
|
if err := yaml.Unmarshal(data, &inv); err != nil {
|
|
return nil, fmt.Errorf("parse inventory: %w", err)
|
|
}
|
|
|
|
return &inv, nil
|
|
}
|
|
|
|
// ParseTasks parses a tasks file (used by include_tasks).
|
|
func (p *Parser) ParseTasks(path string) ([]Task, error) {
|
|
data, err := os.ReadFile(path)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("read tasks: %w", err)
|
|
}
|
|
|
|
var tasks []Task
|
|
if err := yaml.Unmarshal(data, &tasks); err != nil {
|
|
return nil, fmt.Errorf("parse tasks: %w", err)
|
|
}
|
|
|
|
for i := range tasks {
|
|
if err := p.extractModule(&tasks[i]); err != nil {
|
|
return nil, fmt.Errorf("task %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
return tasks, nil
|
|
}
|
|
|
|
// ParseRole parses a role and returns its tasks.
|
|
func (p *Parser) ParseRole(name string, tasksFrom string) ([]Task, error) {
|
|
if tasksFrom == "" {
|
|
tasksFrom = "main.yml"
|
|
}
|
|
|
|
// Search paths for roles (in order of precedence)
|
|
searchPaths := []string{
|
|
// Relative to playbook
|
|
filepath.Join(p.basePath, "roles", name, "tasks", tasksFrom),
|
|
// Parent directory roles
|
|
filepath.Join(filepath.Dir(p.basePath), "roles", name, "tasks", tasksFrom),
|
|
// Sibling roles directory
|
|
filepath.Join(p.basePath, "..", "roles", name, "tasks", tasksFrom),
|
|
// playbooks/roles pattern
|
|
filepath.Join(p.basePath, "playbooks", "roles", name, "tasks", tasksFrom),
|
|
// Common DevOps structure
|
|
filepath.Join(filepath.Dir(filepath.Dir(p.basePath)), "roles", name, "tasks", tasksFrom),
|
|
}
|
|
|
|
var tasksPath string
|
|
for _, sp := range searchPaths {
|
|
// Clean the path to resolve .. segments
|
|
sp = filepath.Clean(sp)
|
|
if _, err := os.Stat(sp); err == nil {
|
|
tasksPath = sp
|
|
break
|
|
}
|
|
}
|
|
|
|
if tasksPath == "" {
|
|
return nil, log.E("parser.ParseRole", fmt.Sprintf("role %s not found in search paths: %v", name, searchPaths), nil)
|
|
}
|
|
|
|
// Load role defaults
|
|
defaultsPath := filepath.Join(filepath.Dir(filepath.Dir(tasksPath)), "defaults", "main.yml")
|
|
if data, err := os.ReadFile(defaultsPath); err == nil {
|
|
var defaults map[string]any
|
|
if yaml.Unmarshal(data, &defaults) == nil {
|
|
for k, v := range defaults {
|
|
if _, exists := p.vars[k]; !exists {
|
|
p.vars[k] = v
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// Load role vars
|
|
varsPath := filepath.Join(filepath.Dir(filepath.Dir(tasksPath)), "vars", "main.yml")
|
|
if data, err := os.ReadFile(varsPath); err == nil {
|
|
var roleVars map[string]any
|
|
if yaml.Unmarshal(data, &roleVars) == nil {
|
|
for k, v := range roleVars {
|
|
p.vars[k] = v
|
|
}
|
|
}
|
|
}
|
|
|
|
return p.ParseTasks(tasksPath)
|
|
}
|
|
|
|
// processPlay processes a play and extracts modules from tasks.
|
|
func (p *Parser) processPlay(play *Play) error {
|
|
// Merge play vars
|
|
for k, v := range play.Vars {
|
|
p.vars[k] = v
|
|
}
|
|
|
|
for i := range play.PreTasks {
|
|
if err := p.extractModule(&play.PreTasks[i]); err != nil {
|
|
return fmt.Errorf("pre_task %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
for i := range play.Tasks {
|
|
if err := p.extractModule(&play.Tasks[i]); err != nil {
|
|
return fmt.Errorf("task %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
for i := range play.PostTasks {
|
|
if err := p.extractModule(&play.PostTasks[i]); err != nil {
|
|
return fmt.Errorf("post_task %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
for i := range play.Handlers {
|
|
if err := p.extractModule(&play.Handlers[i]); err != nil {
|
|
return fmt.Errorf("handler %d: %w", i, err)
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// extractModule extracts the module name and args from a task.
|
|
func (p *Parser) extractModule(task *Task) error {
|
|
// First, unmarshal the raw YAML to get all keys
|
|
// This is a workaround since we need to find the module key dynamically
|
|
|
|
// Handle block tasks
|
|
for i := range task.Block {
|
|
if err := p.extractModule(&task.Block[i]); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
for i := range task.Rescue {
|
|
if err := p.extractModule(&task.Rescue[i]); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
for i := range task.Always {
|
|
if err := p.extractModule(&task.Always[i]); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// UnmarshalYAML implements custom YAML unmarshaling for Task.
|
|
func (t *Task) UnmarshalYAML(node *yaml.Node) error {
|
|
// First decode known fields
|
|
type rawTask Task
|
|
var raw rawTask
|
|
|
|
// Create a map to capture all fields
|
|
var m map[string]any
|
|
if err := node.Decode(&m); err != nil {
|
|
return err
|
|
}
|
|
|
|
// Decode into struct
|
|
if err := node.Decode(&raw); err != nil {
|
|
return err
|
|
}
|
|
*t = Task(raw)
|
|
t.raw = m
|
|
|
|
// Find the module key
|
|
knownKeys := map[string]bool{
|
|
"name": true, "register": true, "when": true, "loop": true,
|
|
"loop_control": true, "vars": true, "environment": true,
|
|
"changed_when": true, "failed_when": true, "ignore_errors": true,
|
|
"no_log": true, "become": true, "become_user": true,
|
|
"delegate_to": true, "run_once": true, "tags": true,
|
|
"block": true, "rescue": true, "always": true, "notify": true,
|
|
"retries": true, "delay": true, "until": true,
|
|
"include_tasks": true, "import_tasks": true,
|
|
"include_role": true, "import_role": true,
|
|
"with_items": true, "with_dict": true, "with_file": true,
|
|
}
|
|
|
|
for key, val := range m {
|
|
if knownKeys[key] {
|
|
continue
|
|
}
|
|
|
|
// Check if this is a module
|
|
if isModule(key) {
|
|
t.Module = key
|
|
t.Args = make(map[string]any)
|
|
|
|
switch v := val.(type) {
|
|
case string:
|
|
// Free-form args (e.g., shell: echo hello)
|
|
t.Args["_raw_params"] = v
|
|
case map[string]any:
|
|
t.Args = v
|
|
case nil:
|
|
// Module with no args
|
|
default:
|
|
t.Args["_raw_params"] = v
|
|
}
|
|
break
|
|
}
|
|
}
|
|
|
|
// Handle with_items as loop
|
|
if items, ok := m["with_items"]; ok && t.Loop == nil {
|
|
t.Loop = items
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// isModule checks if a key is a known module.
|
|
func isModule(key string) bool {
|
|
for _, m := range KnownModules {
|
|
if key == m {
|
|
return true
|
|
}
|
|
// Also check without ansible.builtin. prefix
|
|
if strings.HasPrefix(m, "ansible.builtin.") {
|
|
if key == strings.TrimPrefix(m, "ansible.builtin.") {
|
|
return true
|
|
}
|
|
}
|
|
}
|
|
// Accept any key with dots (likely a module)
|
|
return strings.Contains(key, ".")
|
|
}
|
|
|
|
// NormalizeModule normalizes a module name to its canonical form.
|
|
func NormalizeModule(name string) string {
|
|
// Add ansible.builtin. prefix if missing
|
|
if !strings.Contains(name, ".") {
|
|
return "ansible.builtin." + name
|
|
}
|
|
return name
|
|
}
|
|
|
|
// GetHosts returns hosts matching a pattern from inventory.
|
|
func GetHosts(inv *Inventory, pattern string) []string {
|
|
if pattern == "all" {
|
|
return getAllHosts(inv.All)
|
|
}
|
|
if pattern == "localhost" {
|
|
return []string{"localhost"}
|
|
}
|
|
|
|
// Check if it's a group name
|
|
hosts := getGroupHosts(inv.All, pattern)
|
|
if len(hosts) > 0 {
|
|
return hosts
|
|
}
|
|
|
|
// Check if it's a specific host
|
|
if hasHost(inv.All, pattern) {
|
|
return []string{pattern}
|
|
}
|
|
|
|
// Handle patterns with : (intersection/union)
|
|
// For now, just return empty
|
|
return nil
|
|
}
|
|
|
|
func getAllHosts(group *InventoryGroup) []string {
|
|
if group == nil {
|
|
return nil
|
|
}
|
|
|
|
var hosts []string
|
|
for name := range group.Hosts {
|
|
hosts = append(hosts, name)
|
|
}
|
|
for _, child := range group.Children {
|
|
hosts = append(hosts, getAllHosts(child)...)
|
|
}
|
|
return hosts
|
|
}
|
|
|
|
func getGroupHosts(group *InventoryGroup, name string) []string {
|
|
if group == nil {
|
|
return nil
|
|
}
|
|
|
|
// Check children for the group name
|
|
if child, ok := group.Children[name]; ok {
|
|
return getAllHosts(child)
|
|
}
|
|
|
|
// Recurse
|
|
for _, child := range group.Children {
|
|
if hosts := getGroupHosts(child, name); len(hosts) > 0 {
|
|
return hosts
|
|
}
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func hasHost(group *InventoryGroup, name string) bool {
|
|
if group == nil {
|
|
return false
|
|
}
|
|
|
|
if _, ok := group.Hosts[name]; ok {
|
|
return true
|
|
}
|
|
|
|
for _, child := range group.Children {
|
|
if hasHost(child, name) {
|
|
return true
|
|
}
|
|
}
|
|
|
|
return false
|
|
}
|
|
|
|
// GetHostVars returns variables for a specific host.
|
|
func GetHostVars(inv *Inventory, hostname string) map[string]any {
|
|
vars := make(map[string]any)
|
|
|
|
// Collect vars from all levels
|
|
collectHostVars(inv.All, hostname, vars)
|
|
|
|
return vars
|
|
}
|
|
|
|
func collectHostVars(group *InventoryGroup, hostname string, vars map[string]any) bool {
|
|
if group == nil {
|
|
return false
|
|
}
|
|
|
|
// Check if host is in this group
|
|
found := false
|
|
if host, ok := group.Hosts[hostname]; ok {
|
|
found = true
|
|
// Apply group vars first
|
|
for k, v := range group.Vars {
|
|
vars[k] = v
|
|
}
|
|
// Then host vars
|
|
if host != nil {
|
|
if host.AnsibleHost != "" {
|
|
vars["ansible_host"] = host.AnsibleHost
|
|
}
|
|
if host.AnsiblePort != 0 {
|
|
vars["ansible_port"] = host.AnsiblePort
|
|
}
|
|
if host.AnsibleUser != "" {
|
|
vars["ansible_user"] = host.AnsibleUser
|
|
}
|
|
if host.AnsiblePassword != "" {
|
|
vars["ansible_password"] = host.AnsiblePassword
|
|
}
|
|
if host.AnsibleSSHPrivateKeyFile != "" {
|
|
vars["ansible_ssh_private_key_file"] = host.AnsibleSSHPrivateKeyFile
|
|
}
|
|
if host.AnsibleConnection != "" {
|
|
vars["ansible_connection"] = host.AnsibleConnection
|
|
}
|
|
for k, v := range host.Vars {
|
|
vars[k] = v
|
|
}
|
|
}
|
|
}
|
|
|
|
// Check children
|
|
for _, child := range group.Children {
|
|
if collectHostVars(child, hostname, vars) {
|
|
// Apply this group's vars (parent vars)
|
|
for k, v := range group.Vars {
|
|
if _, exists := vars[k]; !exists {
|
|
vars[k] = v
|
|
}
|
|
}
|
|
found = true
|
|
}
|
|
}
|
|
|
|
return found
|
|
}
|