feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
|
|
|
package ansible
|
|
|
|
|
|
|
|
|
|
import (
|
|
|
|
|
"bytes"
|
|
|
|
|
"context"
|
|
|
|
|
"fmt"
|
|
|
|
|
"io"
|
|
|
|
|
"net"
|
|
|
|
|
"os"
|
|
|
|
|
"path/filepath"
|
|
|
|
|
"strings"
|
|
|
|
|
"sync"
|
|
|
|
|
"time"
|
|
|
|
|
|
|
|
|
|
"github.com/host-uk/core/pkg/log"
|
|
|
|
|
"golang.org/x/crypto/ssh"
|
|
|
|
|
"golang.org/x/crypto/ssh/knownhosts"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
// SSHClient handles SSH connections to remote hosts.
|
|
|
|
|
type SSHClient struct {
|
|
|
|
|
host string
|
|
|
|
|
port int
|
|
|
|
|
user string
|
|
|
|
|
password string
|
|
|
|
|
keyFile string
|
|
|
|
|
client *ssh.Client
|
|
|
|
|
mu sync.Mutex
|
|
|
|
|
become bool
|
|
|
|
|
becomeUser string
|
|
|
|
|
becomePass string
|
|
|
|
|
timeout time.Duration
|
|
|
|
|
insecure bool
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// SSHConfig holds SSH connection configuration.
|
|
|
|
|
type SSHConfig struct {
|
|
|
|
|
Host string
|
|
|
|
|
Port int
|
|
|
|
|
User string
|
|
|
|
|
Password string
|
|
|
|
|
KeyFile string
|
|
|
|
|
Become bool
|
|
|
|
|
BecomeUser string
|
|
|
|
|
BecomePass string
|
|
|
|
|
Timeout time.Duration
|
|
|
|
|
Insecure bool
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// NewSSHClient creates a new SSH client.
|
|
|
|
|
func NewSSHClient(cfg SSHConfig) (*SSHClient, error) {
|
|
|
|
|
if cfg.Port == 0 {
|
|
|
|
|
cfg.Port = 22
|
|
|
|
|
}
|
|
|
|
|
if cfg.User == "" {
|
|
|
|
|
cfg.User = "root"
|
|
|
|
|
}
|
|
|
|
|
if cfg.Timeout == 0 {
|
|
|
|
|
cfg.Timeout = 30 * time.Second
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
client := &SSHClient{
|
|
|
|
|
host: cfg.Host,
|
|
|
|
|
port: cfg.Port,
|
|
|
|
|
user: cfg.User,
|
|
|
|
|
password: cfg.Password,
|
|
|
|
|
keyFile: cfg.KeyFile,
|
|
|
|
|
become: cfg.Become,
|
|
|
|
|
becomeUser: cfg.BecomeUser,
|
|
|
|
|
becomePass: cfg.BecomePass,
|
|
|
|
|
timeout: cfg.Timeout,
|
|
|
|
|
insecure: cfg.Insecure,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return client, nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Connect establishes the SSH connection.
|
|
|
|
|
func (c *SSHClient) Connect(ctx context.Context) error {
|
|
|
|
|
c.mu.Lock()
|
|
|
|
|
defer c.mu.Unlock()
|
|
|
|
|
|
|
|
|
|
if c.client != nil {
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
var authMethods []ssh.AuthMethod
|
|
|
|
|
|
|
|
|
|
// Try key-based auth first
|
|
|
|
|
if c.keyFile != "" {
|
|
|
|
|
keyPath := c.keyFile
|
|
|
|
|
if strings.HasPrefix(keyPath, "~") {
|
|
|
|
|
home, _ := os.UserHomeDir()
|
|
|
|
|
keyPath = filepath.Join(home, keyPath[1:])
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if key, err := os.ReadFile(keyPath); err == nil {
|
|
|
|
|
if signer, err := ssh.ParsePrivateKey(key); err == nil {
|
|
|
|
|
authMethods = append(authMethods, ssh.PublicKeys(signer))
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Try default SSH keys
|
|
|
|
|
if len(authMethods) == 0 {
|
|
|
|
|
home, _ := os.UserHomeDir()
|
|
|
|
|
defaultKeys := []string{
|
|
|
|
|
filepath.Join(home, ".ssh", "id_ed25519"),
|
|
|
|
|
filepath.Join(home, ".ssh", "id_rsa"),
|
|
|
|
|
}
|
|
|
|
|
for _, keyPath := range defaultKeys {
|
|
|
|
|
if key, err := os.ReadFile(keyPath); err == nil {
|
|
|
|
|
if signer, err := ssh.ParsePrivateKey(key); err == nil {
|
|
|
|
|
authMethods = append(authMethods, ssh.PublicKeys(signer))
|
|
|
|
|
break
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Fall back to password auth
|
|
|
|
|
if c.password != "" {
|
|
|
|
|
authMethods = append(authMethods, ssh.Password(c.password))
|
|
|
|
|
authMethods = append(authMethods, ssh.KeyboardInteractive(func(user, instruction string, questions []string, echos []bool) ([]string, error) {
|
|
|
|
|
answers := make([]string, len(questions))
|
|
|
|
|
for i := range questions {
|
|
|
|
|
answers[i] = c.password
|
|
|
|
|
}
|
|
|
|
|
return answers, nil
|
|
|
|
|
}))
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if len(authMethods) == 0 {
|
|
|
|
|
return log.E("ssh.Connect", "no authentication method available", nil)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Host key verification
|
|
|
|
|
var hostKeyCallback ssh.HostKeyCallback
|
|
|
|
|
|
|
|
|
|
if c.insecure {
|
|
|
|
|
hostKeyCallback = ssh.InsecureIgnoreHostKey()
|
|
|
|
|
} else {
|
|
|
|
|
home, err := os.UserHomeDir()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Connect", "failed to get user home dir", err)
|
|
|
|
|
}
|
|
|
|
|
knownHostsPath := filepath.Join(home, ".ssh", "known_hosts")
|
|
|
|
|
|
|
|
|
|
cb, err := knownhosts.New(knownHostsPath)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Connect", "failed to load known_hosts (use Insecure=true to bypass)", err)
|
|
|
|
|
}
|
|
|
|
|
hostKeyCallback = cb
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
config := &ssh.ClientConfig{
|
|
|
|
|
User: c.user,
|
|
|
|
|
Auth: authMethods,
|
|
|
|
|
HostKeyCallback: hostKeyCallback,
|
|
|
|
|
Timeout: c.timeout,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
addr := fmt.Sprintf("%s:%d", c.host, c.port)
|
|
|
|
|
|
|
|
|
|
// Connect with context timeout
|
|
|
|
|
var d net.Dialer
|
|
|
|
|
conn, err := d.DialContext(ctx, "tcp", addr)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Connect", fmt.Sprintf("dial %s", addr), err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
sshConn, chans, reqs, err := ssh.NewClientConn(conn, addr, config)
|
|
|
|
|
if err != nil {
|
|
|
|
|
// conn is closed by NewClientConn on error
|
|
|
|
|
return log.E("ssh.Connect", fmt.Sprintf("ssh connect %s", addr), err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
c.client = ssh.NewClient(sshConn, chans, reqs)
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Close closes the SSH connection.
|
|
|
|
|
func (c *SSHClient) Close() error {
|
|
|
|
|
c.mu.Lock()
|
|
|
|
|
defer c.mu.Unlock()
|
|
|
|
|
|
|
|
|
|
if c.client != nil {
|
|
|
|
|
err := c.client.Close()
|
|
|
|
|
c.client = nil
|
|
|
|
|
return err
|
|
|
|
|
}
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Run executes a command on the remote host.
|
|
|
|
|
func (c *SSHClient) Run(ctx context.Context, cmd string) (stdout, stderr string, exitCode int, err error) {
|
|
|
|
|
if err := c.Connect(ctx); err != nil {
|
|
|
|
|
return "", "", -1, err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
session, err := c.client.NewSession()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return "", "", -1, log.E("ssh.Run", "new session", err)
|
|
|
|
|
}
|
|
|
|
|
defer func() { _ = session.Close() }()
|
|
|
|
|
|
|
|
|
|
var stdoutBuf, stderrBuf bytes.Buffer
|
|
|
|
|
session.Stdout = &stdoutBuf
|
|
|
|
|
session.Stderr = &stderrBuf
|
|
|
|
|
|
|
|
|
|
// Apply become if needed
|
|
|
|
|
if c.become {
|
|
|
|
|
becomeUser := c.becomeUser
|
|
|
|
|
if becomeUser == "" {
|
|
|
|
|
becomeUser = "root"
|
|
|
|
|
}
|
|
|
|
|
// Escape single quotes in the command
|
|
|
|
|
escapedCmd := strings.ReplaceAll(cmd, "'", "'\\''")
|
|
|
|
|
if c.becomePass != "" {
|
|
|
|
|
// Use sudo with password via stdin (-S flag)
|
|
|
|
|
// We launch a goroutine to write the password to stdin
|
|
|
|
|
cmd = fmt.Sprintf("sudo -S -u %s bash -c '%s'", becomeUser, escapedCmd)
|
|
|
|
|
stdin, err := session.StdinPipe()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return "", "", -1, log.E("ssh.Run", "stdin pipe", err)
|
|
|
|
|
}
|
|
|
|
|
go func() {
|
|
|
|
|
defer stdin.Close()
|
|
|
|
|
_, _ = io.WriteString(stdin, c.becomePass+"\n")
|
|
|
|
|
}()
|
|
|
|
|
} else if c.password != "" {
|
|
|
|
|
// Try using connection password for sudo
|
|
|
|
|
cmd = fmt.Sprintf("sudo -S -u %s bash -c '%s'", becomeUser, escapedCmd)
|
|
|
|
|
stdin, err := session.StdinPipe()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return "", "", -1, log.E("ssh.Run", "stdin pipe", err)
|
|
|
|
|
}
|
|
|
|
|
go func() {
|
|
|
|
|
defer stdin.Close()
|
|
|
|
|
_, _ = io.WriteString(stdin, c.password+"\n")
|
|
|
|
|
}()
|
|
|
|
|
} else {
|
|
|
|
|
// Try passwordless sudo
|
|
|
|
|
cmd = fmt.Sprintf("sudo -n -u %s bash -c '%s'", becomeUser, escapedCmd)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Run with context
|
|
|
|
|
done := make(chan error, 1)
|
|
|
|
|
go func() {
|
|
|
|
|
done <- session.Run(cmd)
|
|
|
|
|
}()
|
|
|
|
|
|
|
|
|
|
select {
|
|
|
|
|
case <-ctx.Done():
|
|
|
|
|
_ = session.Signal(ssh.SIGKILL)
|
|
|
|
|
return "", "", -1, ctx.Err()
|
|
|
|
|
case err := <-done:
|
|
|
|
|
exitCode = 0
|
|
|
|
|
if err != nil {
|
|
|
|
|
if exitErr, ok := err.(*ssh.ExitError); ok {
|
|
|
|
|
exitCode = exitErr.ExitStatus()
|
|
|
|
|
} else {
|
|
|
|
|
return stdoutBuf.String(), stderrBuf.String(), -1, err
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return stdoutBuf.String(), stderrBuf.String(), exitCode, nil
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// RunScript runs a script on the remote host.
|
|
|
|
|
func (c *SSHClient) RunScript(ctx context.Context, script string) (stdout, stderr string, exitCode int, err error) {
|
|
|
|
|
// Escape the script for heredoc
|
|
|
|
|
cmd := fmt.Sprintf("bash <<'ANSIBLE_SCRIPT_EOF'\n%s\nANSIBLE_SCRIPT_EOF", script)
|
|
|
|
|
return c.Run(ctx, cmd)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Upload copies a file to the remote host.
|
|
|
|
|
func (c *SSHClient) Upload(ctx context.Context, local io.Reader, remote string, mode os.FileMode) error {
|
|
|
|
|
if err := c.Connect(ctx); err != nil {
|
|
|
|
|
return err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Read content
|
|
|
|
|
content, err := io.ReadAll(local)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "read content", err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Create parent directory
|
|
|
|
|
dir := filepath.Dir(remote)
|
|
|
|
|
dirCmd := fmt.Sprintf("mkdir -p %q", dir)
|
|
|
|
|
if c.become {
|
|
|
|
|
dirCmd = fmt.Sprintf("sudo mkdir -p %q", dir)
|
|
|
|
|
}
|
|
|
|
|
if _, _, _, err := c.Run(ctx, dirCmd); err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "create parent dir", err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Use cat to write the file (simpler than SCP)
|
|
|
|
|
writeCmd := fmt.Sprintf("cat > %q && chmod %o %q", remote, mode, remote)
|
2026-02-04 01:27:01 +00:00
|
|
|
|
feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
|
|
|
// If become is needed, we construct a command that reads password then content from stdin
|
|
|
|
|
// But we need to be careful with handling stdin for sudo + cat.
|
|
|
|
|
// We'll use a session with piped stdin.
|
2026-02-04 01:27:01 +00:00
|
|
|
|
feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
|
|
|
session2, err := c.client.NewSession()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "new session for write", err)
|
|
|
|
|
}
|
|
|
|
|
defer func() { _ = session2.Close() }()
|
|
|
|
|
|
|
|
|
|
stdin, err := session2.StdinPipe()
|
|
|
|
|
if err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "stdin pipe", err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
var stderrBuf bytes.Buffer
|
|
|
|
|
session2.Stderr = &stderrBuf
|
|
|
|
|
|
|
|
|
|
if c.become {
|
|
|
|
|
becomeUser := c.becomeUser
|
|
|
|
|
if becomeUser == "" {
|
|
|
|
|
becomeUser = "root"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pass := c.becomePass
|
|
|
|
|
if pass == "" {
|
|
|
|
|
pass = c.password
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if pass != "" {
|
|
|
|
|
// Use sudo -S with password from stdin
|
|
|
|
|
writeCmd = fmt.Sprintf("sudo -S -u %s bash -c 'cat > %q && chmod %o %q'",
|
|
|
|
|
becomeUser, remote, mode, remote)
|
|
|
|
|
} else {
|
|
|
|
|
// Use passwordless sudo (sudo -n) to avoid consuming file content as password
|
|
|
|
|
writeCmd = fmt.Sprintf("sudo -n -u %s bash -c 'cat > %q && chmod %o %q'",
|
|
|
|
|
becomeUser, remote, mode, remote)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if err := session2.Start(writeCmd); err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "start write", err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
go func() {
|
|
|
|
|
defer stdin.Close()
|
|
|
|
|
if pass != "" {
|
|
|
|
|
_, _ = io.WriteString(stdin, pass+"\n")
|
|
|
|
|
}
|
|
|
|
|
_, _ = stdin.Write(content)
|
|
|
|
|
}()
|
|
|
|
|
} else {
|
|
|
|
|
// Normal write
|
|
|
|
|
if err := session2.Start(writeCmd); err != nil {
|
|
|
|
|
return log.E("ssh.Upload", "start write", err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
go func() {
|
|
|
|
|
defer stdin.Close()
|
|
|
|
|
_, _ = stdin.Write(content)
|
|
|
|
|
}()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if err := session2.Wait(); err != nil {
|
|
|
|
|
return log.E("ssh.Upload", fmt.Sprintf("write failed (stderr: %s)", stderrBuf.String()), err)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Download copies a file from the remote host.
|
|
|
|
|
func (c *SSHClient) Download(ctx context.Context, remote string) ([]byte, error) {
|
|
|
|
|
if err := c.Connect(ctx); err != nil {
|
|
|
|
|
return nil, err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
cmd := fmt.Sprintf("cat %q", remote)
|
|
|
|
|
|
|
|
|
|
stdout, stderr, exitCode, err := c.Run(ctx, cmd)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return nil, err
|
|
|
|
|
}
|
|
|
|
|
if exitCode != 0 {
|
|
|
|
|
return nil, log.E("ssh.Download", fmt.Sprintf("cat failed: %s", stderr), nil)
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return []byte(stdout), nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// FileExists checks if a file exists on the remote host.
|
|
|
|
|
func (c *SSHClient) FileExists(ctx context.Context, path string) (bool, error) {
|
|
|
|
|
cmd := fmt.Sprintf("test -e %q && echo yes || echo no", path)
|
|
|
|
|
stdout, _, exitCode, err := c.Run(ctx, cmd)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return false, err
|
|
|
|
|
}
|
|
|
|
|
if exitCode != 0 {
|
|
|
|
|
// test command failed but didn't error - file doesn't exist
|
|
|
|
|
return false, nil
|
|
|
|
|
}
|
|
|
|
|
return strings.TrimSpace(stdout) == "yes", nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Stat returns file info from the remote host.
|
|
|
|
|
func (c *SSHClient) Stat(ctx context.Context, path string) (map[string]any, error) {
|
|
|
|
|
// Simple approach - get basic file info
|
|
|
|
|
cmd := fmt.Sprintf(`
|
|
|
|
|
if [ -e %q ]; then
|
|
|
|
|
if [ -d %q ]; then
|
|
|
|
|
echo "exists=true isdir=true"
|
|
|
|
|
else
|
|
|
|
|
echo "exists=true isdir=false"
|
|
|
|
|
fi
|
|
|
|
|
else
|
|
|
|
|
echo "exists=false"
|
|
|
|
|
fi
|
|
|
|
|
`, path, path)
|
|
|
|
|
|
|
|
|
|
stdout, _, _, err := c.Run(ctx, cmd)
|
|
|
|
|
if err != nil {
|
|
|
|
|
return nil, err
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
result := make(map[string]any)
|
|
|
|
|
parts := strings.Fields(strings.TrimSpace(stdout))
|
|
|
|
|
for _, part := range parts {
|
|
|
|
|
kv := strings.SplitN(part, "=", 2)
|
|
|
|
|
if len(kv) == 2 {
|
|
|
|
|
result[kv[0]] = kv[1] == "true"
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return result, nil
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// SetBecome enables privilege escalation.
|
|
|
|
|
func (c *SSHClient) SetBecome(become bool, user, password string) {
|
|
|
|
|
c.mu.Lock()
|
|
|
|
|
defer c.mu.Unlock()
|
|
|
|
|
c.become = become
|
|
|
|
|
if user != "" {
|
|
|
|
|
c.becomeUser = user
|
|
|
|
|
}
|
|
|
|
|
if password != "" {
|
|
|
|
|
c.becomePass = password
|
|
|
|
|
}
|
|
|
|
|
}
|