* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
392 lines
9.9 KiB
Go
392 lines
9.9 KiB
Go
package build
|
|
|
|
import (
|
|
"archive/tar"
|
|
"archive/zip"
|
|
"bytes"
|
|
"compress/gzip"
|
|
"io"
|
|
"os"
|
|
"path/filepath"
|
|
"testing"
|
|
|
|
"github.com/Snider/Borg/pkg/compress"
|
|
"github.com/stretchr/testify/assert"
|
|
"github.com/stretchr/testify/require"
|
|
)
|
|
|
|
// setupArchiveTestFile creates a test binary file in a temp directory with the standard structure.
|
|
// Returns the path to the binary and the output directory.
|
|
func setupArchiveTestFile(t *testing.T, name, os_, arch string) (binaryPath string, outputDir string) {
|
|
t.Helper()
|
|
|
|
outputDir = t.TempDir()
|
|
|
|
// Create platform directory: dist/os_arch
|
|
platformDir := filepath.Join(outputDir, os_+"_"+arch)
|
|
err := os.MkdirAll(platformDir, 0755)
|
|
require.NoError(t, err)
|
|
|
|
// Create test binary
|
|
binaryPath = filepath.Join(platformDir, name)
|
|
content := []byte("#!/bin/bash\necho 'Hello, World!'\n")
|
|
err = os.WriteFile(binaryPath, content, 0755)
|
|
require.NoError(t, err)
|
|
|
|
return binaryPath, outputDir
|
|
}
|
|
|
|
func TestArchive_Good(t *testing.T) {
|
|
t.Run("creates tar.gz for linux", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp", "linux", "amd64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
require.NoError(t, err)
|
|
|
|
// Verify archive was created
|
|
expectedPath := filepath.Join(outputDir, "myapp_linux_amd64.tar.gz")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
// Verify OS and Arch are preserved
|
|
assert.Equal(t, "linux", result.OS)
|
|
assert.Equal(t, "amd64", result.Arch)
|
|
|
|
// Verify archive content
|
|
verifyTarGzContent(t, result.Path, "myapp")
|
|
})
|
|
|
|
t.Run("creates tar.gz for darwin", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp", "darwin", "arm64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "darwin",
|
|
Arch: "arm64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
require.NoError(t, err)
|
|
|
|
expectedPath := filepath.Join(outputDir, "myapp_darwin_arm64.tar.gz")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
verifyTarGzContent(t, result.Path, "myapp")
|
|
})
|
|
|
|
t.Run("creates zip for windows", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp.exe", "windows", "amd64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "windows",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
require.NoError(t, err)
|
|
|
|
// Windows archives should strip .exe from archive name
|
|
expectedPath := filepath.Join(outputDir, "myapp_windows_amd64.zip")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
verifyZipContent(t, result.Path, "myapp.exe")
|
|
})
|
|
|
|
t.Run("preserves checksum field", func(t *testing.T) {
|
|
binaryPath, _ := setupArchiveTestFile(t, "myapp", "linux", "amd64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
Checksum: "abc123",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
require.NoError(t, err)
|
|
assert.Equal(t, "abc123", result.Checksum)
|
|
})
|
|
|
|
t.Run("creates tar.xz for linux with ArchiveXZ", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp", "linux", "amd64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := ArchiveXZ(artifact)
|
|
require.NoError(t, err)
|
|
|
|
expectedPath := filepath.Join(outputDir, "myapp_linux_amd64.tar.xz")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
verifyTarXzContent(t, result.Path, "myapp")
|
|
})
|
|
|
|
t.Run("creates tar.xz for darwin with ArchiveWithFormat", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp", "darwin", "arm64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "darwin",
|
|
Arch: "arm64",
|
|
}
|
|
|
|
result, err := ArchiveWithFormat(artifact, ArchiveFormatXZ)
|
|
require.NoError(t, err)
|
|
|
|
expectedPath := filepath.Join(outputDir, "myapp_darwin_arm64.tar.xz")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
verifyTarXzContent(t, result.Path, "myapp")
|
|
})
|
|
|
|
t.Run("windows still uses zip even with xz format", func(t *testing.T) {
|
|
binaryPath, outputDir := setupArchiveTestFile(t, "myapp.exe", "windows", "amd64")
|
|
|
|
artifact := Artifact{
|
|
Path: binaryPath,
|
|
OS: "windows",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := ArchiveWithFormat(artifact, ArchiveFormatXZ)
|
|
require.NoError(t, err)
|
|
|
|
// Windows should still get .zip regardless of format
|
|
expectedPath := filepath.Join(outputDir, "myapp_windows_amd64.zip")
|
|
assert.Equal(t, expectedPath, result.Path)
|
|
assert.FileExists(t, result.Path)
|
|
|
|
verifyZipContent(t, result.Path, "myapp.exe")
|
|
})
|
|
}
|
|
|
|
func TestArchive_Bad(t *testing.T) {
|
|
t.Run("returns error for empty path", func(t *testing.T) {
|
|
artifact := Artifact{
|
|
Path: "",
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
assert.Error(t, err)
|
|
assert.Contains(t, err.Error(), "artifact path is empty")
|
|
assert.Empty(t, result.Path)
|
|
})
|
|
|
|
t.Run("returns error for non-existent file", func(t *testing.T) {
|
|
artifact := Artifact{
|
|
Path: "/nonexistent/path/binary",
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
assert.Error(t, err)
|
|
assert.Contains(t, err.Error(), "source file not found")
|
|
assert.Empty(t, result.Path)
|
|
})
|
|
|
|
t.Run("returns error for directory path", func(t *testing.T) {
|
|
dir := t.TempDir()
|
|
|
|
artifact := Artifact{
|
|
Path: dir,
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
result, err := Archive(artifact)
|
|
assert.Error(t, err)
|
|
assert.Contains(t, err.Error(), "source path is a directory")
|
|
assert.Empty(t, result.Path)
|
|
})
|
|
}
|
|
|
|
func TestArchiveAll_Good(t *testing.T) {
|
|
t.Run("archives multiple artifacts", func(t *testing.T) {
|
|
outputDir := t.TempDir()
|
|
|
|
// Create multiple binaries
|
|
var artifacts []Artifact
|
|
targets := []struct {
|
|
os_ string
|
|
arch string
|
|
}{
|
|
{"linux", "amd64"},
|
|
{"linux", "arm64"},
|
|
{"darwin", "arm64"},
|
|
{"windows", "amd64"},
|
|
}
|
|
|
|
for _, target := range targets {
|
|
platformDir := filepath.Join(outputDir, target.os_+"_"+target.arch)
|
|
err := os.MkdirAll(platformDir, 0755)
|
|
require.NoError(t, err)
|
|
|
|
name := "myapp"
|
|
if target.os_ == "windows" {
|
|
name = "myapp.exe"
|
|
}
|
|
|
|
binaryPath := filepath.Join(platformDir, name)
|
|
err = os.WriteFile(binaryPath, []byte("binary content"), 0755)
|
|
require.NoError(t, err)
|
|
|
|
artifacts = append(artifacts, Artifact{
|
|
Path: binaryPath,
|
|
OS: target.os_,
|
|
Arch: target.arch,
|
|
})
|
|
}
|
|
|
|
results, err := ArchiveAll(artifacts)
|
|
require.NoError(t, err)
|
|
require.Len(t, results, 4)
|
|
|
|
// Verify all archives were created
|
|
for i, result := range results {
|
|
assert.FileExists(t, result.Path)
|
|
assert.Equal(t, artifacts[i].OS, result.OS)
|
|
assert.Equal(t, artifacts[i].Arch, result.Arch)
|
|
}
|
|
})
|
|
|
|
t.Run("returns nil for empty slice", func(t *testing.T) {
|
|
results, err := ArchiveAll([]Artifact{})
|
|
assert.NoError(t, err)
|
|
assert.Nil(t, results)
|
|
})
|
|
|
|
t.Run("returns nil for nil slice", func(t *testing.T) {
|
|
results, err := ArchiveAll(nil)
|
|
assert.NoError(t, err)
|
|
assert.Nil(t, results)
|
|
})
|
|
}
|
|
|
|
func TestArchiveAll_Bad(t *testing.T) {
|
|
t.Run("returns partial results on error", func(t *testing.T) {
|
|
binaryPath, _ := setupArchiveTestFile(t, "myapp", "linux", "amd64")
|
|
|
|
artifacts := []Artifact{
|
|
{Path: binaryPath, OS: "linux", Arch: "amd64"},
|
|
{Path: "/nonexistent/binary", OS: "linux", Arch: "arm64"}, // This will fail
|
|
}
|
|
|
|
results, err := ArchiveAll(artifacts)
|
|
assert.Error(t, err)
|
|
// Should have the first successful result
|
|
assert.Len(t, results, 1)
|
|
assert.FileExists(t, results[0].Path)
|
|
})
|
|
}
|
|
|
|
func TestArchiveFilename_Good(t *testing.T) {
|
|
t.Run("generates correct tar.gz filename", func(t *testing.T) {
|
|
artifact := Artifact{
|
|
Path: "/output/linux_amd64/myapp",
|
|
OS: "linux",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
filename := archiveFilename(artifact, ".tar.gz")
|
|
assert.Equal(t, "/output/myapp_linux_amd64.tar.gz", filename)
|
|
})
|
|
|
|
t.Run("generates correct zip filename", func(t *testing.T) {
|
|
artifact := Artifact{
|
|
Path: "/output/windows_amd64/myapp.exe",
|
|
OS: "windows",
|
|
Arch: "amd64",
|
|
}
|
|
|
|
filename := archiveFilename(artifact, ".zip")
|
|
assert.Equal(t, "/output/myapp_windows_amd64.zip", filename)
|
|
})
|
|
|
|
t.Run("handles nested output directories", func(t *testing.T) {
|
|
artifact := Artifact{
|
|
Path: "/project/dist/linux_arm64/cli",
|
|
OS: "linux",
|
|
Arch: "arm64",
|
|
}
|
|
|
|
filename := archiveFilename(artifact, ".tar.gz")
|
|
assert.Equal(t, "/project/dist/cli_linux_arm64.tar.gz", filename)
|
|
})
|
|
}
|
|
|
|
// verifyTarGzContent opens a tar.gz file and verifies it contains the expected file.
|
|
func verifyTarGzContent(t *testing.T, archivePath, expectedName string) {
|
|
t.Helper()
|
|
|
|
file, err := os.Open(archivePath)
|
|
require.NoError(t, err)
|
|
defer file.Close()
|
|
|
|
gzReader, err := gzip.NewReader(file)
|
|
require.NoError(t, err)
|
|
defer gzReader.Close()
|
|
|
|
tarReader := tar.NewReader(gzReader)
|
|
|
|
header, err := tarReader.Next()
|
|
require.NoError(t, err)
|
|
assert.Equal(t, expectedName, header.Name)
|
|
|
|
// Verify there's only one file
|
|
_, err = tarReader.Next()
|
|
assert.Equal(t, io.EOF, err)
|
|
}
|
|
|
|
// verifyZipContent opens a zip file and verifies it contains the expected file.
|
|
func verifyZipContent(t *testing.T, archivePath, expectedName string) {
|
|
t.Helper()
|
|
|
|
reader, err := zip.OpenReader(archivePath)
|
|
require.NoError(t, err)
|
|
defer reader.Close()
|
|
|
|
require.Len(t, reader.File, 1)
|
|
assert.Equal(t, expectedName, reader.File[0].Name)
|
|
}
|
|
|
|
// verifyTarXzContent opens a tar.xz file and verifies it contains the expected file.
|
|
func verifyTarXzContent(t *testing.T, archivePath, expectedName string) {
|
|
t.Helper()
|
|
|
|
// Read the xz-compressed file
|
|
xzData, err := os.ReadFile(archivePath)
|
|
require.NoError(t, err)
|
|
|
|
// Decompress with Borg
|
|
tarData, err := compress.Decompress(xzData)
|
|
require.NoError(t, err)
|
|
|
|
// Read tar archive
|
|
tarReader := tar.NewReader(bytes.NewReader(tarData))
|
|
|
|
header, err := tarReader.Next()
|
|
require.NoError(t, err)
|
|
assert.Equal(t, expectedName, header.Name)
|
|
|
|
// Verify there's only one file
|
|
_, err = tarReader.Next()
|
|
assert.Equal(t, io.EOF, err)
|
|
}
|