cli/internal/cmd/security/cmd_security.go
Snider 5e2765fd5f feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts

- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
  - setup: Interactive with variant selection
  - ci: Minimal for CI/CD environments
  - dev: Full development variant
  - go/php/agent: Targeted development variants
- All scripts include security hardening:
  - Secure temp directories (mktemp -d)
  - Architecture validation
  - Version validation after GitHub API call
  - Proper cleanup on exit
  - PowerShell PATH updates on Windows (avoids setx truncation)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(build): add tar.xz support and unified installer scripts

- Add tar.xz archive support using Borg's compress package
  - ArchiveXZ() and ArchiveWithFormat() for configurable compression
  - Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
  - install.sh and install.bat with BunnyCDN edge variable support
  - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
  - MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
  verify_install params, VARIANT validation, CI PATH persistence

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add build and release config files

- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: move plans from docs/ to tasks/

Consolidate planning documents in tasks/plans/ directory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(install): address CodeRabbit review feedback

- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(rag): add Go RAG implementation with Qdrant + Ollama

Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.

New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections

Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration

Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(deploy): add pure-Go Ansible executor and Coolify API integration

Implement infrastructure deployment system with:

- pkg/ansible: Pure Go Ansible executor
  - Playbook/inventory parsing (types.go, parser.go)
  - Full execution engine with variable templating, loops, blocks,
    conditionals, handlers, and fact gathering (executor.go)
  - SSH client with key/password auth and privilege escalation (ssh.go)
  - 35+ module implementations: shell, command, copy, template, file,
    apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)

- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
  - List/get servers, projects, applications, databases, services
  - Generic Call() for any OpenAPI operation

- pkg/deploy/python: Embedded Python runtime for swagger client integration

- internal/cmd/deploy: CLI commands
  - core deploy servers/projects/apps/databases/services/team
  - core deploy call <operation> [params-json]

This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): address linter warnings and build errors

- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go

All golangci-lint checks now pass for deploy packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style(deploy): fix gofmt formatting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): use known_hosts for SSH host key verification

Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app

Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.

Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.

Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review comments from CodeRabbit, Copilot, and Gemini

Fixes across 25 files addressing 46+ review comments:

- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
  fix Upload with become=true and no password (use sudo -n), honour
  SSH timeout config, use E() helper for contextual errors, quote git
  refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
  use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
  handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
  for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
  for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
  pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(build): make release dry-run by default and remove darwin/amd64 target

Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00

256 lines
6.5 KiB
Go

package security
import (
"errors"
"fmt"
"os/exec"
"strings"
"github.com/host-uk/core/pkg/cli"
"github.com/host-uk/core/pkg/i18n"
"github.com/host-uk/core/pkg/repos"
)
var (
// Command flags
securityRegistryPath string
securityRepo string
securitySeverity string
securityJSON bool
securityTarget string // External repo target (e.g. "wailsapp/wails")
)
// AddSecurityCommands adds the 'security' command to the root.
func AddSecurityCommands(root *cli.Command) {
secCmd := &cli.Command{
Use: "security",
Short: i18n.T("cmd.security.short"),
Long: i18n.T("cmd.security.long"),
}
addAlertsCommand(secCmd)
addDepsCommand(secCmd)
addScanCommand(secCmd)
addSecretsCommand(secCmd)
addJobsCommand(secCmd)
root.AddCommand(secCmd)
}
// DependabotAlert represents a Dependabot vulnerability alert.
type DependabotAlert struct {
Number int `json:"number"`
State string `json:"state"`
Advisory struct {
Severity string `json:"severity"`
CVEID string `json:"cve_id"`
Summary string `json:"summary"`
Description string `json:"description"`
} `json:"security_advisory"`
Dependency struct {
Package struct {
Name string `json:"name"`
Ecosystem string `json:"ecosystem"`
} `json:"package"`
ManifestPath string `json:"manifest_path"`
} `json:"dependency"`
SecurityVulnerability struct {
Package struct {
Name string `json:"name"`
Ecosystem string `json:"ecosystem"`
} `json:"package"`
FirstPatchedVersion struct {
Identifier string `json:"identifier"`
} `json:"first_patched_version"`
VulnerableVersionRange string `json:"vulnerable_version_range"`
} `json:"security_vulnerability"`
}
// CodeScanningAlert represents a code scanning alert.
type CodeScanningAlert struct {
Number int `json:"number"`
State string `json:"state"`
DismissedReason string `json:"dismissed_reason"`
Rule struct {
ID string `json:"id"`
Severity string `json:"severity"`
Description string `json:"description"`
Tags []string `json:"tags"`
} `json:"rule"`
Tool struct {
Name string `json:"name"`
Version string `json:"version"`
} `json:"tool"`
MostRecentInstance struct {
Location struct {
Path string `json:"path"`
StartLine int `json:"start_line"`
EndLine int `json:"end_line"`
} `json:"location"`
Message struct {
Text string `json:"text"`
} `json:"message"`
} `json:"most_recent_instance"`
}
// SecretScanningAlert represents a secret scanning alert.
type SecretScanningAlert struct {
Number int `json:"number"`
State string `json:"state"`
SecretType string `json:"secret_type"`
Secret string `json:"secret"`
PushProtection bool `json:"push_protection_bypassed"`
Resolution string `json:"resolution"`
}
// loadRegistry loads the repository registry.
func loadRegistry(registryPath string) (*repos.Registry, error) {
if registryPath != "" {
reg, err := repos.LoadRegistry(registryPath)
if err != nil {
return nil, cli.Wrap(err, "load registry")
}
return reg, nil
}
path, err := repos.FindRegistry()
if err != nil {
return nil, cli.Wrap(err, "find registry")
}
reg, err := repos.LoadRegistry(path)
if err != nil {
return nil, cli.Wrap(err, "load registry")
}
return reg, nil
}
// checkGH verifies gh CLI is available.
func checkGH() error {
if _, err := exec.LookPath("gh"); err != nil {
return errors.New(i18n.T("error.gh_not_found"))
}
return nil
}
// runGHAPI runs a gh api command and returns the output.
func runGHAPI(endpoint string) ([]byte, error) {
cmd := exec.Command("gh", "api", endpoint, "--paginate")
output, err := cmd.Output()
if err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
stderr := string(exitErr.Stderr)
// Handle common errors gracefully
if strings.Contains(stderr, "404") || strings.Contains(stderr, "Not Found") {
return []byte("[]"), nil // Return empty array for not found
}
if strings.Contains(stderr, "403") {
return nil, fmt.Errorf("access denied (check token permissions)")
}
}
return nil, cli.Wrap(err, "run gh api")
}
return output, nil
}
// severityStyle returns the appropriate style for a severity level.
func severityStyle(severity string) *cli.AnsiStyle {
switch strings.ToLower(severity) {
case "critical":
return cli.ErrorStyle
case "high":
return cli.WarningStyle
case "medium":
return cli.ValueStyle
default:
return cli.DimStyle
}
}
// filterBySeverity checks if the severity matches the filter.
func filterBySeverity(severity, filter string) bool {
if filter == "" {
return true
}
severities := strings.Split(strings.ToLower(filter), ",")
sev := strings.ToLower(severity)
for _, s := range severities {
if strings.TrimSpace(s) == sev {
return true
}
}
return false
}
// getReposToCheck returns the list of repos to check based on flags.
func getReposToCheck(reg *repos.Registry, repoFilter string) []*repos.Repo {
if repoFilter != "" {
if repo, ok := reg.Get(repoFilter); ok {
return []*repos.Repo{repo}
}
return nil
}
return reg.List()
}
// buildTargetRepo creates a synthetic Repo entry for an external target (e.g. "wailsapp/wails").
func buildTargetRepo(target string) (*repos.Repo, string) {
parts := strings.SplitN(target, "/", 2)
if len(parts) != 2 || parts[0] == "" || parts[1] == "" {
return nil, ""
}
return &repos.Repo{Name: parts[1]}, target
}
// AlertSummary holds aggregated alert counts.
type AlertSummary struct {
Critical int
High int
Medium int
Low int
Unknown int
Total int
}
// Add increments summary counters for the provided severity.
func (s *AlertSummary) Add(severity string) {
s.Total++
switch strings.ToLower(severity) {
case "critical":
s.Critical++
case "high":
s.High++
case "medium":
s.Medium++
case "low":
s.Low++
default:
s.Unknown++
}
}
// String renders a human-readable summary of alert counts.
func (s *AlertSummary) String() string {
parts := []string{}
if s.Critical > 0 {
parts = append(parts, cli.ErrorStyle.Render(fmt.Sprintf("%d critical", s.Critical)))
}
if s.High > 0 {
parts = append(parts, cli.WarningStyle.Render(fmt.Sprintf("%d high", s.High)))
}
if s.Medium > 0 {
parts = append(parts, cli.ValueStyle.Render(fmt.Sprintf("%d medium", s.Medium)))
}
if s.Low > 0 {
parts = append(parts, cli.DimStyle.Render(fmt.Sprintf("%d low", s.Low)))
}
if s.Unknown > 0 {
parts = append(parts, cli.DimStyle.Render(fmt.Sprintf("%d unknown", s.Unknown)))
}
if len(parts) == 0 {
return cli.SuccessStyle.Render("No alerts")
}
return strings.Join(parts, " | ")
}