cli/pkg/agentic/context.go
Snider 989b7e1e65
feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts

- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
  - setup: Interactive with variant selection
  - ci: Minimal for CI/CD environments
  - dev: Full development variant
  - go/php/agent: Targeted development variants
- All scripts include security hardening:
  - Secure temp directories (mktemp -d)
  - Architecture validation
  - Version validation after GitHub API call
  - Proper cleanup on exit
  - PowerShell PATH updates on Windows (avoids setx truncation)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(build): add tar.xz support and unified installer scripts

- Add tar.xz archive support using Borg's compress package
  - ArchiveXZ() and ArchiveWithFormat() for configurable compression
  - Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
  - install.sh and install.bat with BunnyCDN edge variable support
  - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
  - MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
  verify_install params, VARIANT validation, CI PATH persistence

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add build and release config files

- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: move plans from docs/ to tasks/

Consolidate planning documents in tasks/plans/ directory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(install): address CodeRabbit review feedback

- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(rag): add Go RAG implementation with Qdrant + Ollama

Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.

New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections

Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration

Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(deploy): add pure-Go Ansible executor and Coolify API integration

Implement infrastructure deployment system with:

- pkg/ansible: Pure Go Ansible executor
  - Playbook/inventory parsing (types.go, parser.go)
  - Full execution engine with variable templating, loops, blocks,
    conditionals, handlers, and fact gathering (executor.go)
  - SSH client with key/password auth and privilege escalation (ssh.go)
  - 35+ module implementations: shell, command, copy, template, file,
    apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)

- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
  - List/get servers, projects, applications, databases, services
  - Generic Call() for any OpenAPI operation

- pkg/deploy/python: Embedded Python runtime for swagger client integration

- internal/cmd/deploy: CLI commands
  - core deploy servers/projects/apps/databases/services/team
  - core deploy call <operation> [params-json]

This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): address linter warnings and build errors

- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go

All golangci-lint checks now pass for deploy packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style(deploy): fix gofmt formatting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): use known_hosts for SSH host key verification

Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app

Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.

Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.

Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review comments from CodeRabbit, Copilot, and Gemini

Fixes across 25 files addressing 46+ review comments:

- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
  fix Upload with become=true and no password (use sudo -n), honour
  SSH timeout config, use E() helper for contextual errors, quote git
  refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
  use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
  handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
  for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
  for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
  pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(build): make release dry-run by default and remove darwin/amd64 target

Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00

352 lines
8.6 KiB
Go

// Package agentic provides AI collaboration features for task management.
package agentic
import (
"bytes"
"os"
"os/exec"
"path/filepath"
"regexp"
"strings"
"github.com/host-uk/core/pkg/ai"
"github.com/host-uk/core/pkg/log"
)
// FileContent represents the content of a file for AI context.
type FileContent struct {
// Path is the relative path to the file.
Path string `json:"path"`
// Content is the file content.
Content string `json:"content"`
// Language is the detected programming language.
Language string `json:"language"`
}
// TaskContext contains gathered context for AI collaboration.
type TaskContext struct {
// Task is the task being worked on.
Task *Task `json:"task"`
// Files is a list of relevant file contents.
Files []FileContent `json:"files"`
// GitStatus is the current git status output.
GitStatus string `json:"git_status"`
// RecentCommits is the recent commit log.
RecentCommits string `json:"recent_commits"`
// RelatedCode contains code snippets related to the task.
RelatedCode []FileContent `json:"related_code"`
// RAGContext contains relevant documentation from the vector database.
RAGContext string `json:"rag_context,omitempty"`
}
// BuildTaskContext gathers context for AI collaboration on a task.
func BuildTaskContext(task *Task, dir string) (*TaskContext, error) {
const op = "agentic.BuildTaskContext"
if task == nil {
return nil, log.E(op, "task is required", nil)
}
if dir == "" {
cwd, err := os.Getwd()
if err != nil {
return nil, log.E(op, "failed to get working directory", err)
}
dir = cwd
}
ctx := &TaskContext{
Task: task,
}
// Gather files mentioned in the task
files, err := GatherRelatedFiles(task, dir)
if err != nil {
// Non-fatal: continue without files
files = nil
}
ctx.Files = files
// Get git status
gitStatus, _ := runGitCommand(dir, "status", "--porcelain")
ctx.GitStatus = gitStatus
// Get recent commits
recentCommits, _ := runGitCommand(dir, "log", "--oneline", "-10")
ctx.RecentCommits = recentCommits
// Find related code by searching for keywords
relatedCode, err := findRelatedCode(task, dir)
if err != nil {
relatedCode = nil
}
ctx.RelatedCode = relatedCode
// Query RAG for relevant documentation (graceful degradation)
ragCtx := ai.QueryRAGForTask(ai.TaskInfo{
Title: task.Title,
Description: task.Description,
})
ctx.RAGContext = ragCtx
return ctx, nil
}
// GatherRelatedFiles reads files mentioned in the task.
func GatherRelatedFiles(task *Task, dir string) ([]FileContent, error) {
const op = "agentic.GatherRelatedFiles"
if task == nil {
return nil, log.E(op, "task is required", nil)
}
var files []FileContent
// Read files explicitly mentioned in the task
for _, relPath := range task.Files {
fullPath := filepath.Join(dir, relPath)
content, err := os.ReadFile(fullPath)
if err != nil {
// Skip files that don't exist
continue
}
files = append(files, FileContent{
Path: relPath,
Content: string(content),
Language: detectLanguage(relPath),
})
}
return files, nil
}
// findRelatedCode searches for code related to the task by keywords.
func findRelatedCode(task *Task, dir string) ([]FileContent, error) {
const op = "agentic.findRelatedCode"
if task == nil {
return nil, log.E(op, "task is required", nil)
}
// Extract keywords from title and description
keywords := extractKeywords(task.Title + " " + task.Description)
if len(keywords) == 0 {
return nil, nil
}
var files []FileContent
seen := make(map[string]bool)
// Search for each keyword using git grep
for _, keyword := range keywords {
if len(keyword) < 3 {
continue
}
output, err := runGitCommand(dir, "grep", "-l", "-i", keyword, "--", "*.go", "*.ts", "*.js", "*.py")
if err != nil {
continue
}
// Parse matched files
for _, line := range strings.Split(output, "\n") {
line = strings.TrimSpace(line)
if line == "" || seen[line] {
continue
}
seen[line] = true
// Limit to 10 related files
if len(files) >= 10 {
break
}
fullPath := filepath.Join(dir, line)
content, err := os.ReadFile(fullPath)
if err != nil {
continue
}
// Truncate large files
contentStr := string(content)
if len(contentStr) > 5000 {
contentStr = contentStr[:5000] + "\n... (truncated)"
}
files = append(files, FileContent{
Path: line,
Content: contentStr,
Language: detectLanguage(line),
})
}
if len(files) >= 10 {
break
}
}
return files, nil
}
// extractKeywords extracts meaningful words from text for searching.
func extractKeywords(text string) []string {
// Remove common words and extract identifiers
text = strings.ToLower(text)
// Split by non-alphanumeric characters
re := regexp.MustCompile(`[^a-zA-Z0-9]+`)
words := re.Split(text, -1)
// Filter stop words and short words
stopWords := map[string]bool{
"the": true, "a": true, "an": true, "and": true, "or": true, "but": true,
"in": true, "on": true, "at": true, "to": true, "for": true, "of": true,
"with": true, "by": true, "from": true, "is": true, "are": true, "was": true,
"be": true, "been": true, "being": true, "have": true, "has": true, "had": true,
"do": true, "does": true, "did": true, "will": true, "would": true, "could": true,
"should": true, "may": true, "might": true, "must": true, "shall": true,
"this": true, "that": true, "these": true, "those": true, "it": true,
"add": true, "create": true, "update": true, "fix": true, "remove": true,
"implement": true, "new": true, "file": true, "code": true,
}
var keywords []string
for _, word := range words {
word = strings.TrimSpace(word)
if len(word) >= 3 && !stopWords[word] {
keywords = append(keywords, word)
}
}
// Limit to first 5 keywords
if len(keywords) > 5 {
keywords = keywords[:5]
}
return keywords
}
// detectLanguage detects the programming language from a file extension.
func detectLanguage(path string) string {
ext := strings.ToLower(filepath.Ext(path))
languages := map[string]string{
".go": "go",
".ts": "typescript",
".tsx": "typescript",
".js": "javascript",
".jsx": "javascript",
".py": "python",
".rs": "rust",
".java": "java",
".kt": "kotlin",
".swift": "swift",
".c": "c",
".cpp": "cpp",
".h": "c",
".hpp": "cpp",
".rb": "ruby",
".php": "php",
".cs": "csharp",
".fs": "fsharp",
".scala": "scala",
".sh": "bash",
".bash": "bash",
".zsh": "zsh",
".yaml": "yaml",
".yml": "yaml",
".json": "json",
".xml": "xml",
".html": "html",
".css": "css",
".scss": "scss",
".sql": "sql",
".md": "markdown",
}
if lang, ok := languages[ext]; ok {
return lang
}
return "text"
}
// runGitCommand runs a git command and returns the output.
func runGitCommand(dir string, args ...string) (string, error) {
cmd := exec.Command("git", args...)
cmd.Dir = dir
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return "", err
}
return stdout.String(), nil
}
// FormatContext formats the TaskContext for AI consumption.
func (tc *TaskContext) FormatContext() string {
var sb strings.Builder
sb.WriteString("# Task Context\n\n")
// Task info
sb.WriteString("## Task\n")
sb.WriteString("ID: " + tc.Task.ID + "\n")
sb.WriteString("Title: " + tc.Task.Title + "\n")
sb.WriteString("Priority: " + string(tc.Task.Priority) + "\n")
sb.WriteString("Status: " + string(tc.Task.Status) + "\n")
sb.WriteString("\n### Description\n")
sb.WriteString(tc.Task.Description + "\n\n")
// Files
if len(tc.Files) > 0 {
sb.WriteString("## Task Files\n")
for _, f := range tc.Files {
sb.WriteString("### " + f.Path + " (" + f.Language + ")\n")
sb.WriteString("```" + f.Language + "\n")
sb.WriteString(f.Content)
sb.WriteString("\n```\n\n")
}
}
// Git status
if tc.GitStatus != "" {
sb.WriteString("## Git Status\n")
sb.WriteString("```\n")
sb.WriteString(tc.GitStatus)
sb.WriteString("\n```\n\n")
}
// Recent commits
if tc.RecentCommits != "" {
sb.WriteString("## Recent Commits\n")
sb.WriteString("```\n")
sb.WriteString(tc.RecentCommits)
sb.WriteString("\n```\n\n")
}
// Related code
if len(tc.RelatedCode) > 0 {
sb.WriteString("## Related Code\n")
for _, f := range tc.RelatedCode {
sb.WriteString("### " + f.Path + " (" + f.Language + ")\n")
sb.WriteString("```" + f.Language + "\n")
sb.WriteString(f.Content)
sb.WriteString("\n```\n\n")
}
}
// Relevant documentation from RAG
if tc.RAGContext != "" {
sb.WriteString("## Relevant Documentation\n")
sb.WriteString(tc.RAGContext)
sb.WriteString("\n\n")
}
return sb.String()
}