* feat(cli): wire release command and add installer scripts
- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
- setup: Interactive with variant selection
- ci: Minimal for CI/CD environments
- dev: Full development variant
- go/php/agent: Targeted development variants
- All scripts include security hardening:
- Secure temp directories (mktemp -d)
- Architecture validation
- Version validation after GitHub API call
- Proper cleanup on exit
- PowerShell PATH updates on Windows (avoids setx truncation)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(build): add tar.xz support and unified installer scripts
- Add tar.xz archive support using Borg's compress package
- ArchiveXZ() and ArchiveWithFormat() for configurable compression
- Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
- install.sh and install.bat with BunnyCDN edge variable support
- Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
- MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
verify_install params, VARIANT validation, CI PATH persistence
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add build and release config files
- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: move plans from docs/ to tasks/
Consolidate planning documents in tasks/plans/ directory.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(install): address CodeRabbit review feedback
- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(rag): add Go RAG implementation with Qdrant + Ollama
Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.
New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections
Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration
Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(deploy): add pure-Go Ansible executor and Coolify API integration
Implement infrastructure deployment system with:
- pkg/ansible: Pure Go Ansible executor
- Playbook/inventory parsing (types.go, parser.go)
- Full execution engine with variable templating, loops, blocks,
conditionals, handlers, and fact gathering (executor.go)
- SSH client with key/password auth and privilege escalation (ssh.go)
- 35+ module implementations: shell, command, copy, template, file,
apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)
- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
- List/get servers, projects, applications, databases, services
- Generic Call() for any OpenAPI operation
- pkg/deploy/python: Embedded Python runtime for swagger client integration
- internal/cmd/deploy: CLI commands
- core deploy servers/projects/apps/databases/services/team
- core deploy call <operation> [params-json]
This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): address linter warnings and build errors
- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go
All golangci-lint checks now pass for deploy packages.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style(deploy): fix gofmt formatting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(deploy): use known_hosts for SSH host key verification
Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app
Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.
Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.
Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address PR review comments from CodeRabbit, Copilot, and Gemini
Fixes across 25 files addressing 46+ review comments:
- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
fix Upload with become=true and no password (use sudo -n), honour
SSH timeout config, use E() helper for contextual errors, quote git
refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(build): make release dry-run by default and remove darwin/amd64 target
Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
171 lines
4.1 KiB
Go
171 lines
4.1 KiB
Go
package ai
|
|
|
|
import (
|
|
"bufio"
|
|
"encoding/json"
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"sort"
|
|
"time"
|
|
)
|
|
|
|
// Event represents a recorded AI/security metric event.
|
|
type Event struct {
|
|
Type string `json:"type"`
|
|
Timestamp time.Time `json:"timestamp"`
|
|
AgentID string `json:"agent_id,omitempty"`
|
|
Repo string `json:"repo,omitempty"`
|
|
Duration time.Duration `json:"duration,omitempty"`
|
|
Data map[string]any `json:"data,omitempty"`
|
|
}
|
|
|
|
// metricsDir returns the base directory for metrics storage.
|
|
func metricsDir() (string, error) {
|
|
home, err := os.UserHomeDir()
|
|
if err != nil {
|
|
return "", fmt.Errorf("get home directory: %w", err)
|
|
}
|
|
return filepath.Join(home, ".core", "ai", "metrics"), nil
|
|
}
|
|
|
|
// metricsFilePath returns the JSONL file path for the given date.
|
|
func metricsFilePath(dir string, t time.Time) string {
|
|
return filepath.Join(dir, t.Format("2006-01-02")+".jsonl")
|
|
}
|
|
|
|
// Record appends an event to the daily JSONL file at
|
|
// ~/.core/ai/metrics/YYYY-MM-DD.jsonl.
|
|
func Record(event Event) (err error) {
|
|
if event.Timestamp.IsZero() {
|
|
event.Timestamp = time.Now()
|
|
}
|
|
|
|
dir, err := metricsDir()
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
if err := os.MkdirAll(dir, 0o755); err != nil {
|
|
return fmt.Errorf("create metrics directory: %w", err)
|
|
}
|
|
|
|
path := metricsFilePath(dir, event.Timestamp)
|
|
|
|
f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
|
|
if err != nil {
|
|
return fmt.Errorf("open metrics file: %w", err)
|
|
}
|
|
defer func() {
|
|
if cerr := f.Close(); cerr != nil && err == nil {
|
|
err = fmt.Errorf("close metrics file: %w", cerr)
|
|
}
|
|
}()
|
|
|
|
data, err := json.Marshal(event)
|
|
if err != nil {
|
|
return fmt.Errorf("marshal event: %w", err)
|
|
}
|
|
|
|
if _, err := f.Write(append(data, '\n')); err != nil {
|
|
return fmt.Errorf("write event: %w", err)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// ReadEvents reads events from JSONL files within the given time range.
|
|
func ReadEvents(since time.Time) ([]Event, error) {
|
|
dir, err := metricsDir()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var events []Event
|
|
now := time.Now()
|
|
|
|
// Iterate each day from since to now.
|
|
for d := time.Date(since.Year(), since.Month(), since.Day(), 0, 0, 0, 0, time.Local); !d.After(now); d = d.AddDate(0, 0, 1) {
|
|
path := metricsFilePath(dir, d)
|
|
|
|
dayEvents, err := readMetricsFile(path, since)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
events = append(events, dayEvents...)
|
|
}
|
|
|
|
return events, nil
|
|
}
|
|
|
|
// readMetricsFile reads events from a single JSONL file, returning only those at or after since.
|
|
func readMetricsFile(path string, since time.Time) ([]Event, error) {
|
|
f, err := os.Open(path)
|
|
if err != nil {
|
|
if os.IsNotExist(err) {
|
|
return nil, nil
|
|
}
|
|
return nil, fmt.Errorf("open metrics file %s: %w", path, err)
|
|
}
|
|
defer f.Close()
|
|
|
|
var events []Event
|
|
scanner := bufio.NewScanner(f)
|
|
for scanner.Scan() {
|
|
var ev Event
|
|
if err := json.Unmarshal(scanner.Bytes(), &ev); err != nil {
|
|
continue // skip malformed lines
|
|
}
|
|
if !ev.Timestamp.Before(since) {
|
|
events = append(events, ev)
|
|
}
|
|
}
|
|
if err := scanner.Err(); err != nil {
|
|
return nil, fmt.Errorf("read metrics file %s: %w", path, err)
|
|
}
|
|
return events, nil
|
|
}
|
|
|
|
// Summary aggregates events into counts by type, repo, and agent.
|
|
func Summary(events []Event) map[string]any {
|
|
byType := make(map[string]int)
|
|
byRepo := make(map[string]int)
|
|
byAgent := make(map[string]int)
|
|
|
|
for _, ev := range events {
|
|
byType[ev.Type]++
|
|
if ev.Repo != "" {
|
|
byRepo[ev.Repo]++
|
|
}
|
|
if ev.AgentID != "" {
|
|
byAgent[ev.AgentID]++
|
|
}
|
|
}
|
|
|
|
return map[string]any{
|
|
"total": len(events),
|
|
"by_type": sortedMap(byType),
|
|
"by_repo": sortedMap(byRepo),
|
|
"by_agent": sortedMap(byAgent),
|
|
}
|
|
}
|
|
|
|
// sortedMap returns a slice of key-count pairs sorted by count descending.
|
|
func sortedMap(m map[string]int) []map[string]any {
|
|
type entry struct {
|
|
key string
|
|
count int
|
|
}
|
|
entries := make([]entry, 0, len(m))
|
|
for k, v := range m {
|
|
entries = append(entries, entry{k, v})
|
|
}
|
|
sort.Slice(entries, func(i, j int) bool {
|
|
return entries[i].count > entries[j].count
|
|
})
|
|
result := make([]map[string]any, len(entries))
|
|
for i, e := range entries {
|
|
result[i] = map[string]any{"key": e.key, "count": e.count}
|
|
}
|
|
return result
|
|
}
|