* feat: add security logging and fix framework regressions This commit implements comprehensive security event logging and resolves critical regressions in the core framework. Security Logging: - Enhanced `pkg/log` with a `Security` level and helper. - Added `log.Username()` to consistently identify the executing user. - Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs. - Added `SecurityStyle` to the CLI for consistent visual representation of security events. UniFi Security (CodeQL): - Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert. - Added a `--verify-tls` flag and configuration option to control TLS verification. - Updated command handlers to support the new verification parameter. Framework Fixes: - Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge. - Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures. - These fixes resolve project-wide compilation errors caused by the signature mismatches. I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability. * feat(mcp): add RAG tools (query, ingest, collections) Add vector database tools to the MCP server for RAG operations: - rag_query: Search for relevant documentation using semantic similarity - rag_ingest: Ingest files or directories into the vector database - rag_collections: List available collections Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile) and pkg/rag for Qdrant client access. Default collection is "hostuk-docs" with topK=5 for queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add metrics tools (record, query) Add MCP tools for recording and querying AI/security metrics events. The metrics_record tool writes events to daily JSONL files, and the metrics_query tool provides aggregated statistics by type, repo, and agent. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add 'core mcp serve' command Add CLI command to start the MCP server for AI tool integration. - Create internal/cmd/mcpcmd package with serve subcommand - Support --workspace flag for directory restriction - Handle SIGINT/SIGTERM for clean shutdown - Register in full.go build variant Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(ws): add WebSocket hub package for real-time streaming Add pkg/ws package implementing a hub pattern for WebSocket connections: - Hub manages client connections, broadcasts, and channel subscriptions - Client struct represents connected WebSocket clients - Message types: process_output, process_status, event, error, ping/pong - Channel-based subscription system (subscribe/unsubscribe) - SendProcessOutput and SendProcessStatus for process streaming integration - Full test coverage including concurrency tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(mcp): add process management and WebSocket MCP tools Add MCP tools for process management: - process_start: Start a new external process - process_stop: Gracefully stop a running process - process_kill: Force kill a process - process_list: List all managed processes - process_output: Get captured process output - process_input: Send input to process stdin Add MCP tools for WebSocket: - ws_start: Start WebSocket server for real-time streaming - ws_info: Get hub statistics (clients, channels) Update Service struct with optional process.Service and ws.Hub fields, new WithProcessService and WithWSHub options, getter methods, and Shutdown method for cleanup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(webview): add browser automation package via Chrome DevTools Protocol Add pkg/webview package for browser automation: - webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate - cdp.go: Chrome DevTools Protocol WebSocket client implementation - actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder - console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher - angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability Add MCP tools for webview: - webview_connect/disconnect: Connection management - webview_navigate: Page navigation - webview_click/type/query/wait: DOM interaction - webview_console: Console output capture - webview_eval: JavaScript execution - webview_screenshot: Screenshot capture Add documentation: - docs/mcp/angular-testing.md: Guide for Angular application testing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: document new packages and BugSETI application - Update CLAUDE.md with documentation for: - pkg/ws (WebSocket hub for real-time streaming) - pkg/webview (Browser automation via CDP) - pkg/mcp (MCP server tools: process, ws, webview) - BugSETI application overview - Add comprehensive README for BugSETI with: - Installation and configuration guide - Usage workflow documentation - Architecture overview - Contributing guidelines Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(bugseti): add BugSETI system tray app with auto-update BugSETI - Distributed Bug Fixing like SETI@home but for code Features: - System tray app with Wails v3 - GitHub issue fetching with label filters - Issue queue with priority management - AI context seeding via seed-agent-developer skill - Automated PR submission flow - Stats tracking and leaderboard - Cross-platform notifications - Self-updating with stable/beta/nightly channels Includes: - cmd/bugseti: Main application with Angular frontend - internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify) - internal/bugseti/updater: Auto-update system (checker, downloader, installer) - .github/workflows/bugseti-release.yml: CI/CD for all platforms Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve import cycle and code duplication - Remove pkg/log import from pkg/io/local to break import cycle (pkg/log/rotation.go imports pkg/io, creating circular dependency) - Use stderr logging for security events in sandbox escape detection - Remove unused sync/atomic import from core.go - Fix duplicate LogSecurity function declarations in cli/log.go - Update workspace/service.go Crypt() call to match interface Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: update tests for new function signatures and format code - Update core_test.go: Config(), Display() now panic instead of returning error - Update runtime_pkg_test.go: sr.Config() now panics instead of returning error - Update MustServiceFor tests to use assert.Panics - Format BugSETI, MCP tools, and webview packages with gofmt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Snider <631881+Snider@users.noreply.github.com> Co-authored-by: Claude <developers@lethean.io> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
430 lines
12 KiB
Go
430 lines
12 KiB
Go
package gocmd
|
|
|
|
import (
|
|
"bufio"
|
|
"errors"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"regexp"
|
|
"strconv"
|
|
"strings"
|
|
|
|
"github.com/host-uk/core/pkg/cli"
|
|
"github.com/host-uk/core/pkg/i18n"
|
|
)
|
|
|
|
var (
|
|
testCoverage bool
|
|
testPkg string
|
|
testRun string
|
|
testShort bool
|
|
testRace bool
|
|
testJSON bool
|
|
testVerbose bool
|
|
)
|
|
|
|
func addGoTestCommand(parent *cli.Command) {
|
|
testCmd := &cli.Command{
|
|
Use: "test",
|
|
Short: "Run Go tests",
|
|
Long: "Run Go tests with optional coverage, filtering, and race detection",
|
|
RunE: func(cmd *cli.Command, args []string) error {
|
|
return runGoTest(testCoverage, testPkg, testRun, testShort, testRace, testJSON, testVerbose)
|
|
},
|
|
}
|
|
|
|
testCmd.Flags().BoolVar(&testCoverage, "coverage", false, "Generate coverage report")
|
|
testCmd.Flags().StringVar(&testPkg, "pkg", "", "Package to test")
|
|
testCmd.Flags().StringVar(&testRun, "run", "", "Run only tests matching pattern")
|
|
testCmd.Flags().BoolVar(&testShort, "short", false, "Run only short tests")
|
|
testCmd.Flags().BoolVar(&testRace, "race", false, "Enable race detector")
|
|
testCmd.Flags().BoolVar(&testJSON, "json", false, "Output as JSON")
|
|
testCmd.Flags().BoolVarP(&testVerbose, "verbose", "v", false, "Verbose output")
|
|
|
|
parent.AddCommand(testCmd)
|
|
}
|
|
|
|
func runGoTest(coverage bool, pkg, run string, short, race, jsonOut, verbose bool) error {
|
|
if pkg == "" {
|
|
pkg = "./..."
|
|
}
|
|
|
|
args := []string{"test"}
|
|
|
|
var covPath string
|
|
if coverage {
|
|
args = append(args, "-cover", "-covermode=atomic")
|
|
covFile, err := os.CreateTemp("", "coverage-*.out")
|
|
if err == nil {
|
|
covPath = covFile.Name()
|
|
_ = covFile.Close()
|
|
args = append(args, "-coverprofile="+covPath)
|
|
defer os.Remove(covPath)
|
|
}
|
|
}
|
|
|
|
if run != "" {
|
|
args = append(args, "-run", run)
|
|
}
|
|
if short {
|
|
args = append(args, "-short")
|
|
}
|
|
if race {
|
|
args = append(args, "-race")
|
|
}
|
|
if verbose {
|
|
args = append(args, "-v")
|
|
}
|
|
|
|
args = append(args, pkg)
|
|
|
|
if !jsonOut {
|
|
cli.Print("%s %s\n", dimStyle.Render(i18n.Label("test")), i18n.ProgressSubject("run", "tests"))
|
|
cli.Print(" %s %s\n", dimStyle.Render(i18n.Label("package")), pkg)
|
|
cli.Blank()
|
|
}
|
|
|
|
cmd := exec.Command("go", args...)
|
|
cmd.Env = append(os.Environ(), "MACOSX_DEPLOYMENT_TARGET=26.0", "CGO_ENABLED=0")
|
|
cmd.Dir, _ = os.Getwd()
|
|
|
|
output, err := cmd.CombinedOutput()
|
|
outputStr := string(output)
|
|
|
|
// Filter linker warnings
|
|
lines := strings.Split(outputStr, "\n")
|
|
var filtered []string
|
|
for _, line := range lines {
|
|
if !strings.Contains(line, "ld: warning:") {
|
|
filtered = append(filtered, line)
|
|
}
|
|
}
|
|
outputStr = strings.Join(filtered, "\n")
|
|
|
|
// Parse results
|
|
passed, failed, skipped := parseTestResults(outputStr)
|
|
cov := parseOverallCoverage(outputStr)
|
|
|
|
if jsonOut {
|
|
cli.Print(`{"passed":%d,"failed":%d,"skipped":%d,"coverage":%.1f,"exit_code":%d}`,
|
|
passed, failed, skipped, cov, cmd.ProcessState.ExitCode())
|
|
cli.Blank()
|
|
return err
|
|
}
|
|
|
|
// Print filtered output if verbose or failed
|
|
if verbose || err != nil {
|
|
cli.Text(outputStr)
|
|
}
|
|
|
|
// Summary
|
|
if err == nil {
|
|
cli.Print(" %s %s\n", successStyle.Render(cli.Glyph(":check:")), i18n.T("i18n.count.test", passed)+" "+i18n.T("i18n.done.pass"))
|
|
} else {
|
|
cli.Print(" %s %s, %s\n", errorStyle.Render(cli.Glyph(":cross:")),
|
|
i18n.T("i18n.count.test", passed)+" "+i18n.T("i18n.done.pass"),
|
|
i18n.T("i18n.count.test", failed)+" "+i18n.T("i18n.done.fail"))
|
|
}
|
|
|
|
if cov > 0 {
|
|
cli.Print("\n %s %s\n", cli.KeyStyle.Render(i18n.Label("statements")), formatCoverage(cov))
|
|
if covPath != "" {
|
|
branchCov, err := calculateBlockCoverage(covPath)
|
|
if err != nil {
|
|
cli.Print(" %s %s\n", cli.KeyStyle.Render(i18n.Label("branches")), cli.ErrorStyle.Render("unable to calculate"))
|
|
} else {
|
|
cli.Print(" %s %s\n", cli.KeyStyle.Render(i18n.Label("branches")), formatCoverage(branchCov))
|
|
}
|
|
}
|
|
}
|
|
|
|
if err == nil {
|
|
cli.Print("\n%s\n", successStyle.Render(i18n.T("i18n.done.pass")))
|
|
} else {
|
|
cli.Print("\n%s\n", errorStyle.Render(i18n.T("i18n.done.fail")))
|
|
}
|
|
|
|
return err
|
|
}
|
|
|
|
func parseTestResults(output string) (passed, failed, skipped int) {
|
|
passRe := regexp.MustCompile(`(?m)^ok\s+`)
|
|
failRe := regexp.MustCompile(`(?m)^FAIL\s+`)
|
|
skipRe := regexp.MustCompile(`(?m)^\?\s+`)
|
|
|
|
passed = len(passRe.FindAllString(output, -1))
|
|
failed = len(failRe.FindAllString(output, -1))
|
|
skipped = len(skipRe.FindAllString(output, -1))
|
|
return
|
|
}
|
|
|
|
func parseOverallCoverage(output string) float64 {
|
|
re := regexp.MustCompile(`coverage:\s+([\d.]+)%`)
|
|
matches := re.FindAllStringSubmatch(output, -1)
|
|
if len(matches) == 0 {
|
|
return 0
|
|
}
|
|
|
|
var total float64
|
|
for _, m := range matches {
|
|
var cov float64
|
|
_, _ = fmt.Sscanf(m[1], "%f", &cov)
|
|
total += cov
|
|
}
|
|
return total / float64(len(matches))
|
|
}
|
|
|
|
var (
|
|
covPkg string
|
|
covHTML bool
|
|
covOpen bool
|
|
covThreshold float64
|
|
covBranchThreshold float64
|
|
covOutput string
|
|
)
|
|
|
|
func addGoCovCommand(parent *cli.Command) {
|
|
covCmd := &cli.Command{
|
|
Use: "cov",
|
|
Short: "Run tests with coverage report",
|
|
Long: "Run tests with detailed coverage reports, HTML output, and threshold checking",
|
|
RunE: func(cmd *cli.Command, args []string) error {
|
|
pkg := covPkg
|
|
if pkg == "" {
|
|
// Auto-discover packages with tests
|
|
pkgs, err := findTestPackages(".")
|
|
if err != nil {
|
|
return cli.Wrap(err, i18n.T("i18n.fail.find", "test packages"))
|
|
}
|
|
if len(pkgs) == 0 {
|
|
return errors.New("no test packages found")
|
|
}
|
|
pkg = strings.Join(pkgs, " ")
|
|
}
|
|
|
|
// Create temp file for coverage data
|
|
covFile, err := os.CreateTemp("", "coverage-*.out")
|
|
if err != nil {
|
|
return cli.Wrap(err, i18n.T("i18n.fail.create", "coverage file"))
|
|
}
|
|
covPath := covFile.Name()
|
|
_ = covFile.Close()
|
|
defer func() {
|
|
if covOutput == "" {
|
|
_ = os.Remove(covPath)
|
|
} else {
|
|
// Copy to output destination before removing
|
|
src, _ := os.Open(covPath)
|
|
dst, _ := os.Create(covOutput)
|
|
if src != nil && dst != nil {
|
|
_, _ = io.Copy(dst, src)
|
|
_ = src.Close()
|
|
_ = dst.Close()
|
|
}
|
|
_ = os.Remove(covPath)
|
|
}
|
|
}()
|
|
|
|
cli.Print("%s %s\n", dimStyle.Render(i18n.Label("coverage")), i18n.ProgressSubject("run", "tests"))
|
|
// Truncate package list if too long for display
|
|
displayPkg := pkg
|
|
if len(displayPkg) > 60 {
|
|
displayPkg = displayPkg[:57] + "..."
|
|
}
|
|
cli.Print(" %s %s\n", dimStyle.Render(i18n.Label("package")), displayPkg)
|
|
cli.Blank()
|
|
|
|
// Run tests with coverage
|
|
// We need to split pkg into individual arguments if it contains spaces
|
|
pkgArgs := strings.Fields(pkg)
|
|
cmdArgs := append([]string{"test", "-coverprofile=" + covPath, "-covermode=atomic"}, pkgArgs...)
|
|
|
|
goCmd := exec.Command("go", cmdArgs...)
|
|
goCmd.Env = append(os.Environ(), "MACOSX_DEPLOYMENT_TARGET=26.0")
|
|
goCmd.Stdout = os.Stdout
|
|
goCmd.Stderr = os.Stderr
|
|
|
|
testErr := goCmd.Run()
|
|
|
|
// Get coverage percentage
|
|
coverCmd := exec.Command("go", "tool", "cover", "-func="+covPath)
|
|
covOutput, err := coverCmd.Output()
|
|
if err != nil {
|
|
if testErr != nil {
|
|
return testErr
|
|
}
|
|
return cli.Wrap(err, i18n.T("i18n.fail.get", "coverage"))
|
|
}
|
|
|
|
// Parse total coverage from last line
|
|
lines := strings.Split(strings.TrimSpace(string(covOutput)), "\n")
|
|
var statementCov float64
|
|
if len(lines) > 0 {
|
|
lastLine := lines[len(lines)-1]
|
|
// Format: "total: (statements) XX.X%"
|
|
if strings.Contains(lastLine, "total:") {
|
|
parts := strings.Fields(lastLine)
|
|
if len(parts) >= 3 {
|
|
covStr := strings.TrimSuffix(parts[len(parts)-1], "%")
|
|
_, _ = fmt.Sscanf(covStr, "%f", &statementCov)
|
|
}
|
|
}
|
|
}
|
|
|
|
// Calculate branch coverage (block coverage)
|
|
branchCov, err := calculateBlockCoverage(covPath)
|
|
if err != nil {
|
|
return cli.Wrap(err, "calculate branch coverage")
|
|
}
|
|
|
|
// Print coverage summary
|
|
cli.Blank()
|
|
cli.Print(" %s %s\n", cli.KeyStyle.Render(i18n.Label("statements")), formatCoverage(statementCov))
|
|
cli.Print(" %s %s\n", cli.KeyStyle.Render(i18n.Label("branches")), formatCoverage(branchCov))
|
|
|
|
// Generate HTML if requested
|
|
if covHTML || covOpen {
|
|
htmlPath := "coverage.html"
|
|
htmlCmd := exec.Command("go", "tool", "cover", "-html="+covPath, "-o="+htmlPath)
|
|
if err := htmlCmd.Run(); err != nil {
|
|
return cli.Wrap(err, i18n.T("i18n.fail.generate", "HTML"))
|
|
}
|
|
cli.Print(" %s %s\n", dimStyle.Render(i18n.Label("html")), htmlPath)
|
|
|
|
if covOpen {
|
|
// Open in browser
|
|
var openCmd *exec.Cmd
|
|
switch {
|
|
case exec.Command("which", "open").Run() == nil:
|
|
openCmd = exec.Command("open", htmlPath)
|
|
case exec.Command("which", "xdg-open").Run() == nil:
|
|
openCmd = exec.Command("xdg-open", htmlPath)
|
|
default:
|
|
cli.Print(" %s\n", dimStyle.Render("Open coverage.html in your browser"))
|
|
}
|
|
if openCmd != nil {
|
|
_ = openCmd.Run()
|
|
}
|
|
}
|
|
}
|
|
|
|
// Check thresholds
|
|
if covThreshold > 0 && statementCov < covThreshold {
|
|
cli.Print("\n%s Statements: %.1f%% < %.1f%%\n", errorStyle.Render(i18n.T("i18n.fail.meet", "threshold")), statementCov, covThreshold)
|
|
return errors.New("statement coverage below threshold")
|
|
}
|
|
if covBranchThreshold > 0 && branchCov < covBranchThreshold {
|
|
cli.Print("\n%s Branches: %.1f%% < %.1f%%\n", errorStyle.Render(i18n.T("i18n.fail.meet", "threshold")), branchCov, covBranchThreshold)
|
|
return errors.New("branch coverage below threshold")
|
|
}
|
|
|
|
if testErr != nil {
|
|
return testErr
|
|
}
|
|
|
|
cli.Print("\n%s\n", successStyle.Render(i18n.T("i18n.done.pass")))
|
|
return nil
|
|
},
|
|
}
|
|
|
|
covCmd.Flags().StringVar(&covPkg, "pkg", "", "Package to test")
|
|
covCmd.Flags().BoolVar(&covHTML, "html", false, "Generate HTML report")
|
|
covCmd.Flags().BoolVar(&covOpen, "open", false, "Open HTML report in browser")
|
|
covCmd.Flags().Float64Var(&covThreshold, "threshold", 0, "Minimum statement coverage percentage")
|
|
covCmd.Flags().Float64Var(&covBranchThreshold, "branch-threshold", 0, "Minimum branch coverage percentage")
|
|
covCmd.Flags().StringVarP(&covOutput, "output", "o", "", "Output file for coverage profile")
|
|
|
|
parent.AddCommand(covCmd)
|
|
}
|
|
|
|
// calculateBlockCoverage parses a Go coverage profile and returns the percentage of basic
|
|
// blocks that have a non-zero execution count. Go's coverage profile contains one line per
|
|
// basic block, where the last field is the execution count, not explicit branch coverage.
|
|
// The resulting block coverage is used here only as a proxy for branch coverage; computing
|
|
// true branch coverage would require more detailed control-flow analysis.
|
|
func calculateBlockCoverage(path string) (float64, error) {
|
|
file, err := os.Open(path)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
defer file.Close()
|
|
|
|
scanner := bufio.NewScanner(file)
|
|
var totalBlocks, coveredBlocks int
|
|
|
|
// Skip the first line (mode: atomic/set/count)
|
|
if !scanner.Scan() {
|
|
return 0, nil
|
|
}
|
|
|
|
for scanner.Scan() {
|
|
line := scanner.Text()
|
|
if line == "" {
|
|
continue
|
|
}
|
|
fields := strings.Fields(line)
|
|
if len(fields) < 3 {
|
|
continue
|
|
}
|
|
|
|
// Last field is the count
|
|
count, err := strconv.Atoi(fields[len(fields)-1])
|
|
if err != nil {
|
|
continue
|
|
}
|
|
|
|
totalBlocks++
|
|
if count > 0 {
|
|
coveredBlocks++
|
|
}
|
|
}
|
|
|
|
if err := scanner.Err(); err != nil {
|
|
return 0, err
|
|
}
|
|
|
|
if totalBlocks == 0 {
|
|
return 0, nil
|
|
}
|
|
|
|
return (float64(coveredBlocks) / float64(totalBlocks)) * 100, nil
|
|
}
|
|
|
|
func findTestPackages(root string) ([]string, error) {
|
|
pkgMap := make(map[string]bool)
|
|
err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
|
|
if err != nil {
|
|
return nil
|
|
}
|
|
if !info.IsDir() && strings.HasSuffix(info.Name(), "_test.go") {
|
|
dir := filepath.Dir(path)
|
|
if !strings.HasPrefix(dir, ".") {
|
|
dir = "./" + dir
|
|
}
|
|
pkgMap[dir] = true
|
|
}
|
|
return nil
|
|
})
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var pkgs []string
|
|
for pkg := range pkgMap {
|
|
pkgs = append(pkgs, pkg)
|
|
}
|
|
return pkgs, nil
|
|
}
|
|
|
|
func formatCoverage(cov float64) string {
|
|
s := fmt.Sprintf("%.1f%%", cov)
|
|
if cov >= 80 {
|
|
return cli.SuccessStyle.Render(s)
|
|
} else if cov >= 50 {
|
|
return cli.WarningStyle.Render(s)
|
|
}
|
|
return cli.ErrorStyle.Render(s)
|
|
}
|