Commit graph

8 commits

Author SHA1 Message Date
Snider
315b4fc052 Merge branch 'fix/consolidate-workflows' into new 2026-02-08 22:00:21 +00:00
Vi
4debdc1449 feat: BugSETI app, WebSocket hub, browser automation, and MCP tools (#336)
* feat: add security logging and fix framework regressions

This commit implements comprehensive security event logging and resolves critical regressions in the core framework.

Security Logging:
- Enhanced `pkg/log` with a `Security` level and helper.
- Added `log.Username()` to consistently identify the executing user.
- Instrumented GitHub CLI auth, Agentic configuration, filesystem sandbox, MCP handlers, and MCP TCP transport with security logs.
- Added `SecurityStyle` to the CLI for consistent visual representation of security events.

UniFi Security (CodeQL):
- Refactored `pkg/unifi` to remove hardcoded `InsecureSkipVerify`, resolving a high-severity alert.
- Added a `--verify-tls` flag and configuration option to control TLS verification.
- Updated command handlers to support the new verification parameter.

Framework Fixes:
- Restored original signatures for `MustServiceFor`, `Config()`, and `Display()` in `pkg/framework/core`, which had been corrupted during a merge.
- Fixed `pkg/framework/framework.go` and `pkg/framework/core/runtime_pkg.go` to match the restored signatures.
- These fixes resolve project-wide compilation errors caused by the signature mismatches.

I encountered significant blockers due to a corrupted state of the `dev` branch after a merge, which introduced breaking changes in the core framework's DI system. I had to manually reconcile these signatures with the expected usage across the codebase to restore build stability.

* feat(mcp): add RAG tools (query, ingest, collections)

Add vector database tools to the MCP server for RAG operations:
- rag_query: Search for relevant documentation using semantic similarity
- rag_ingest: Ingest files or directories into the vector database
- rag_collections: List available collections

Uses existing internal/cmd/rag exports (QueryDocs, IngestDirectory, IngestFile)
and pkg/rag for Qdrant client access. Default collection is "hostuk-docs"
with topK=5 for queries.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(mcp): add metrics tools (record, query)

Add MCP tools for recording and querying AI/security metrics events.
The metrics_record tool writes events to daily JSONL files, and the
metrics_query tool provides aggregated statistics by type, repo, and agent.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add 'core mcp serve' command

Add CLI command to start the MCP server for AI tool integration.

- Create internal/cmd/mcpcmd package with serve subcommand
- Support --workspace flag for directory restriction
- Handle SIGINT/SIGTERM for clean shutdown
- Register in full.go build variant

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(ws): add WebSocket hub package for real-time streaming

Add pkg/ws package implementing a hub pattern for WebSocket connections:
- Hub manages client connections, broadcasts, and channel subscriptions
- Client struct represents connected WebSocket clients
- Message types: process_output, process_status, event, error, ping/pong
- Channel-based subscription system (subscribe/unsubscribe)
- SendProcessOutput and SendProcessStatus for process streaming integration
- Full test coverage including concurrency tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(mcp): add process management and WebSocket MCP tools

Add MCP tools for process management:
- process_start: Start a new external process
- process_stop: Gracefully stop a running process
- process_kill: Force kill a process
- process_list: List all managed processes
- process_output: Get captured process output
- process_input: Send input to process stdin

Add MCP tools for WebSocket:
- ws_start: Start WebSocket server for real-time streaming
- ws_info: Get hub statistics (clients, channels)

Update Service struct with optional process.Service and ws.Hub fields,
new WithProcessService and WithWSHub options, getter methods, and
Shutdown method for cleanup.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(webview): add browser automation package via Chrome DevTools Protocol

Add pkg/webview package for browser automation:
- webview.go: Main interface with Connect, Navigate, Click, Type, QuerySelector, Screenshot, Evaluate
- cdp.go: Chrome DevTools Protocol WebSocket client implementation
- actions.go: DOM action types (Click, Type, Hover, Scroll, etc.) and ActionSequence builder
- console.go: Console message capture and filtering with ConsoleWatcher and ExceptionWatcher
- angular.go: Angular-specific helpers for router navigation, component access, and Zone.js stability

Add MCP tools for webview:
- webview_connect/disconnect: Connection management
- webview_navigate: Page navigation
- webview_click/type/query/wait: DOM interaction
- webview_console: Console output capture
- webview_eval: JavaScript execution
- webview_screenshot: Screenshot capture

Add documentation:
- docs/mcp/angular-testing.md: Guide for Angular application testing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: document new packages and BugSETI application

- Update CLAUDE.md with documentation for:
  - pkg/ws (WebSocket hub for real-time streaming)
  - pkg/webview (Browser automation via CDP)
  - pkg/mcp (MCP server tools: process, ws, webview)
  - BugSETI application overview
- Add comprehensive README for BugSETI with:
  - Installation and configuration guide
  - Usage workflow documentation
  - Architecture overview
  - Contributing guidelines

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(bugseti): add BugSETI system tray app with auto-update

BugSETI - Distributed Bug Fixing like SETI@home but for code

Features:
- System tray app with Wails v3
- GitHub issue fetching with label filters
- Issue queue with priority management
- AI context seeding via seed-agent-developer skill
- Automated PR submission flow
- Stats tracking and leaderboard
- Cross-platform notifications
- Self-updating with stable/beta/nightly channels

Includes:
- cmd/bugseti: Main application with Angular frontend
- internal/bugseti: Core services (fetcher, queue, seeder, submit, config, stats, notify)
- internal/bugseti/updater: Auto-update system (checker, downloader, installer)
- .github/workflows/bugseti-release.yml: CI/CD for all platforms

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve import cycle and code duplication

- Remove pkg/log import from pkg/io/local to break import cycle
  (pkg/log/rotation.go imports pkg/io, creating circular dependency)
- Use stderr logging for security events in sandbox escape detection
- Remove unused sync/atomic import from core.go
- Fix duplicate LogSecurity function declarations in cli/log.go
- Update workspace/service.go Crypt() call to match interface

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update tests for new function signatures and format code

- Update core_test.go: Config(), Display() now panic instead of returning error
- Update runtime_pkg_test.go: sr.Config() now panics instead of returning error
- Update MustServiceFor tests to use assert.Panics
- Format BugSETI, MCP tools, and webview packages with gofmt

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
Co-authored-by: Claude <developers@lethean.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 17:22:05 +00:00
Snider
6d65b70e0c Add streaming API to pkg/io and optimize agentic context gathering (#313)
* feat(io): add streaming API to Medium interface and optimize agentic context

- Added ReadStream and WriteStream to io.Medium interface.
- Implemented streaming methods in local and mock mediums.
- Updated pkg/agentic/context.go to use streaming I/O with LimitReader.
- Added 5000-byte truncation limit for all AI context file reads to reduce memory usage.
- Documented when to use streaming vs full-file APIs in io.Medium.

* feat(io): optimize streaming API and fix PR feedback

- Fixed resource leak in agentic context by using defer for closing file streams.
- Improved truncation logic in agentic context to handle multibyte characters correctly by checking byte length before string conversion.
- Added comprehensive documentation to ReadStream and WriteStream in local medium.
- Added unit tests for ReadStream and WriteStream in local medium.
- Applied formatting and fixed auto-merge CI configuration.

* feat(io): add streaming API and fix CI failures (syntax fix)

- Introduced ReadStream and WriteStream to io.Medium interface.
- Implemented streaming methods in local and mock mediums.
- Optimized agentic context with streaming reads and truncation logic.
- Fixed syntax error in local client tests by overwriting the file.
- Fixed auto-merge CI by adding checkout and repository context.
- Applied formatting fixes.
2026-02-05 11:00:49 +00:00
Snider
9f4007c409 Remove StrictHostKeyChecking=no from SSH commands (#315)
* Remove StrictHostKeyChecking=no and implement proper host key verification

This commit addresses security concerns from the OWASP audit by enforcing
strict host key verification for all SSH and SCP commands.

Key changes:
- Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and pkg/devops.
- Removed insecure host key verification from pkg/ansible SSH client.
- Implemented a synchronous host key discovery mechanism during VM boot
  using ssh-keyscan to populate ~/.core/known_hosts.
- Updated the devops Boot lifecycle to wait until the host key is verified.
- Ensured pkg/ansible correctly handles missing known_hosts files.
- Refactored hardcoded SSH port 2222 to a package constant DefaultSSHPort.
- Added CORE_SKIP_SSH_SCAN environment variable for test environments.

* Remove StrictHostKeyChecking=no and implement proper host key verification

Addresses security concerns from OWASP audit by enforcing strict host key
verification.

Changes:
- Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and devops.
- Removed insecure host key verification from pkg/ansible.
- Added synchronous host key discovery using ssh-keyscan during VM boot.
- Updated Boot lifecycle to wait for host key verification.
- Handled missing known_hosts file in pkg/ansible.
- Refactored hardcoded SSH port to DefaultSSHPort constant.
- Fixed formatting issues identified by QA check.

* Secure SSH commands and fix auto-merge CI failure

Addresses OWASP security audit by enforcing strict host key verification
and fixes a CI failure in the auto-merge workflow.

Key changes:
- Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and pkg/devops.
- Removed insecure host key verification from pkg/ansible.
- Implemented synchronous host key discovery using ssh-keyscan during VM boot.
- Handled missing known_hosts file in pkg/ansible.
- Refactored hardcoded SSH port to DefaultSSHPort constant.
- Added pkg/ansible/ssh_test.go to verify SSH client initialization.
- Fixed formatting in pkg/io/local/client.go.
- Fixed auto-merge.yml by inlining the script and providing repository context
  to 'gh' command, resolving the "not a git repository" error in CI.

* Secure SSH, fix CI auto-merge, and resolve merge conflicts

This commit addresses the OWASP security audit by enforcing strict host key
verification and resolves persistent CI issues.

Security Changes:
- Replaced StrictHostKeyChecking=accept-new with yes in pkg/container and devops.
- Removed insecure host key verification from pkg/ansible.
- Implemented synchronous host key discovery using ssh-keyscan during VM boot.
- Updated Boot lifecycle to wait for host key verification.
- Handled missing known_hosts file in pkg/ansible.
- Refactored hardcoded SSH port to DefaultSSHPort constant.

CI and Maintenance:
- Fixed auto-merge.yml by inlining the script and adding repository context
  to 'gh' command, resolving the "not a git repository" error in CI.
- Resolved merge conflicts in .github/workflows/auto-merge.yml with dev branch.
- Added pkg/ansible/ssh_test.go for SSH client verification.
- Fixed formatting in pkg/io/local/client.go to pass QA checks.

* Secure SSH and TLS connections, and fix CI issues

Addresses security concerns from OWASP audit and CodeQL by enforcing strict
host key verification and TLS certificate verification.

Security Changes:
- Enforced strict SSH host key checking in pkg/container and devops.
- Removed insecure SSH host key verification from pkg/ansible.
- Added synchronous host key discovery during VM boot using ssh-keyscan.
- Updated UniFi client to enforce TLS certificate verification by default.
- Added --insecure flag and config option for UniFi to allow opt-in to
  skipping TLS verification for self-signed certificates.

CI and Maintenance:
- Fixed auto-merge workflow by providing repository context to 'gh' command.
- Resolved merge conflicts in .github/workflows/auto-merge.yml.
- Added unit tests for secured Ansible SSH client.
- Fixed formatting issues identified by QA checks.

* fix: gofmt alignment in cmd_config.go

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Secure connections, fix CI auto-merge, and resolve formatting

Addresses OWASP security audit and CodeQL security alerts by enforcing
secure defaults for SSH and TLS connections.

Key changes:
- Enforced strict SSH host key checking (StrictHostKeyChecking=yes).
- Implemented synchronous host key verification during VM boot using ssh-keyscan.
- Updated UniFi client to enforce TLS certificate verification by default.
- Added --insecure flag and config option for UniFi to allow opt-in to
  skipping TLS verification.
- Fixed auto-merge workflow by providing repository context to 'gh' command.
- Resolved merge conflicts in .github/workflows/auto-merge.yml.
- Fixed formatting in internal/cmd/unifi/cmd_config.go and pkg/io/local/client.go.
- Added unit tests for secured Ansible SSH client.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Claude <developers@lethean.io>
2026-02-05 07:09:34 +00:00
Snider
f2bc912ebe feat: infrastructure packages and lint cleanup (#281)
* ci: consolidate duplicate workflows and merge CodeQL configs

Remove 17 duplicate workflow files that were split copies of the
combined originals. Each family (CI, CodeQL, Coverage, PR Build,
Alpha Release) had the same job duplicated across separate
push/pull_request/schedule/manual trigger files.

Merge codeql.yml and codescan.yml into a single codeql.yml with
a language matrix covering go, javascript-typescript, python,
and actions — matching the previous default setup coverage.

Remaining workflows (one per family):
- ci.yml (push + PR + manual)
- codeql.yml (push + PR + schedule, all languages)
- coverage.yml (push + PR + manual)
- alpha-release.yml (push + manual)
- pr-build.yml (PR + manual)
- release.yml (tag push)
- agent-verify.yml, auto-label.yml, auto-project.yml

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add collect, config, crypt, plugin packages and fix all lint issues

Add four new infrastructure packages with CLI commands:
- pkg/config: layered configuration (defaults → file → env → flags)
- pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums)
- pkg/plugin: plugin system with GitHub-based install/update/remove
- pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate)

Fix all golangci-lint issues across the entire codebase (~100 errcheck,
staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that
`core go qa` passes with 0 issues.

Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:34:43 +00:00
Snider
32a3613a3a feat: add collect, config, crypt, plugin packages and fix all lint issues
Add four new infrastructure packages with CLI commands:
- pkg/config: layered configuration (defaults → file → env → flags)
- pkg/crypt: crypto primitives (Argon2id, AES-GCM, ChaCha20, HMAC, checksums)
- pkg/plugin: plugin system with GitHub-based install/update/remove
- pkg/collect: collection subsystem (GitHub, BitcoinTalk, market, papers, excavate)

Fix all golangci-lint issues across the entire codebase (~100 errcheck,
staticcheck SA1012/SA1019/ST1005, unused, ineffassign fixes) so that
`core go qa` passes with 0 issues.

Closes #167, #168, #170, #250, #251, #252, #253, #254, #255, #256

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:32:41 +00:00
Snider
1d18339a97 docs(audit): add dependency security audit report (#248)
* feat(devops): migrate filesystem operations to io.Local abstraction

Migrate config.go:
- os.ReadFile → io.Local.Read

Migrate devops.go:
- os.Stat → io.Local.IsFile

Migrate images.go:
- os.MkdirAll → io.Local.EnsureDir
- os.Stat → io.Local.IsFile
- os.ReadFile → io.Local.Read
- os.WriteFile → io.Local.Write

Migrate test.go:
- os.ReadFile → io.Local.Read
- os.Stat → io.Local.IsFile

Migrate claude.go:
- os.Stat → io.Local.IsDir

Updated tests to reflect improved behavior:
- Manifest.Save() now creates parent directories
- hasFile() correctly returns false for directories

Part of #101 (io.Medium migration tracking issue).

Closes #107

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(io): migrate remaining packages to io.Local abstraction

Migrate filesystem operations to use the io.Local abstraction for
improved security, testability, and consistency:

- pkg/cache: Replace os.ReadFile, WriteFile, Remove, RemoveAll with
  io.Local equivalents. io.Local.Write creates parent dirs automatically.
- pkg/agentic: Migrate config.go and context.go to use io.Local for
  reading config files and gathering file context.
- pkg/repos: Use io.Local.Read, Exists, IsDir, List for registry
  operations and git repo detection.
- pkg/release: Use io.Local for config loading, existence checks,
  and artifact discovery.
- pkg/devops/sources: Use io.Local.EnsureDir for CDN download.

All paths are converted to absolute using filepath.Abs() before
calling io.Local methods to handle relative paths correctly.

Closes #104, closes #106, closes #108, closes #111

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(io): migrate pkg/cli and pkg/container to io.Local abstraction

Continue io.Medium migration for the remaining packages:

- pkg/cli/daemon.go: PIDFile Acquire/Release now use io.Local.Read,
  Delete, and Write for managing daemon PID files.
- pkg/container/state.go: LoadState and SaveState use io.Local for
  JSON state persistence. EnsureLogsDir uses io.Local.EnsureDir.
- pkg/container/templates.go: Template loading and directory scanning
  now use io.Local.IsFile, IsDir, Read, and List.
- pkg/container/linuxkit.go: Image validation uses io.Local.IsFile,
  log file check uses io.Local.IsFile. Streaming log file creation
  (os.Create) remains unchanged as io.Local doesn't support streaming.

Closes #105, closes #107

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs(audit): add dependency security audit report

Complete security audit of all project dependencies:

- Run govulncheck: No vulnerabilities found
- Run go mod verify: All modules verified
- Document 15 direct dependencies and 161 indirect
- Assess supply chain risks: Low risk overall
- Verify lock files are committed with integrity hashes
- Provide CI integration recommendations

Closes #185

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(ci): build core CLI from source instead of downloading release

The workflows were trying to download from a non-existent release URL.
Now builds the CLI directly using `go build` with version injection.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: trigger CI with updated workflow

* chore(ci): add workflow_dispatch trigger for manual runs

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 08:04:26 +00:00
Snider
c00b374147 feat(devops): add Claude sandbox session
Starts Claude in immutable dev environment with auth forwarding.
Auto-boots VM, mounts project, forwards credentials.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 02:17:34 +00:00