Commit graph

7 commits

Author SHA1 Message Date
Snider
26b47ee073
Migrate pkg/build to io.Medium abstraction (#287)
* chore(io): Migrate pkg/build to Medium abstraction

- Updated io.Medium interface with Open() and Create() methods to support streaming.
- Migrated pkg/build, pkg/build/builders, and pkg/build/signing to use io.Medium.
- Added FS field to build.Config and updated build.Builder interface.
- Refactored checksum and archive logic to use io.Medium streaming.
- Updated pkg/release and pkg/build/buildcmd to use io.Local.
- Updated unit tests to match new signatures.

* chore(io): Migrate pkg/build to Medium abstraction (fix CI)

- Fixed formatting in pkg/build/builders/wails.go.
- Fixed TestLoadConfig_Testdata and TestDiscover_Testdata to use absolute paths with io.Local to ensure compatibility with GitHub CI.
- Verified that all build and release tests pass.

* chore(io): Migrate pkg/build to Medium abstraction (fix CI paths)

- Ensured that outputDir and configPath are absolute in runProjectBuild.
- Fixed TestLoadConfig_Testdata and TestDiscover_Testdata to use absolute paths correctly.
- Verified that all build and release tests pass locally.

* chore(io): Migrate pkg/build to Medium abstraction (final fix)

- Improved io.Local to handle relative paths relative to CWD when rooted at "/".
- This makes io.Local a drop-in replacement for the 'os' package for most use cases.
- Ensured absolute paths are used in build logic and tests where appropriate.
- Fixed formatting and cleaned up debug prints.

* chore(io): address code review and fix CI

- Fix MockFile.Read to return io.EOF
- Use filepath.Match in TaskfileBuilder for precise globbing
- Stream xz data in createTarXzArchive to avoid in-memory string conversion
- Fix TestPath_RootFilesystem in local medium tests
- Fix formatting in pkg/build/buildcmd/cmd_project.go

* chore(io): resolve merge conflicts and final migration of pkg/build

- Resolved merge conflicts in pkg/io/io.go, pkg/io/local/client.go, and pkg/release/release.go.
- Reconciled io.Medium interface with upstream changes (unifying to fs.File for Open).
- Integrated upstream validatePath logic into the local medium.
- Completed migration of pkg/build and related packages to io.Medium.
- Addressed previous code review feedback on MockMedium and TaskfileBuilder.

* chore(io): resolve merge conflicts and finalize migration

- Resolved merge conflicts with dev branch.
- Unified io.Medium interface (Open returns fs.File, Create returns io.WriteCloser).
- Integrated upstream validatePath logic.
- Ensured all tests pass across pkg/io, pkg/build, and pkg/release.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 17:59:10 +00:00
Snider
7be325302f
Migrate pkg/release to io.Medium abstraction (#290)
* chore(io): migrate pkg/release to io.Medium abstraction

Migrated `pkg/release` and its subpackages to use the `io.Medium` abstraction for filesystem operations. This enables better testability and support for alternative storage backends.

Changes:
- Added `FS io.Medium` field to `release.Release` and `publishers.Release` structs.
- Updated `LoadConfig`, `ConfigExists`, and `WriteConfig` in `pkg/release/config.go` to accept `io.Medium`.
- Updated `Publish`, `Run`, `findArtifacts`, and `buildArtifacts` in `pkg/release/release.go` to use `io.Medium`.
- Migrated all publishers (`aur`, `chocolatey`, `docker`, `github`, `homebrew`, `linuxkit`, `npm`, `scoop`) to use `io.Medium` for file operations.
- Implemented custom template overrides in publishers by checking for templates in `.core/templates/<publisher>/` via `io.Medium`.
- Updated all relevant tests to provide `io.Medium`.

* chore(io): fix missing callers in pkg/release migration

Updated callers of `release` package functions that had their signatures changed during the `io.Medium` migration.

Fixed files:
- `internal/cmd/ci/cmd_init.go`
- `internal/cmd/ci/cmd_publish.go`
- `pkg/build/buildcmd/cmd_release.go`

These changes ensure the project compiles successfully by providing `io.Local` to `LoadConfig`, `WriteConfig`, and `ConfigExists`.

* chore(io): fix build errors in pkg/release migration

Fixed compilation errors by updating all callers of `release.LoadConfig`, `release.ConfigExists`, and `release.WriteConfig` to provide the required `io.Medium` argument.

Files updated:
- `internal/cmd/ci/cmd_init.go`
- `internal/cmd/ci/cmd_publish.go`
- `pkg/build/buildcmd/cmd_release.go`

These entry points now correctly pass `io.Local` to the `release` package functions.
2026-02-04 15:07:13 +00:00
Snider
45648858a8
style: fix gofmt formatting across all affected files (#279)
Adds missing trailing newlines, fixes indentation alignment, removes
extra blank lines, and corrects import ordering. Fixes CI qa format
check failures blocking all open PRs.

Files fixed:
- pkg/rag/{ingest,ollama,qdrant,query}.go (missing trailing newline)
- internal/cmd/rag/cmd_ingest.go (extra blank lines)
- internal/cmd/security/cmd_jobs.go (var alignment)
- internal/cmd/security/cmd_security.go (extra blank line)
- internal/core-ide/claude_bridge.go (indentation)
- internal/variants/core_ide.go (import ordering)
- pkg/ansible/{modules,ssh}.go (whitespace)
- pkg/build/buildcmd/cmd_release.go (var alignment)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 01:27:01 +00:00
Snider
989b7e1e65
feat: wire release command, add tar.xz support, unified installers (#277)
* feat(cli): wire release command and add installer scripts

- Wire up `core build release` subcommand (was orphaned)
- Wire up `core monitor` command (missing import in full variant)
- Add installer scripts for Unix (.sh) and Windows (.bat)
  - setup: Interactive with variant selection
  - ci: Minimal for CI/CD environments
  - dev: Full development variant
  - go/php/agent: Targeted development variants
- All scripts include security hardening:
  - Secure temp directories (mktemp -d)
  - Architecture validation
  - Version validation after GitHub API call
  - Proper cleanup on exit
  - PowerShell PATH updates on Windows (avoids setx truncation)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(build): add tar.xz support and unified installer scripts

- Add tar.xz archive support using Borg's compress package
  - ArchiveXZ() and ArchiveWithFormat() for configurable compression
  - Better compression ratio than gzip for release artifacts
- Consolidate 12 installer scripts into 2 unified scripts
  - install.sh and install.bat with BunnyCDN edge variable support
  - Subdomains: setup.core.help, ci.core.help, dev.core.help, etc.
  - MODE and VARIANT transformed at edge based on subdomain
- Installers prefer tar.xz with automatic fallback to tar.gz
- Fixed CodeRabbit issues: HTTP status patterns, tar error handling,
  verify_install params, VARIANT validation, CI PATH persistence

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add build and release config files

- .core/build.yaml - cross-platform build configuration
- .core/release.yaml - release workflow configuration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: move plans from docs/ to tasks/

Consolidate planning documents in tasks/plans/ directory.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(install): address CodeRabbit review feedback

- Add curl timeout (--max-time) to prevent hanging on slow networks
- Rename TMPDIR to WORK_DIR to avoid clobbering system env var
- Add chmod +x to ensure binary has execute permissions
- Add error propagation after subroutine calls in batch file
- Remove System32 install attempt in CI mode (use consistent INSTALL_DIR)
- Fix HTTP status regex for HTTP/2 compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(rag): add Go RAG implementation with Qdrant + Ollama

Add RAG (Retrieval Augmented Generation) tools for storing documentation
in Qdrant vector database and querying with semantic search. This replaces
the Python tools/rag implementation with a native Go solution.

New commands:
- core rag ingest [directory] - Ingest markdown files into Qdrant
- core rag query [question] - Query vector database with semantic search
- core rag collections - List and manage Qdrant collections

Features:
- Markdown chunking by sections and paragraphs with overlap
- UTF-8 safe text handling for international content
- Automatic category detection from file paths
- Multiple output formats: text, JSON, LLM context injection
- Environment variable support for host configuration

Dependencies:
- github.com/qdrant/go-client (gRPC client)
- github.com/ollama/ollama/api (embeddings API)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(deploy): add pure-Go Ansible executor and Coolify API integration

Implement infrastructure deployment system with:

- pkg/ansible: Pure Go Ansible executor
  - Playbook/inventory parsing (types.go, parser.go)
  - Full execution engine with variable templating, loops, blocks,
    conditionals, handlers, and fact gathering (executor.go)
  - SSH client with key/password auth and privilege escalation (ssh.go)
  - 35+ module implementations: shell, command, copy, template, file,
    apt, service, systemd, user, group, git, docker_compose, etc. (modules.go)

- pkg/deploy/coolify: Coolify API client wrapping Python swagger client
  - List/get servers, projects, applications, databases, services
  - Generic Call() for any OpenAPI operation

- pkg/deploy/python: Embedded Python runtime for swagger client integration

- internal/cmd/deploy: CLI commands
  - core deploy servers/projects/apps/databases/services/team
  - core deploy call <operation> [params-json]

This enables Docker-free infrastructure deployment with Ansible-compatible
playbooks executed natively in Go.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): address linter warnings and build errors

- Fix fmt.Sprintf format verb error in ssh.go (remove unused stat command)
- Fix errcheck warnings by explicitly ignoring best-effort operations
- Fix ineffassign warning in cmd_ansible.go

All golangci-lint checks now pass for deploy packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style(deploy): fix gofmt formatting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(deploy): use known_hosts for SSH host key verification

Address CodeQL security alert by using the user's known_hosts file
for SSH host key verification when available. Falls back to accepting
any key only when known_hosts doesn't exist (common in containerized
or ephemeral environments).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(ai,security,ide): add agentic MVP, security jobs, and Core IDE desktop app

Wire up AI infrastructure with unified pkg/ai package (metrics JSONL,
RAG integration), move RAG under `core ai rag`, add `core ai metrics`
command, and enrich task context with Qdrant documentation.

Add `--target` flag to all security commands for external repo scanning,
`core security jobs` for distributing findings as GitHub Issues, and
consistent error logging across scan/deps/alerts/secrets commands.

Add Core IDE Wails v3 desktop app with Angular 20 frontend, MCP bridge
(loopback-only HTTP server), WebSocket hub, and Claude Code bridge.
Production-ready with Lethean CIC branding, macOS code signing support,
and security hardening (origin validation, body size limits, URL scheme
checks, memory leak prevention, XSS mitigation).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review comments from CodeRabbit, Copilot, and Gemini

Fixes across 25 files addressing 46+ review comments:

- pkg/ai/metrics.go: handle error from Close() on writable file handle
- pkg/ansible: restore loop vars after loop, restore become settings,
  fix Upload with become=true and no password (use sudo -n), honour
  SSH timeout config, use E() helper for contextual errors, quote git
  refs in checkout commands
- pkg/rag: validate chunk config, guard negative-to-uint64 conversion,
  use E() helper for errors, add context timeout to Ollama HTTP calls
- pkg/deploy/python: fix exec.ExitError type assertion (was os.PathError),
  handle os.UserHomeDir() error
- pkg/build/buildcmd: use cmd.Context() instead of context.Background()
  for proper Ctrl+C cancellation
- install.bat: add curl timeouts, CRLF line endings, use --connect-timeout
  for archive downloads
- install.sh: use absolute path for version check in CI mode
- tools/rag: fix broken ingest.py function def, escape HTML in query.py,
  pin qdrant-client version, add markdown code block languages
- internal/cmd/rag: add chunk size validation, env override handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(build): make release dry-run by default and remove darwin/amd64 target

Replace --dry-run (default false) with --we-are-go-for-launch (default
false) so `core build release` is safe by default. Remove darwin/amd64
from default build targets (arm64 only for macOS). Fix cmd_project.go
to use command context instead of context.Background().

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:49:57 +00:00
Snider
3365bfd5ba
feat(mcp): add workspace root validation to prevent path traversal (#100)
* feat(mcp): add workspace root validation to prevent path traversal

- Add workspaceRoot field to Service for restricting file operations
- Add WithWorkspaceRoot() option for configuring the workspace directory
- Add validatePath() helper to check paths are within workspace
- Apply validation to all file operation handlers
- Default to current working directory for security
- Add comprehensive tests for path validation

Closes #82

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: move CLI commands from pkg/ to internal/cmd/

- Move 18 CLI command packages to internal/cmd/ (not externally importable)
- Keep 16 library packages in pkg/ (externally importable)
- Update all import paths throughout codebase
- Cleaner separation between CLI logic and reusable libraries

CLI commands moved: ai, ci, dev, docs, doctor, gitcmd, go, monitor,
php, pkgcmd, qa, sdk, security, setup, test, updater, vm, workspace

Libraries remaining: agentic, build, cache, cli, container, devops,
errors, framework, git, i18n, io, log, mcp, process, release, repos

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(mcp): use pkg/io Medium for sandboxed file operations

Replace manual path validation with pkg/io.Medium for all file operations.
This delegates security (path traversal, symlink bypass) to the sandboxed
local.Medium implementation.

Changes:
- Add io.NewSandboxed() for creating sandboxed Medium instances
- Refactor MCP Service to use io.Medium instead of direct os.* calls
- Remove validatePath and resolvePathWithSymlinks functions
- Update tests to verify Medium-based behaviour

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct import path and workflow references

- Fix pkg/io/io.go import from core-gui to core
- Update CI workflows to use internal/cmd/updater path

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(security): address CodeRabbit review issues for path validation

- pkg/io/local: add symlink resolution and boundary-aware containment
  - Reject absolute paths in sandboxed Medium
  - Use filepath.EvalSymlinks to prevent symlink bypass attacks
  - Fix prefix check to prevent /tmp/root matching /tmp/root2

- pkg/mcp: fix resolvePath to validate and return errors
  - Changed resolvePath from (string) to (string, error)
  - Update deleteFile, renameFile, listDirectory, fileExists to handle errors
  - Changed New() to return (*Service, error) instead of *Service
  - Properly propagate option errors instead of silently discarding

- pkg/io: wrap errors with E() helper for consistent context
  - Copy() and MockMedium.Read() now use coreerr.E()

- tests: rename to use _Good/_Bad/_Ugly suffixes per coding guidelines
  - Fix hardcoded /tmp in TestPath to use t.TempDir()
  - Add TestResolvePath_Bad_SymlinkTraversal test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: fix gofmt formatting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: fix gofmt formatting across all files

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 21:59:34 +00:00
Snider
fdc108c69e
feat: git command, build improvements, and go fmt git-aware (#74)
* feat(go): make go fmt git-aware by default

- By default, only check changed Go files (modified, staged, untracked)
- Add --all flag to check all files (previous behaviour)
- Reduces noise when running fmt on large codebases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(build): minimal output by default, add missing i18n

- Default output now shows single line: "Success Built N artifacts (dir)"
- Add --verbose/-v flag to show full detailed output
- Add all missing i18n translations for build commands
- Errors still show failure reason in minimal mode

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: add root-level `core git` command

- Create pkg/gitcmd with git workflow commands as root menu
- Export command builders from pkg/dev (AddCommitCommand, etc.)
- Commands available under both `core git` and `core dev` for compatibility
- Git commands: health, commit, push, pull, work, sync, apply
- GitHub orchestration stays in dev: issues, reviews, ci, impact

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(qa): add docblock coverage checking

Implement docblock/docstring coverage analysis for Go code:
- New `core qa docblock` command to check coverage
- Shows compact file:line list when under threshold
- Integrate with `core go qa` as a default check
- Add --docblock-threshold flag (default 80%)

The checker uses Go AST parsing to find exported symbols
(functions, types, consts, vars) without documentation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback

- Fix doc comment: "status" → "health" in gitcmd package
- Implement --check flag for `core go fmt` (exits non-zero if files need formatting)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* docs: add docstrings for 100% coverage

Add documentation comments to all exported symbols:
- pkg/build: ProjectType constants
- pkg/cli: LogLevel, RenderStyle, TableStyle
- pkg/framework: ServiceFor, MustServiceFor, Core.Core
- pkg/git: GitError.Error, GitError.Unwrap
- pkg/i18n: Handler Match/Handle methods
- pkg/log: Level constants
- pkg/mcp: Tool input/output types
- pkg/php: Service constants, QA types, service methods
- pkg/process: ServiceError.Error
- pkg/repos: RepoType constants
- pkg/setup: ChangeType, ChangeCategory constants
- pkg/workspace: AddWorkspaceCommands

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: standardize line endings to LF

Add .gitattributes to enforce LF line endings for all text files.
Normalize all existing files to use Unix-style line endings.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback

- cmd_format.go: validate --check/--fix mutual exclusivity, capture stderr
- cmd_docblock.go: return error instead of os.Exit(1) for proper error handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address CodeRabbit review feedback (round 2)

- linuxkit.go: propagate state update errors, handle cmd.Wait() errors in waitForExit
- mcp.go: guard against empty old_string in editDiff to prevent runaway edits
- cmd_docblock.go: log parse errors instead of silently skipping

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 10:48:44 +00:00
Snider
9931593f9d refactor(cli): move commands from cmd/ to pkg/ with self-registration
Implements defence in depth through build variants - only compiled code
exists in the binary. Commands now self-register via cli.RegisterCommands()
in their init() functions, mirroring the i18n.RegisterLocales() pattern.

Structure changes:
- cmd/{ai,build,ci,dev,docs,doctor,go,php,pkg,sdk,setup,test,vm}/ → pkg/*/cmd_*.go
- cmd/core_dev.go, cmd/core_ci.go → cmd/variants/{full,ci,php,minimal}.go
- Added pkg/cli/commands.go with RegisterCommands API
- Updated pkg/cli/runtime.go to attach registered commands

Build variants:
- go build           → full (21MB, all 13 command groups)
- go build -tags ci  → ci (18MB, build/ci/sdk/doctor)
- go build -tags php → php (14MB, php/doctor)
- go build -tags minimal → minimal (11MB, doctor only)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 21:55:55 +00:00