Adds a flexible hook system to the `borg collect` commands, allowing users to run custom scripts at various stages of the collection lifecycle.
This feature introduces a new `pkg/hooks` package that encapsulates the core logic for parsing a `.borg-hooks.yaml` configuration file and executing external scripts.
Key features:
- Four hook events are supported: `on_file_collected`, `on_url_found`, `on_collection_complete`, and `on_error`.
- A `--hooks` flag has been added to the `collect website` and `collect pwa` commands.
- The system automatically detects and loads a `.borg-hooks.yaml` file from the current directory if the `--hooks` flag is not provided.
- File-based hooks (`on_file_collected`) support glob pattern matching against the base filename.
- Hook scripts receive a JSON payload on stdin with relevant event context.
- Commands with arguments are correctly handled by executing them through `sh -c`.
The implementation includes a comprehensive test suite with both unit tests for the new `hooks` package and integration tests to validate the end-to-end functionality. All existing tests and examples have been updated to reflect the necessary function signature changes.
Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
Refactored the existing tests to use the `_Good`, `_Bad`, and `_Ugly`
testing convention. This provides a more structured approach to testing
and ensures that a wider range of scenarios are covered, including
valid inputs, invalid inputs, and edge cases.
In addition to refactoring the tests, this change also includes several
bug fixes that were uncovered by the new tests. These fixes improve the
robustness and reliability of the codebase.
The following packages and commands were affected:
- `pkg/datanode`
- `pkg/compress`
- `pkg/github`
- `pkg/matrix`
- `pkg/pwa`
- `pkg/vcs`
- `pkg/website`
- `cmd/all`
- `cmd/collect`
- `cmd/collect_github_repo`
- `cmd/collect_website`
- `cmd/compile`
- `cmd/root`
- `cmd/run`
- `cmd/serve`
This commit improves the user experience of the application by adding progress bars to long-running operations.
The following commands now display a progress bar:
- `collect github repo`
- `collect website`
- `collect pwa`
The underlying packages (`pkg/vcs`, `pkg/website`, and `pkg/pwa`) have been updated to support progress reporting.
This commit introduces a new `collect website` command that recursively downloads a website to a specified depth.
- A new `pkg/website` package contains the logic for the recursive download.
- A new `pkg/ui` package provides a progress bar for long-running operations, which is used by the website downloader.
- The `collect pwa` subcommand has been restored to be PWA-specific.
