This commit introduces configurable timeouts for HTTP requests made by the `collect` commands.
Key changes:
- Created a new `pkg/httpclient` package with a `NewClient` function that returns an `http.Client` with configurable timeouts for total, connect, TLS, and header stages.
- Added `--timeout`, `--connect-timeout`, `--tls-timeout`, and `--header-timeout` persistent flags to the `collect` command, making them available to all its subcommands.
- Refactored the `pkg/website`, `pkg/pwa`, and `pkg/github` packages to accept and use a custom `http.Client`, allowing the timeout configurations to be injected.
- Updated the `collect website`, `collect pwa`, and `collect github repos` commands to create a configured HTTP client based on the new flags and pass it to the respective packages.
- Added unit tests for the `pkg/httpclient` package to verify correct timeout configuration.
- Fixed all test and build failures that resulted from the refactoring.
- Addressed an unrelated build failure by creating a placeholder file (`pkg/player/frontend/demo-track.smsg`).
This work addresses the initial requirement for configurable timeouts via command-line flags. Further work is needed to implement per-domain overrides from a configuration file and idle timeouts for large file downloads.
Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
Refactored the existing tests to use the `_Good`, `_Bad`, and `_Ugly`
testing convention. This provides a more structured approach to testing
and ensures that a wider range of scenarios are covered, including
valid inputs, invalid inputs, and edge cases.
In addition to refactoring the tests, this change also includes several
bug fixes that were uncovered by the new tests. These fixes improve the
robustness and reliability of the codebase.
The following packages and commands were affected:
- `pkg/datanode`
- `pkg/compress`
- `pkg/github`
- `pkg/matrix`
- `pkg/pwa`
- `pkg/vcs`
- `pkg/website`
- `cmd/all`
- `cmd/collect`
- `cmd/collect_github_repo`
- `cmd/collect_website`
- `cmd/compile`
- `cmd/root`
- `cmd/run`
- `cmd/serve`
This commit introduces a significant refactoring of the `cmd` package to improve testability and increases test coverage across the application.
Key changes include:
- Refactored Cobra commands to use `RunE` for better error handling and testing.
- Extracted business logic from command handlers into separate, testable functions.
- Added comprehensive unit tests for the `cmd`, `compress`, `github`, `logger`, and `pwa` packages.
- Added tests for missing command-line arguments, as requested.
- Implemented the `borg all` command to clone all public repositories for a GitHub user or organization.
- Restored and improved the `collect pwa` functionality.
- Removed duplicate code and fixed various bugs.
- Addressed a resource leak in the `all` command.
- Improved error handling in the `pwa` package.
- Refactored `main.go` to remove duplicated logic.
- Fixed several other minor bugs and inconsistencies.
- Made tests platform-independent by removing hardcoded `/dev/null` paths.
- Fixed potential panics in tests by adding `nil` checks for errors.
- Fixed test state leakage by using `t.Cleanup` to restore mocked package-level variables.
This commit introduces a significant refactoring of the `cmd` package to improve testability and increases test coverage across the application.
Key changes include:
- Refactored Cobra commands to use `RunE` for better error handling and testing.
- Extracted business logic from command handlers into separate, testable functions.
- Added comprehensive unit tests for the `cmd`, `compress`, `github`, `logger`, and `pwa` packages.
- Added tests for missing command-line arguments, as requested.
- Implemented the `borg all` command to clone all public repositories for a GitHub user or organization.
- Restored and improved the `collect pwa` functionality.
- Removed duplicate code and fixed various bugs.
- Addressed a resource leak in the `all` command.
- Improved error handling in the `pwa` package.
- Refactored `main.go` to remove duplicated logic.
- Fixed several other minor bugs and inconsistencies.
