Borg/pkg/httpclient
google-labs-jules[bot] e057ddfa7f feat: Add Proxy and Tor Support for Website Collector
This commit introduces support for routing website collection requests through HTTP/SOCKS5 proxies or Tor.

New flags for the `borg collect website` command:
- `--proxy <url>`: Route requests through a single HTTP or SOCKS5 proxy.
- `--proxy-list <file>`: Select a random proxy from a file for all requests.
- `--tor`: Route requests through a Tor SOCKS5 proxy (defaults to 127.0.0.1:9050).

Key changes:
- Created a new `pkg/httpclient` to centralize the creation of proxy-configured `http.Client` instances.
- Refactored `pkg/website` to use a `Downloader` interface, allowing for dependency injection of the HTTP client.
- Added validation to ensure the new proxy flags are mutually exclusive.
- Implemented support for SOCKS5 authentication via credentials in the proxy URL.
- Added comprehensive unit tests for the new functionality.

Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
2026-02-02 00:52:32 +00:00
..
client.go feat: Add Proxy and Tor Support for Website Collector 2026-02-02 00:52:32 +00:00
client_test.go feat: Add Proxy and Tor Support for Website Collector 2026-02-02 00:52:32 +00:00