Commit graph

3 commits

Author SHA1 Message Date
Snider
dad5cd2588 fix: Resolve CI failure
This commit resolves the CI failure by correcting a typo in the
`downloadURL` function. The `httpClient.Get(u)` call was replaced with
the correct `http.Get(u)` call, and an unused import was removed.
2026-02-02 06:49:57 +00:00
Snider
ef08cdf807
Update cmd/collect_batch_test.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-02 06:45:22 +00:00
google-labs-jules[bot]
fd271f8bd9 feat: Add collect batch command
This commit introduces a new `collect batch` command that allows users to
collect multiple URLs from a file, stdin, or a JSON registry.

The command supports the following features:

- Parallel downloads with a configurable number of workers.
- Rate limiting with a configurable delay between requests.
- The ability to skip already downloaded files.
- Progress reporting with a progress bar.
- Reading URLs from a file, stdin, or a JSON file with a `jq` filter.

Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
2026-02-02 00:40:53 +00:00