Commit graph

4 commits

Author SHA1 Message Date
Snider
dad5cd2588 fix: Resolve CI failure
This commit resolves the CI failure by correcting a typo in the
`downloadURL` function. The `httpClient.Get(u)` call was replaced with
the correct `http.Get(u)` call, and an unused import was removed.
2026-02-02 06:49:57 +00:00
Snider
2f45e5806f
Update cmd/collect_batch.go
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-02-02 06:44:39 +00:00
Snider
0521d835e7
Update cmd/collect_batch.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-02-02 06:44:19 +00:00
google-labs-jules[bot]
fd271f8bd9 feat: Add collect batch command
This commit introduces a new `collect batch` command that allows users to
collect multiple URLs from a file, stdin, or a JSON registry.

The command supports the following features:

- Parallel downloads with a configurable number of workers.
- Rate limiting with a configurable delay between requests.
- The ability to skip already downloaded files.
- Progress reporting with a progress bar.
- Reading URLs from a file, stdin, or a JSON file with a `jq` filter.

Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
2026-02-02 00:40:53 +00:00