b36990cdec
This feature adds sitemap.xml parsing to the `borg collect website` command. It introduces three new flags: - `--use-sitemap`: Auto-detects and uses the sitemap in combination with crawling. - `--sitemap-only`: Collects only the URLs found in the sitemap. - `--sitemap`: Specifies an explicit URL for the sitemap. The implementation supports standard sitemaps, sitemap indexes, and compressed sitemaps (.xml.gz). Co-authored-by: Snider <631881+Snider@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| compress | ||
| console | ||
| datanode | ||
| github | ||
| logger | ||
| mocks | ||
| player | ||
| pwa | ||
| smsg | ||
| stmf | ||
| tarfs | ||
| tim | ||
| trix | ||
| ui | ||
| vcs | ||
| wasm/stmf | ||
| website | ||