This commit introduces a new `collect website` command that recursively downloads a website to a specified depth.
- A new `pkg/website` package contains the logic for the recursive download.
- A new `pkg/ui` package provides a progress bar for long-running operations, which is used by the website downloader.
- The `collect pwa` subcommand has been restored to be PWA-specific.
This commit refactors the repository collection functionality to use the new `DataNode` package instead of the old `trix` package.
The `collect` and `all` commands have been updated to use the new `vcs` package, which clones Git repositories and packages them into a `DataNode`. The `trix` package and its related commands (`cat`, `ingest`) have been removed.
This commit introduces a new `DataNode` package, which provides an in-memory, `fs.FS`-compatible filesystem with a `debme`-like interface. The `DataNode` can be serialized to and from a TAR archive, making it suitable for storing downloaded assets.
The `pwa` and `serve` commands have been refactored to use the `DataNode`. The `pwa` command now packages downloaded PWA assets into a `DataNode` and saves it as a `.dat` file. The `serve` command loads a `.dat` file into a `DataNode` and serves its contents.
This commit introduces two new commands: `pwa` and `serve`.
The `pwa` command downloads a Progressive Web Application (PWA) from a given URL. It discovers the PWA's manifest, downloads the assets referenced in the manifest (start URL and icons), and packages them into a single `.tar` file.
The `serve` command takes a `.tar` file created by the `pwa` command and serves its contents using a standard Go HTTP file server. It unpacks the tarball into an in-memory filesystem, making it a self-contained and efficient way to host the downloaded PWA.
This commit introduces the core functionality of the Borg Data Collector.
- Adds the `collect` command to clone a single Git repository and store it in a Trix cube.
- Adds the `collect all` command to clone all public repositories from a GitHub user or organization.
- Implements the Trix cube as a `tar` archive.
- Adds the `ingest` command to add files to a Trix cube.
- Adds the `cat` command to extract files from a Trix cube.
- Integrates Borg-themed status messages for a more engaging user experience.
