74de15af8a
This commit introduces a documentation website built with MkDocs and the Material theme. Key changes: - Added a `mkdocs.yml` configuration file. - Renamed `docs/README.md` to `docs/index.md` to serve as the site's homepage. - Created a GitHub Actions workflow (`.github/workflows/mkdocs.yml`) to automatically build and deploy the site to GitHub Pages on pushes to the `main` branch.
3.7 KiB
Borg Data Collector
Borg Data Collector is a command-line tool for collecting and managing data from various sources.
Commands
collect
This command is used to collect resources from different sources and store them in a DataNode.
collect github repo
Collects a single Git repository and stores it in a DataNode.
Usage:
borg collect github repo [repository-url] [flags]
Flags:
--output string: Output file for the DataNode (default "repo.dat")--format string: Output format (datanode or matrix) (default "datanode")--compression string: Compression format (none, gz, or xz) (default "none")
Example:
./borg collect github repo https://github.com/Snider/Borg --output borg.dat
collect website
Collects a single website and stores it in a DataNode.
Usage:
borg collect website [url] [flags]
Flags:
--output string: Output file for the DataNode (default "website.dat")--depth int: Recursion depth for downloading (default 2)--format string: Output format (datanode or matrix) (default "datanode")--compression string: Compression format (none, gz, or xz) (default "none")
Example:
./borg collect website https://google.com --output website.dat --depth 1
collect pwa
Collects a single PWA and stores it in a DataNode.
Usage:
borg collect pwa [flags]
Flags:
--uri string: The URI of the PWA to collect--output string: Output file for the DataNode (default "pwa.dat")--format string: Output format (datanode or matrix) (default "datanode")--compression string: Compression format (none, gz, or xz) (default "none")
Example:
./borg collect pwa --uri https://squoosh.app --output squoosh.dat
serve
Serves the contents of a packaged DataNode or Terminal Isolation Matrix file using a static file server.
Usage:
borg serve [file] [flags]
Flags:
--port string: Port to serve the DataNode on (default "8080")
Example:
# Serve a DataNode
./borg serve squoosh.dat --port 8888
# Serve a Terminal Isolation Matrix
./borg serve borg.matrix --port 9999
Compression
All collect commands support optional compression. The following compression formats are available:
none: No compression (default)gz: Gzip compressionxz: XZ compression
To use compression, specify the desired format with the --compression flag. The output filename will be automatically updated with the appropriate extension (e.g., .gz, .xz).
Example:
./borg collect github repo https://github.com/Snider/Borg --compression gz
The serve command can transparently serve compressed files.
Terminal Isolation Matrix
The matrix format creates a runc compatible bundle. This bundle can be executed by runc to create a container with the collected files. This is useful for creating isolated environments for testing or analysis.
To create a Matrix, use the --format matrix flag with any of the collect subcommands.
Example:
./borg collect github repo https://github.com/Snider/Borg --output borg.matrix --format matrix
You can then execute the Matrix with runc:
# Create a directory for the bundle
mkdir borg-bundle
# Unpack the matrix into the bundle directory
tar -xf borg.matrix -C borg-bundle
# Run the bundle
cd borg-bundle
runc run borg
Inspecting a DataNode
The examples directory contains a Go program that can be used to inspect the contents of a .dat file.
Usage:
go run examples/inspect_datanode.go <path to .dat file>
Example:
# First, create a .dat file
./borg collect github repo https://github.com/Snider/Borg --output borg.dat
# Then, inspect it
go run examples/inspect_datanode.go borg.dat