Borg/docs/index.md

# Borg Data Collector

Borg Data Collector is a command-line tool for collecting and managing data from various sources.

## Commands

### `collect`

This command is used to collect resources from different sources and store them in a DataNode.

#### `collect github repo`

Collects a single Git repository and stores it in a DataNode.

**Usage:**
```
borg collect github repo [repository-url] [flags]
```

**Flags:**
- `--output string`: Output file for the DataNode (default "repo.dat")
- `--format string`: Output format (datanode or matrix) (default "datanode")
- `--compression string`: Compression format (none, gz, or xz) (default "none")

**Example:**
```
./borg collect github repo https://github.com/Snider/Borg --output borg.dat
```

#### `collect website`

Collects a single website and stores it in a DataNode.

**Usage:**
```
borg collect website [url] [flags]
```

**Flags:**
- `--output string`: Output file for the DataNode (default "website.dat")
- `--depth int`: Recursion depth for downloading (default 2)
- `--format string`: Output format (datanode or matrix) (default "datanode")
- `--compression string`: Compression format (none, gz, or xz) (default "none")

**Example:**
```
./borg collect website https://google.com --output website.dat --depth 1
```

#### `collect pwa`

Collects a single PWA and stores it in a DataNode.

**Usage:**
```
borg collect pwa [flags]
```

**Flags:**
- `--uri string`: The URI of the PWA to collect
- `--output string`: Output file for the DataNode (default "pwa.dat")
- `--format string`: Output format (datanode or matrix) (default "datanode")
- `--compression string`: Compression format (none, gz, or xz) (default "none")

**Example:**
```
./borg collect pwa --uri https://squoosh.app --output squoosh.dat
```

### `serve`

Serves the contents of a packaged DataNode or Terminal Isolation Matrix file using a static file server.

**Usage:**
```
borg serve [file] [flags]
```

**Flags:**
- `--port string`: Port to serve the DataNode on (default "8080")

**Example:**
```
# Serve a DataNode
./borg serve squoosh.dat --port 8888

# Serve a Terminal Isolation Matrix
./borg serve borg.matrix --port 9999
```

## Compression

All `collect` commands support optional compression. The following compression formats are available:

- `none`: No compression (default)
- `gz`: Gzip compression
- `xz`: XZ compression

To use compression, specify the desired format with the `--compression` flag. The output filename will be automatically updated with the appropriate extension (e.g., `.gz`, `.xz`).

**Example:**
```
./borg collect github repo https://github.com/Snider/Borg --compression gz
```

The `serve` command can transparently serve compressed files.

## Terminal Isolation Matrix

The `matrix` format creates a `runc` compatible bundle. This bundle can be executed by `runc` to create a container with the collected files. This is useful for creating isolated environments for testing or analysis.

To create a Matrix, use the `--format matrix` flag with any of the `collect` subcommands.

**Example:**
```
./borg collect github repo https://github.com/Snider/Borg --output borg.matrix --format matrix
```

You can then execute the Matrix with `runc`:
```
# Create a directory for the bundle
mkdir borg-bundle

# Unpack the matrix into the bundle directory
tar -xf borg.matrix -C borg-bundle

# Run the bundle
cd borg-bundle
runc run borg
```

## Inspecting a DataNode

The `examples` directory contains a Go program that can be used to inspect the contents of a `.dat` file.

**Usage:**
```
go run examples/inspect_datanode.go <path to .dat file>
```

**Example:**
```
# First, create a .dat file
./borg collect github repo https://github.com/Snider/Borg --output borg.dat

# Then, inspect it
go run examples/inspect_datanode.go borg.dat
```