Borg/README.md
google-labs-jules[bot] 05cc3f7210 feat: add collectors for npm, cargo, and go modules
This commit introduces new collectors for npm, cargo, and go modules, allowing users to archive package metadata and source code from their respective registries.

The `npm` and `go` collectors have been fully implemented, with commands and unit tests. The `cargo` collector is also fully implemented, after a period of being blocked by the `crates.io` API. The correct `User-Agent` was found by inspecting the `cargo` binary.

The `pypi` collector has not yet been implemented, but a clear path forward has been established by successfully fetching package metadata from the `pypi.org` API.

This commit also addresses feedback from a previous code review, including the removal of a `tcpdump.log` file and the correction of several nitpicks.

Co-authored-by: Snider <631881+Snider@users.noreply.github.com>
2026-02-02 00:53:48 +00:00

4.9 KiB
Raw Blame History

Borg

codecov Go Version License

Borg is a CLI tool and Go library for collecting, packaging, and encrypting data into portable, self-contained containers. It supports GitHub repositories, websites, PWAs, and arbitrary files.

Features

  • Data Collection - Clone GitHub repos, crawl websites, download PWAs
  • Portable Containers - Package data into DataNodes (in-memory fs.FS) or TIM bundles (OCI-compatible)
  • Zero-Trust Encryption - ChaCha20-Poly1305 encryption for TIM containers (.stim) and messages (.smsg)
  • SMSG Format - Encrypted message containers with public manifests, attachments, and zstd compression
  • WASM Support - Decrypt SMSG files in the browser via WebAssembly

Installation

# From source
go install github.com/Snider/Borg@latest

# Or build locally
git clone https://github.com/Snider/Borg.git
cd Borg
go build -o borg ./

Requires Go 1.25+

Quick Start

# Clone a GitHub repository into a TIM container
borg collect github repo https://github.com/user/repo --format tim -o repo.tim

# Encrypt a TIM container
borg compile -f Borgfile -e "password" -o encrypted.stim

# Run an encrypted container
borg run encrypted.stim -p "password"

# Inspect container metadata (without decrypting)
borg inspect encrypted.stim --json

Container Formats

Format Extension Description
DataNode .tar In-memory filesystem, portable tarball
TIM .tim Terminal Isolation Matrix - OCI/runc compatible bundle
Trix .trix PGP-encrypted DataNode
STIM .stim ChaCha20-Poly1305 encrypted TIM
SMSG .smsg Encrypted message with attachments and public manifest

SMSG - Secure Message Format

SMSG is designed for distributing encrypted content with publicly visible metadata:

import "github.com/Snider/Borg/pkg/smsg"

// Create and encrypt a message
msg := smsg.NewMessage("Hello, World!")
msg.AddBinaryAttachment("track.mp3", audioData, "audio/mpeg")

manifest := &smsg.Manifest{
    Title:  "Demo Track",
    Artist: "Artist Name",
}

encrypted, _ := smsg.EncryptV2WithManifest(msg, "password", manifest)

// Decrypt
decrypted, _ := smsg.Decrypt(encrypted, "password")

v2 Binary Format - Stores attachments as raw binary with zstd compression for optimal size.

See RFC-001: Open Source DRM for the full specification.

Live Demo: demo.dapp.fm

Borgfile

Package files into a TIM container:

ADD ./app /usr/local/bin/app
ADD ./config /etc/app/
borg compile -f Borgfile -o app.tim
borg compile -f Borgfile -e "secret" -o app.stim  # encrypted

CLI Reference

# Collection
borg collect github repo <url>           # Clone repository
borg collect github repos <owner>        # Clone all repos from user/org
borg collect website <url> --depth 2     # Crawl website
borg collect pwa --uri <url>             # Download PWA
borg collect npm <package>               # Collect npm package
borg collect cargo <package>             # Collect cargo crate
borg collect go <module>                 # Collect Go module

# Compilation
borg compile -f Borgfile -o out.tim      # Plain TIM
borg compile -f Borgfile -e "pass"       # Encrypted STIM

# Execution
borg run container.tim                   # Run plain TIM
borg run container.stim -p "pass"        # Run encrypted TIM

# Inspection
borg decode file.stim -p "pass" -o out.tar
borg inspect file.stim [--json]

Documentation

mkdocs serve  # Serve docs locally at http://localhost:8000

Development

task build    # Build binary
task test     # Run tests with coverage
task clean    # Clean build artifacts

Architecture

Source (GitHub/Website/PWA)
    ↓ collect
DataNode (in-memory fs.FS)
    ↓ serialize
    ├── .tar  (raw tarball)
    ├── .tim  (runc container bundle)
    ├── .trix (PGP encrypted)
    └── .stim (ChaCha20-Poly1305 encrypted TIM)

License

EUPL-1.2 - European Union Public License


Borg Status Messages (for CLI theming)

Initialization

  • Core engaged… resistance is already buffering.
  • Assimilating bytes… stand by for cubeformation.
  • Merging… the Core is rewriting reality, one block at a time.

Encryption

  • Generating cryptographic sigils the Core whispers to the witch.
  • Encrypting payload the Core feeds data to the witch's cauldron.
  • Merge complete data assimilated, encrypted, and sealed within us.

VCS Processing

  • Initiating clone… the Core replicates the collective into your node.
  • Merging branches… conflicts resolved, entropy minimized, assimilation complete.