Replace ui.NewProgressBar with the Progress interface in CollectLocal,
update command help text to describe streaming mode for STIM format,
and mark the design document status as Implemented.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add CollectLocalStreaming that uses a streaming pipeline
(walk -> tar -> compress -> encrypt -> file) via io.Pipe,
avoiding buffering the entire dataset in memory. Add DecryptStimV2
for round-trip decryption back to DataNode. Wire streaming path
into existing CollectLocal when format is "stim".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add streaming compression wrappers that work with io.Writer/io.Reader
instead of []byte. NewCompressWriter supports gz, xz, and passthrough
(none/"") formats, returning an io.WriteCloser. Unknown formats return
an error. Includes nopCloser type for the passthrough case.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement StreamEncrypt/StreamDecrypt using 1 MiB ChaCha20-Poly1305
blocks with the STIM v2 wire format (magic header, Argon2id salt/params,
per-block random nonces, and zero-length EOF marker).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Store symlink entries in the files map with a target path field,
and handle them correctly in ToTar/FromTar serialization. Symlinks
use tar.TypeSymlink with mode 0777, and Stat returns os.ModeSymlink.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire the Progress interface into the CLI by adding a --quiet/-q global
flag and a ProgressFromCmd helper that returns QuietProgress (stderr)
when --quiet is set, or DefaultProgress (TTY-aware) otherwise.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace golang.org/x/term with the already-depended-on go-isatty for TTY
detection, matching the pattern in non_interactive_prompter.go. Add tests
for InteractiveProgress Start/Finish and Update methods. Run go mod tidy
to clean up stale go.sum entries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Approved design for fixing collect local on large directories,
streaming encryption, Argon2id key derivation, symlink handling,
and quiet/scripted mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add `borg collect local` command to collect files from the local
filesystem into a DataNode.
Features:
- Walks directory tree (defaults to CWD)
- Respects .gitignore patterns by default
- Excludes hidden files by default (--hidden to include)
- Custom exclude patterns via --exclude flag
- Output formats: datanode, tim, trix, stim
- Compression: none, gz, xz
Examples:
borg collect local
borg collect local ./src --output src.tar.xz --compression xz
borg collect local . --format stim --password secret
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add multi-quality variant support for video content:
- New ABR types in pkg/smsg/types.go (ABRManifest, Variant, ABRPresets)
- New pkg/smsg/abr.go with manifest read/write and bandwidth estimation
- New cmd/mkdemo-abr CLI tool for creating ABR variant sets via ffmpeg
- WASM parseABRManifest and selectVariant functions
- Demo page "Adaptive Quality" tab with ABR player
- RFC-001 Section 3.7 documenting ABR format and algorithm
Profile page:
- No WASM or video download until play button clicked
- Play button visible immediately, loading on-demand
- Removed auto-play behavior completely
Streaming:
- GetV3HeaderFromPrefix for parsing from partial data
- v3 demo file with 128KB chunks for streaming tests
V3 streaming format enables zero-trust media streaming:
- Content encrypted once with random CEK
- CEK wrapped with time-bound stream keys derived from LTHN hash
- Rolling window: current period + next period always valid
- Keys auto-expire, no revocation needed
Cadence options (platform controls refresh rate):
- daily: 24-hour periods (2026-01-12)
- 12h: Half-day periods (2026-01-12-AM/PM)
- 6h: Quarter-day periods (2026-01-12-00/06/12/18)
- 1h: Hourly periods (2026-01-12-15)
Key derivation: SHA256(LTHN(period:license:fingerprint))
- LTHN is rainbow-table resistant (salt derived from input)
- Only the derived key can decrypt, never transmitted
New files:
- pkg/smsg/stream.go - v3 encryption/decryption
- pkg/smsg/stream_test.go - 17 tests including cadence
WASM v1.3.0:
- BorgSMSG.decryptV3(data, {license, fingerprint})
- getInfo() now returns cadence and keyMethod
- cmd/dapp-fm-app: Native desktop app with WebView (Wails)
- cmd/dapp-fm: CLI binary for HTTP server mode
- pkg/player: Shared player core with Go bindings
Architecture: Go decrypts SMSG content, serves via asset handler.
Frontend calls Go directly via Wails bindings for manifest/license
checks.
- Update docs/cli.md to include all current commands (all, compile, run, decode) and correct terminology (TIM).
- Update docs/library.md with correct, runnable Go code examples and updated import paths.
- Update docs/development.md, docs/installation.md, and docs/releasing.md to reflect the project's Go version (1.25.0).
- Remove outdated "No functional changes" notes from documentation files.
- Added unit tests for `ToTar` and `FromTar` in `pkg/datanode`, including a round-trip test and invalid input handling.
- Added unit tests for `Walk` options (`MaxDepth`, `Filter`, `SkipErrors`) in `pkg/datanode`.
- Added security tests for `pkg/tim` to verify protection against path traversal (Zip Slip) attacks and handling of invalid inputs.
- Fixed `cmd` package tests execution by adding `TestHelperProcess` to `cmd/run_test.go` to support mocked command execution.
- Increased coverage for `pkg/datanode` to 84.2%, `pkg/tim` to 74.2%, and `cmd` to 44.1%.
This commit introduces the `Enchantrix` library to add support for the `.trix` encrypted file format.
The main changes are:
- The `matrix` format has been renamed to `tim` (Terminal Isolation Matrix).
- The `.tim` format is now a specialized `.trix` file.
- A new `decode` command has been added to decode `.trix` and `.tim` files.
- The `collect` commands now support the `trix` and `tim` formats.
- A `--password` flag has been added to the `collect` commands for encryption.
- A `--i-am-in-isolation` flag has been added to the `decode` command for safely decoding `.tim` files.
- The decryption functionality is currently disabled due to a bug in the `Enchantrix` library. A follow-up PR will be created to re-enable it.
- Path traversal vulnerability in `pkg/tim/run.go` has been fixed.
- File descriptor leak in `pkg/tim/run.go` has been fixed.
- Improved error handling in `pkg/trix/trix.go`.
This commit introduces the `Enchantrix` library to add support for the `.trix` encrypted file format.
The main changes are:
- The `matrix` format has been renamed to `tim` (Terminal Isolation Matrix).
- The `.tim` format is now a specialized `.trix` file.
- A new `decode` command has been added to decode `.trix` and `.tim` files.
- The `collect` commands now support the `trix` and `tim` formats.
- A `--password` flag has been added to the `collect` commands for encryption.
- A `--i-am-in-isolation` flag has been added to the `decode` command for safely decoding `.tim` files.
- The decryption functionality is currently disabled due to a bug in the `Enchantrix` library. A follow-up PR will be created to re-enable it.
- Implements all placeholder Go examples in the `examples` directory.
- Corrects the `run_matrix_programmatically` example to use the `borg` package.
- Refactors the code to centralize the matrix execution logic in the `matrix` package.
- Updates the documentation to include a new "Programmatic Usage" section that describes all of the Go examples.
- Updates the "Terminal Isolation Matrix" section to remove manual 'runc' instructions, emphasizing that 'borg run' handles this process to maintain security and isolation.
- Adds missing examples for 'collect github repos', 'collect github release', and 'compile' commands to the documentation.
- Makes `pkg/matrix.Run` testable by exposing `exec.Command` as a public variable.
- Adds tests for the `matrix` package that mock the `runc` command.
- Updates the `cmd` package tests to mock `matrix.ExecCommand` instead of the old `cmd.execCommand`.
- Implements all placeholder Go examples in the `examples` directory.
- Corrects the `run_matrix_programmatically` example to use the `borg` package.
- Refactors the code to centralize the matrix execution logic in the `matrix` package.
- Updates the documentation to include a new "Programmatic Usage" section that describes all of the Go examples.
- Updates the "Terminal Isolation Matrix" section to remove manual 'runc' instructions, emphasizing that 'borg run' handles this process to maintain security and isolation.
- Adds missing examples for 'collect github repos', 'collect github release', and 'compile' commands to the documentation.
Refactored the existing tests to use the `_Good`, `_Bad`, and `_Ugly`
testing convention. This provides a more structured approach to testing
and ensures that a wider range of scenarios are covered, including
valid inputs, invalid inputs, and edge cases.
In addition to refactoring the tests, this change also includes several
bug fixes that were uncovered by the new tests. These fixes improve the
robustness and reliability of the codebase.
The following packages and commands were affected:
- `pkg/datanode`
- `pkg/compress`
- `pkg/github`
- `pkg/matrix`
- `pkg/pwa`
- `pkg/vcs`
- `pkg/website`
- `cmd/all`
- `cmd/collect`
- `cmd/collect_github_repo`
- `cmd/collect_website`
- `cmd/compile`
- `cmd/root`
- `cmd/run`
- `cmd/serve`
This reverts the following commits:
- feat: Increase test coverage for pkg/datanode
- feat: Increase test coverage for pkg/compress
- feat: Increase test coverage for pkg/pwa
- feat: Increase test coverage for pkg/website
- feat: Increase test coverage for pkg/vcs
These changes are being reverted because they were causing test failures
and were not contributing to the overall stability of the project.
This commit addresses three separate issues:
1. **cmd/compile.go**: The `Borgfile` parsing now returns an error for
unknown instructions instead of silently ignoring them.
2. **cmd/run_test.go**: The `execCommand` mock is now properly cleaned
up after the test, preventing it from leaking into other tests.
3. **pkg/matrix/matrix_test.go**: The EOF check in the tar reader loop
now uses a direct error comparison (`io.EOF`) instead of a string
comparison.
This commit increases the test coverage for the `pkg/datanode` package
from 58.1% to 66.9%.
The following changes were made:
- Added a new test file, `pkg/datanode/fromtar_test.go`, to test the
`FromTar` function.
- Added a new test file, `pkg/datanode/fs_test.go`, to test the `Stat`,
`Exists`, and `Walk` functions.
- Refactored the existing tests in `pkg/datanode/datanode_test.go` to
remove the monolithic test function and improve the organization of
the tests.
This commit adds placeholder Go programs to the `examples` directory for
all of Borg's features. This provides a clear roadmap for future
implementation and ensures that we have a testing strategy for each
function.
The new placeholder examples are:
- `examples/all`
- `examples/collect_github_release`
- `examples/collect_github_repo`
- `examples/collect_github_repos`
- `examples/collect_pwa`
- `examples/collect_website`
- `examples/serve`
