Snider/Borg
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: Add ChaCha20-Poly1305 encryption and decryption for TIM files (.stim), enhance CLI for encryption format handling (stim), and include metadata inspection support

Browse source
This commit is contained in:
snider 2025-12-26 01:25:03 +00:00
parent b8f5390fb0
commit 376517d7a2
18 changed files with 2412 additions and 48 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

141
CLAUDE.md Normal file
View file

@ -0,0 +1,141 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Build and Development Commands
```bash
# Build
task build # or: go build -o borg main.go
# Test
task test # all tests with coverage
go test -run TestName ./pkg/tim # single test
go test -v ./pkg/tim/... # verbose package tests
# Clean and utilities
task clean # remove build artifacts
mkdocs serve # serve docs locally
```
## Architecture Overview
Borg collects data from various sources (GitHub, websites, PWAs) and packages it into portable, optionally encrypted containers.
### Core Abstractions
```
Source (GitHub/Website/PWA)
↓ collect
DataNode (in-memory fs.FS)
↓ serialize
├── .tar (raw tarball)
├── .tim (runc container bundle)
├── .trix (PGP encrypted)
└── .stim (ChaCha20-Poly1305 encrypted TIM)
```
**DataNode** (`pkg/datanode/datanode.go`): In-memory filesystem implementing `fs.FS`. Core methods:
- `AddData(path, content)` - add file
- `ToTar()` / `FromTar()` - serialize/deserialize
- `Walk()`, `Open()`, `Stat()` - fs.FS interface
**TIM** (`pkg/tim/tim.go`): Terminal Isolation Matrix - runc-compatible container bundle with:
- `Config []byte` - OCI runtime spec (config.json)
- `RootFS *DataNode` - container filesystem
- `ToTar()` / `ToSigil(password)` - serialize plain or encrypted
### Encryption
Two encryption systems via Enchantrix library:
| Format | Algorithm | Use Case |
|--------|-----------|----------|
| `.trix` | PGP symmetric | Legacy DataNode encryption |
| `.stim` | ChaCha20-Poly1305 | TIM encryption (config + rootfs encrypted separately) |
**ChaChaPolySigil** (`pkg/tim/tim.go`):
```go
// Encrypt TIM
stim, _ := tim.ToSigil(password)
// Decrypt TIM
tim, _ := tim.FromSigil(data, password)
// Run encrypted TIM
tim.RunEncrypted(path, password)
```
**Key derivation**: `trix.DeriveKey(password)` - SHA-256(password) → 32-byte key
**Cache API** (`pkg/tim/cache.go`): Encrypted TIM storage
```go
cache, _ := tim.NewCache("/path/to/cache", password)
cache.Store("name", tim)
tim, _ := cache.Load("name")
```
### Package Structure
| Package | Purpose |
|---------|---------|
| `cmd/` | Cobra CLI commands |
| `pkg/datanode/` | In-memory fs.FS |
| `pkg/tim/` | Container bundles, encryption, execution |
| `pkg/trix/` | Trix format wrapper (PGP + ChaCha) |
| `pkg/compress/` | gzip/xz compression |
| `pkg/vcs/` | Git operations |
| `pkg/github/` | GitHub API client |
| `pkg/website/` | Website crawler |
| `pkg/pwa/` | PWA downloader |
### CLI Reference
```bash
# Collect
borg collect github repo <url> # clone git repo
borg collect github repos <owner> # clone all repos from user/org
borg collect website <url> --depth 2 # crawl website
borg collect pwa --uri <url> # download PWA
# Common flags for collect commands:
# --format datanode|tim|trix|stim
# --compression none|gz|xz
# --password <pass> # required for trix/stim
# Compile TIM from Borgfile
borg compile -f Borgfile -o out.tim
borg compile -f Borgfile -e "password" # encrypted → .stim
# Run
borg run container.tim # plain TIM
borg run container.stim -p "password" # encrypted TIM
# Decode
borg decode file.trix -o decoded.tar
borg decode file.stim -p "pass" --i-am-in-isolation -o decoded.tar
# Inspect (view metadata without decrypting)
borg inspect file.stim # human-readable
borg inspect file.stim --json # JSON output
```
### Borgfile Format
```dockerfile
ADD local/path /container/path
```
### Testing Patterns
Tests use dependency injection for external services:
- `pkg/tim/run.go`: `ExecCommand` var for mocking runc
- `pkg/vcs/git.go`: `GitCloner` interface for mocking git
- `cmd/`: Commands expose `New*Cmd()` for testing
When adding encryption tests, use round-trip pattern:
```go
stim, _ := tim.ToSigil(password)
restored, _ := tim.FromSigil(stim, password)
// verify restored matches original
```

24
cmd/collect_github_repo.go
View file

@ -37,8 +37,8 @@ func NewCollectGithubRepoCmd() *cobra.Command {
compression, _ := cmd.Flags().GetString("compression")
password, _ := cmd.Flags().GetString("password")
if format != "datanode" && format != "tim" && format != "trix" {
return fmt.Errorf("invalid format: %s (must be 'datanode', 'tim', or 'trix')", format)
if format != "datanode" && format != "tim" && format != "trix" && format != "stim" {
return fmt.Errorf("invalid format: %s (must be 'datanode', 'tim', 'trix', or 'stim')", format)
}
if compression != "none" && compression != "gz" && compression != "xz" {
return fmt.Errorf("invalid compression: %s (must be 'none', 'gz', or 'xz')", compression)
@ -61,14 +61,26 @@ func NewCollectGithubRepoCmd() *cobra.Command {
var data []byte
if format == "tim" {
tim, err := tim.FromDataNode(dn)
t, err := tim.FromDataNode(dn)
if err != nil {
return fmt.Errorf("error creating tim: %w", err)
}
data, err = tim.ToTar()
data, err = t.ToTar()
if err != nil {
return fmt.Errorf("error serializing tim: %w", err)
}
} else if format == "stim" {
if password == "" {
return fmt.Errorf("password required for stim format")
}
t, err := tim.FromDataNode(dn)
if err != nil {
return fmt.Errorf("error creating tim: %w", err)
}
data, err = t.ToSigil(password)
if err != nil {
return fmt.Errorf("error encrypting stim: %w", err)
}
} else if format == "trix" {
data, err = trix.ToTrix(dn, password)
if err != nil {
@ -103,9 +115,9 @@ func NewCollectGithubRepoCmd() *cobra.Command {
},
}
cmd.Flags().String("output", "", "Output file for the DataNode")
cmd.Flags().String("format", "datanode", "Output format (datanode, tim, or trix)")
cmd.Flags().String("format", "datanode", "Output format (datanode, tim, trix, or stim)")
cmd.Flags().String("compression", "none", "Compression format (none, gz, or xz)")
cmd.Flags().String("password", "", "Password for encryption")
cmd.Flags().String("password", "", "Password for encryption (required for trix/stim)")
return cmd
}

43
cmd/collect_pwa.go
View file

@ -5,9 +5,9 @@ import (
"os"
"github.com/Snider/Borg/pkg/compress"
"github.com/Snider/Borg/pkg/pwa"
"github.com/Snider/Borg/pkg/tim"
"github.com/Snider/Borg/pkg/trix"
"github.com/Snider/Borg/pkg/pwa"
"github.com/Snider/Borg/pkg/ui"
"github.com/spf13/cobra"
@ -24,14 +24,21 @@ func NewCollectPWACmd() *CollectPWACmd {
PWAClient: pwa.NewPWAClient(),
}
c.Command = cobra.Command{
Use: "pwa",
Use: "pwa [url]",
Short: "Collect a single PWA using a URI",
Long: `Collect a single PWA and store it in a DataNode.
Example:
borg collect pwa --uri https://example.com --output mypwa.dat`,
Examples:
borg collect pwa https://example.com
borg collect pwa https://example.com --output mypwa.dat
borg collect pwa https://example.com --format stim --password secret`,
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
pwaURL, _ := cmd.Flags().GetString("uri")
// Allow URL as positional argument
if len(args) > 0 && pwaURL == "" {
pwaURL = args[0]
}
outputFile, _ := cmd.Flags().GetString("output")
format, _ := cmd.Flags().GetString("format")
compression, _ := cmd.Flags().GetString("compression")
@ -45,11 +52,11 @@ Example:
return nil
},
}
c.Flags().String("uri", "", "The URI of the PWA to collect")
c.Flags().String("uri", "", "The URI of the PWA to collect (can also be passed as positional arg)")
c.Flags().String("output", "", "Output file for the DataNode")
c.Flags().String("format", "datanode", "Output format (datanode, tim, or trix)")
c.Flags().String("format", "datanode", "Output format (datanode, tim, trix, or stim)")
c.Flags().String("compression", "none", "Compression format (none, gz, or xz)")
c.Flags().String("password", "", "Password for encryption")
c.Flags().String("password", "", "Password for encryption (required for stim format)")
return c
}
@ -58,10 +65,13 @@ func init() {
}
func CollectPWA(client pwa.PWAClient, pwaURL string, outputFile string, format string, compression string, password string) (string, error) {
if pwaURL == "" {
return "", fmt.Errorf("uri is required")
return "", fmt.Errorf("url is required")
}
if format != "datanode" && format != "tim" && format != "trix" {
return "", fmt.Errorf("invalid format: %s (must be 'datanode', 'tim', or 'trix')", format)
if format != "datanode" && format != "tim" && format != "trix" && format != "stim" {
return "", fmt.Errorf("invalid format: %s (must be 'datanode', 'tim', 'trix', or 'stim')", format)
}
if format == "stim" && password == "" {
return "", fmt.Errorf("password is required for stim format")
}
if compression != "none" && compression != "gz" && compression != "xz" {
return "", fmt.Errorf("invalid compression: %s (must be 'none', 'gz', or 'xz')", compression)
@ -82,14 +92,23 @@ func CollectPWA(client pwa.PWAClient, pwaURL string, outputFile string, format s
var data []byte
if format == "tim" {
tim, err := tim.FromDataNode(dn)
t, err := tim.FromDataNode(dn)
if err != nil {
return "", fmt.Errorf("error creating tim: %w", err)
}
data, err = tim.ToTar()
data, err = t.ToTar()
if err != nil {
return "", fmt.Errorf("error serializing tim: %w", err)
}
} else if format == "stim" {
t, err := tim.FromDataNode(dn)
if err != nil {
return "", fmt.Errorf("error creating tim: %w", err)
}
data, err = t.ToSigil(password)
if err != nil {
return "", fmt.Errorf("error encrypting stim: %w", err)
}
} else if format == "trix" {
data, err = trix.ToTrix(dn, password)
if err != nil {

18
cmd/compile.go
View file

@ -11,6 +11,7 @@ import (
var borgfile string
var output string
var encryptPassword string
var compileCmd = NewCompileCmd()
@ -52,16 +53,33 @@ func NewCompileCmd() *cobra.Command {
}
}
// If encryption is requested, output as .stim
if encryptPassword != "" {
stimData, err := m.ToSigil(encryptPassword)
if err != nil {
return err
}
outputPath := output
if !strings.HasSuffix(outputPath, ".stim") {
outputPath = strings.TrimSuffix(outputPath, ".tim") + ".stim"
}
fmt.Fprintf(cmd.OutOrStdout(), "Compiled encrypted TIM to %s\n", outputPath)
return os.WriteFile(outputPath, stimData, 0644)
}
// Original unencrypted output
tarball, err := m.ToTar()
if err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "Compiled TIM to %s\n", output)
return os.WriteFile(output, tarball, 0644)
},
}
compileCmd.Flags().StringVarP(&borgfile, "file", "f", "Borgfile", "Path to the Borgfile.")
compileCmd.Flags().StringVarP(&output, "output", "o", "a.tim", "Path to the output tim file.")
compileCmd.Flags().StringVarP(&encryptPassword, "encrypt", "e", "", "Encrypt with ChaCha20-Poly1305 using this password (outputs .stim)")
return compileCmd
}

26
cmd/decode.go
View file

@ -3,7 +3,9 @@ package cmd
import (
"fmt"
"os"
"strings"
"github.com/Snider/Borg/pkg/tim"
"github.com/Snider/Borg/pkg/trix"
trixsdk "github.com/Snider/Enchantrix/pkg/trix"
"github.com/spf13/cobra"
@ -14,7 +16,7 @@ var decodeCmd = NewDecodeCmd()
func NewDecodeCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "decode [file]",
Short: "Decode a .trix or .tim file",
Short: "Decode a .trix, .tim, or .stim file",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
inputFile := args[0]
@ -27,6 +29,27 @@ func NewDecodeCmd() *cobra.Command {
return err
}
// Check if it's a .stim file (encrypted TIM)
if strings.HasSuffix(inputFile, ".stim") || (len(data) >= 4 && string(data[:4]) == "STIM") {
if password == "" {
return fmt.Errorf("password required for .stim files")
}
if !inIsolation {
return fmt.Errorf("this is an encrypted Terminal Isolation Matrix, use the --i-am-in-isolation flag to decode it")
}
m, err := tim.FromSigil(data, password)
if err != nil {
return err
}
tarball, err := m.ToTar()
if err != nil {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "Decoded encrypted TIM to %s\n", outputFile)
return os.WriteFile(outputFile, tarball, 0644)
}
// Try TRIX format
t, err := trixsdk.Decode(data, "TRIX", nil)
if err != nil {
return err
@ -46,6 +69,7 @@ func NewDecodeCmd() *cobra.Command {
return err
}
fmt.Fprintf(cmd.OutOrStdout(), "Decoded to %s\n", outputFile)
return os.WriteFile(outputFile, tarball, 0644)
},
}

114
cmd/inspect.go Normal file
View file

@ -0,0 +1,114 @@
package cmd
import (
"encoding/json"
"fmt"
"os"
"strings"
trixsdk "github.com/Snider/Enchantrix/pkg/trix"
"github.com/spf13/cobra"
)
var inspectCmd = NewInspectCmd()
func NewInspectCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "inspect [file]",
Short: "Inspect metadata of a .trix or .stim file without decrypting",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
inputFile := args[0]
jsonOutput, _ := cmd.Flags().GetBool("json")
data, err := os.ReadFile(inputFile)
if err != nil {
return err
}
if len(data) < 4 {
return fmt.Errorf("file too small to be a valid container")
}
magic := string(data[:4])
var t *trixsdk.Trix
switch magic {
case "STIM":
t, err = trixsdk.Decode(data, "STIM", nil)
if err != nil {
return fmt.Errorf("failed to decode STIM: %w", err)
}
case "TRIX":
t, err = trixsdk.Decode(data, "TRIX", nil)
if err != nil {
return fmt.Errorf("failed to decode TRIX: %w", err)
}
default:
return fmt.Errorf("unknown file format (magic: %q)", magic)
}
if jsonOutput {
info := map[string]interface{}{
"file": inputFile,
"magic": magic,
"header": t.Header,
"payload_size": len(t.Payload),
}
enc := json.NewEncoder(cmd.OutOrStdout())
enc.SetIndent("", " ")
return enc.Encode(info)
}
// Human-readable output
fmt.Fprintf(cmd.OutOrStdout(), "File: %s\n", inputFile)
fmt.Fprintf(cmd.OutOrStdout(), "Format: %s\n", magic)
fmt.Fprintf(cmd.OutOrStdout(), "Payload Size: %d bytes\n", len(t.Payload))
fmt.Fprintf(cmd.OutOrStdout(), "Header:\n")
for k, v := range t.Header {
fmt.Fprintf(cmd.OutOrStdout(), " %s: %v\n", k, v)
}
// Show encryption info
if algo, ok := t.Header["encryption_algorithm"]; ok {
fmt.Fprintf(cmd.OutOrStdout(), "\nEncryption: %v\n", algo)
}
if _, ok := t.Header["tim"]; ok {
fmt.Fprintf(cmd.OutOrStdout(), "Type: Terminal Isolation Matrix\n")
}
if v, ok := t.Header["version"]; ok {
fmt.Fprintf(cmd.OutOrStdout(), "Version: %v\n", v)
}
return nil
},
}
cmd.Flags().Bool("json", false, "Output in JSON format")
return cmd
}
func GetInspectCmd() *cobra.Command {
return inspectCmd
}
func init() {
RootCmd.AddCommand(GetInspectCmd())
}
// isStimFile checks if a file is a .stim file by extension or magic number.
func isStimFile(path string) bool {
if strings.HasSuffix(path, ".stim") {
return true
}
f, err := os.Open(path)
if err != nil {
return false
}
defer f.Close()
magic := make([]byte, 4)
if _, err := f.Read(magic); err != nil {
return false
}
return string(magic) == "STIM"
}

40
cmd/run.go
View file

@ -1,21 +1,57 @@
package cmd
import (
"os"
"strings"
"github.com/Snider/Borg/pkg/tim"
"github.com/spf13/cobra"
)
var runPassword string
var runCmd = NewRunCmd()
func NewRunCmd() *cobra.Command {
return &cobra.Command{
cmd := &cobra.Command{
Use: "run [tim file]",
Short: "Run a Terminal Isolation Matrix.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return tim.Run(args[0])
filePath := args[0]
// Check if encrypted by extension or magic number
if isEncryptedTIM(filePath) {
password, _ := cmd.Flags().GetString("password")
if password == "" {
return tim.ErrPasswordRequired
}
return tim.RunEncrypted(filePath, password)
}
return tim.Run(filePath)
},
}
cmd.Flags().StringVarP(&runPassword, "password", "p", "", "Decryption password for encrypted TIMs (.stim)")
return cmd
}
// isEncryptedTIM checks if a file is an encrypted TIM by extension or magic number.
func isEncryptedTIM(path string) bool {
if strings.HasSuffix(path, ".stim") {
return true
}
// Check magic number
f, err := os.Open(path)
if err != nil {
return false
}
defer f.Close()
magic := make([]byte, 4)
if _, err := f.Read(magic); err != nil {
return false
}
return string(magic) == "STIM"
}
func GetRunCmd() *cobra.Command {

5
go.work
View file

@ -1,3 +1,6 @@
go 1.25.0
use .
use (
.
../Enchantrix
)

9
go.work.sum
View file

@ -12,8 +12,17 @@ github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0 h1:4G4v2dO3VZwixGIRoQ5Lfboy6nUhCyYzaqnIAPPhYs4=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
golang.org/x/crypto v0.11.1-0.20230711161743-2e82bdd1719d/go.mod h1:xgJhtzW8F9jGdVFWZESrid1U1bjeNy4zgy5cRr/CIio=
golang.org/x/net v0.46.0/go.mod h1:Q9BGdFy1y4nkUwiLvT5qtyhAnEHgnQ/zd8PfU6nc210=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/tools v0.37.0 h1:DVSRzp7FwePZW356yEAChSdNcQo6Nsp+fex1SUW09lE=
golang.org/x/tools v0.37.0/go.mod h1:MBN5QPQtLMHVdvsbtarmTNukZDdgwdwlO5qGacAzF0w=
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=

319
pkg/pwa/pwa.go
View file

@ -4,8 +4,10 @@ import (
"encoding/json"
"fmt"
"io"
"io/fs"
"net/http"
"net/url"
"regexp"
"strings"
"sync"
@ -14,6 +16,14 @@ import (
"golang.org/x/net/html"
)
// Common fallback paths for PWA manifests
var manifestFallbackPaths = []string{
"/manifest.json",
"/manifest.webmanifest",
"/site.webmanifest",
"/app.webmanifest",
}
// PWAClient is an interface for interacting with PWAs.
type PWAClient interface {
FindManifest(pwaURL string) (string, error)
@ -30,6 +40,8 @@ type pwaClient struct {
}
// FindManifest finds the manifest for a PWA.
// It first looks for a <link rel="manifest"> tag in the HTML,
// then tries common fallback paths if not found.
func (p *pwaClient) FindManifest(pwaURL string) (string, error) {
resp, err := p.client.Get(pwaURL)
if err != nil {
@ -71,37 +83,104 @@ func (p *pwaClient) FindManifest(pwaURL string) (string, error) {
}
f(doc)
if manifestURL == "" {
return "", fmt.Errorf("manifest not found")
// If manifest found via link tag, resolve and return
if manifestURL != "" {
resolvedURL, err := p.resolveURL(pwaURL, manifestURL)
if err != nil {
return "", err
}
return resolvedURL.String(), nil
}
resolvedURL, err := p.resolveURL(pwaURL, manifestURL)
// Try fallback paths
baseURL, err := url.Parse(pwaURL)
if err != nil {
return "", err
}
return resolvedURL.String(), nil
for _, path := range manifestFallbackPaths {
testURL := &url.URL{
Scheme: baseURL.Scheme,
Host: baseURL.Host,
Path: path,
}
resp, err := p.client.Get(testURL.String())
if err != nil {
continue
}
resp.Body.Close()
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return testURL.String(), nil
}
}
return "", fmt.Errorf("manifest not found (checked HTML and fallback paths: %v)", manifestFallbackPaths)
}
// Manifest represents a PWA manifest with all common fields.
type Manifest struct {
Name string `json:"name"`
ShortName string `json:"short_name"`
StartURL string `json:"start_url"`
Scope string `json:"scope"`
Display string `json:"display"`
BackgroundColor string `json:"background_color"`
ThemeColor string `json:"theme_color"`
Description string `json:"description"`
Icons []struct {
Src string `json:"src"`
Sizes string `json:"sizes"`
Type string `json:"type"`
} `json:"icons"`
Screenshots []struct {
Src string `json:"src"`
Sizes string `json:"sizes"`
Type string `json:"type"`
} `json:"screenshots"`
Shortcuts []struct {
Name string `json:"name"`
URL string `json:"url"`
Icons []struct {
Src string `json:"src"`
} `json:"icons"`
} `json:"shortcuts"`
RelatedApplications []struct {
Platform string `json:"platform"`
URL string `json:"url"`
ID string `json:"id"`
} `json:"related_applications"`
ServiceWorker struct {
Src string `json:"src"`
Scope string `json:"scope"`
} `json:"serviceworker"`
}
// DownloadAndPackagePWA downloads and packages a PWA into a DataNode.
// It downloads the manifest, all referenced assets, and parses HTML pages
// for additional linked resources (CSS, JS, images).
func (p *pwaClient) DownloadAndPackagePWA(pwaURL, manifestURL string, bar *progressbar.ProgressBar) (*datanode.DataNode, error) {
dn := datanode.New()
var wg sync.WaitGroup
var errs []error
var mu sync.Mutex
downloaded := make(map[string]bool)
type Manifest struct {
StartURL string `json:"start_url"`
Icons []struct {
Src string `json:"src"`
} `json:"icons"`
}
downloadAndAdd := func(assetURL string) {
var downloadAndAdd func(assetURL string, parseHTML bool)
downloadAndAdd = func(assetURL string, parseHTML bool) {
defer wg.Done()
if bar != nil {
bar.Add(1)
}
// Skip if already downloaded
mu.Lock()
if downloaded[assetURL] {
mu.Unlock()
return
}
downloaded[assetURL] = true
mu.Unlock()
resp, err := p.client.Get(assetURL)
if err != nil {
mu.Lock()
@ -133,7 +212,25 @@ func (p *pwaClient) DownloadAndPackagePWA(pwaURL, manifestURL string, bar *progr
mu.Unlock()
return
}
dn.AddData(strings.TrimPrefix(u.Path, "/"), body)
path := strings.TrimPrefix(u.Path, "/")
if path == "" {
path = "index.html"
}
dn.AddData(path, body)
// Parse HTML for additional assets
if parseHTML && isHTMLContent(resp.Header.Get("Content-Type"), body) {
additionalAssets := p.extractAssetsFromHTML(assetURL, body)
for _, asset := range additionalAssets {
mu.Lock()
if !downloaded[asset] {
wg.Add(1)
go downloadAndAdd(asset, false) // Don't recursively parse HTML
}
mu.Unlock()
}
}
}
// Download manifest first, synchronously.
@ -154,20 +251,29 @@ func (p *pwaClient) DownloadAndPackagePWA(pwaURL, manifestURL string, bar *progr
u, _ := url.Parse(manifestURL)
dn.AddData(strings.TrimPrefix(u.Path, "/"), manifestData)
downloaded[manifestURL] = true
// Parse manifest and download assets concurrently.
// Parse manifest and collect all assets.
var manifest Manifest
if err := json.Unmarshal(manifestData, &manifest); err != nil {
return nil, fmt.Errorf("failed to parse manifest: %w", err)
}
assetsToDownload := []string{}
htmlPages := []string{}
// Start URL (HTML page)
if manifest.StartURL != "" {
startURL, err := p.resolveURL(manifestURL, manifest.StartURL)
if err == nil {
assetsToDownload = append(assetsToDownload, startURL.String())
htmlPages = append(htmlPages, startURL.String())
}
} else {
// If no start_url, use the PWA URL itself
htmlPages = append(htmlPages, pwaURL)
}
// Icons
for _, icon := range manifest.Icons {
if icon.Src != "" {
iconURL, err := p.resolveURL(manifestURL, icon.Src)
@ -177,12 +283,68 @@ func (p *pwaClient) DownloadAndPackagePWA(pwaURL, manifestURL string, bar *progr
}
}
wg.Add(len(assetsToDownload))
for _, asset := range assetsToDownload {
go downloadAndAdd(asset)
// Screenshots
for _, screenshot := range manifest.Screenshots {
if screenshot.Src != "" {
screenshotURL, err := p.resolveURL(manifestURL, screenshot.Src)
if err == nil {
assetsToDownload = append(assetsToDownload, screenshotURL.String())
}
}
}
// Shortcuts and their icons
for _, shortcut := range manifest.Shortcuts {
if shortcut.URL != "" {
shortcutURL, err := p.resolveURL(manifestURL, shortcut.URL)
if err == nil {
htmlPages = append(htmlPages, shortcutURL.String())
}
}
for _, icon := range shortcut.Icons {
if icon.Src != "" {
iconURL, err := p.resolveURL(manifestURL, icon.Src)
if err == nil {
assetsToDownload = append(assetsToDownload, iconURL.String())
}
}
}
}
// Service worker
if manifest.ServiceWorker.Src != "" {
swURL, err := p.resolveURL(manifestURL, manifest.ServiceWorker.Src)
if err == nil {
assetsToDownload = append(assetsToDownload, swURL.String())
}
}
// Download HTML pages first (with asset extraction)
for _, page := range htmlPages {
wg.Add(1)
go downloadAndAdd(page, true)
}
wg.Wait()
// Download remaining assets
for _, asset := range assetsToDownload {
if !downloaded[asset] {
wg.Add(1)
go downloadAndAdd(asset, false)
}
}
wg.Wait()
// Try to detect service worker from HTML if not in manifest
if manifest.ServiceWorker.Src == "" {
swURL := p.detectServiceWorker(pwaURL, dn)
if swURL != "" && !downloaded[swURL] {
wg.Add(1)
go downloadAndAdd(swURL, false)
wg.Wait()
}
}
if len(errs) > 0 {
var errStrings []string
for _, e := range errs {
@ -194,6 +356,127 @@ func (p *pwaClient) DownloadAndPackagePWA(pwaURL, manifestURL string, bar *progr
return dn, nil
}
// extractAssetsFromHTML parses HTML and extracts linked assets.
func (p *pwaClient) extractAssetsFromHTML(baseURL string, htmlContent []byte) []string {
var assets []string
doc, err := html.Parse(strings.NewReader(string(htmlContent)))
if err != nil {
return assets
}
var extract func(*html.Node)
extract = func(n *html.Node) {
if n.Type == html.ElementNode {
var href string
switch n.Data {
case "link":
// CSS stylesheets and icons
var rel, linkHref string
for _, a := range n.Attr {
if a.Key == "rel" {
rel = a.Val
}
if a.Key == "href" {
linkHref = a.Val
}
}
if linkHref != "" && (rel == "stylesheet" || rel == "icon" || rel == "apple-touch-icon" || rel == "shortcut icon") {
href = linkHref
}
case "script":
// JavaScript files
for _, a := range n.Attr {
if a.Key == "src" && a.Val != "" {
href = a.Val
break
}
}
case "img":
// Images
for _, a := range n.Attr {
if a.Key == "src" && a.Val != "" {
href = a.Val
break
}
}
}
if href != "" && !strings.HasPrefix(href, "data:") {
resolved, err := p.resolveURL(baseURL, href)
if err == nil {
assets = append(assets, resolved.String())
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
extract(c)
}
}
extract(doc)
return assets
}
// detectServiceWorker tries to find service worker registration in HTML/JS.
func (p *pwaClient) detectServiceWorker(baseURL string, dn *datanode.DataNode) string {
// Look for common service worker registration patterns
patterns := []string{
`navigator\.serviceWorker\.register\(['"]([^'"]+)['"]`,
`serviceWorker\.register\(['"]([^'"]+)['"]`,
}
// Check all downloaded HTML and JS files
err := dn.Walk(".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return nil
}
if d.IsDir() {
return nil
}
if strings.HasSuffix(path, ".html") || strings.HasSuffix(path, ".js") || path == "index.html" {
file, err := dn.Open(path)
if err != nil {
return nil
}
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
return nil
}
for _, pattern := range patterns {
re := regexp.MustCompile(pattern)
matches := re.FindSubmatch(content)
if len(matches) > 1 {
swPath := string(matches[1])
resolved, err := p.resolveURL(baseURL, swPath)
if err == nil {
return fmt.Errorf("found:%s", resolved.String())
}
}
}
}
return nil
})
if err != nil && strings.HasPrefix(err.Error(), "found:") {
return strings.TrimPrefix(err.Error(), "found:")
}
return ""
}
// isHTMLContent checks if content is HTML based on Content-Type or content inspection.
func isHTMLContent(contentType string, body []byte) bool {
if strings.Contains(contentType, "text/html") {
return true
}
// Check for HTML doctype or html tag
content := strings.ToLower(string(body[:min(len(body), 1024)]))
return strings.Contains(content, "<!doctype html") || strings.Contains(content, "<html")
}
func (p *pwaClient) resolveURL(base, ref string) (*url.URL, error) {
baseURL, err := url.Parse(base)
if err != nil {

323
pkg/pwa/pwa_test.go
View file

@ -34,8 +34,13 @@ func TestFindManifest_Good(t *testing.T) {
func TestFindManifest_Bad(t *testing.T) {
t.Run("No Manifest Link", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<html><head></head></html>`)
// Return HTML for main page, 404 for everything else (including fallback paths)
if r.URL.Path == "/" {
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<html><head></head></html>`)
} else {
http.NotFound(w, r)
}
}))
defer server.Close()
client := NewPWAClient()
@ -76,6 +81,58 @@ func TestFindManifest_Ugly(t *testing.T) {
t.Errorf("Expected manifest URL %s, but got %s", expectedURL, actualURL)
}
})
t.Run("Fallback to manifest.json", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/":
// No manifest link in HTML
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<html><head></head></html>`)
case "/manifest.json":
// But manifest.json exists at fallback path
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{"name": "Fallback PWA"}`)
default:
http.NotFound(w, r)
}
}))
defer server.Close()
client := NewPWAClient()
expectedURL := server.URL + "/manifest.json"
actualURL, err := client.FindManifest(server.URL)
if err != nil {
t.Fatalf("FindManifest should find fallback manifest.json: %v", err)
}
if actualURL != expectedURL {
t.Errorf("Expected manifest URL %s, but got %s", expectedURL, actualURL)
}
})
t.Run("Fallback to site.webmanifest", func(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<html><head></head></html>`)
case "/site.webmanifest":
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{"name": "Webmanifest PWA"}`)
default:
http.NotFound(w, r)
}
}))
defer server.Close()
client := NewPWAClient()
expectedURL := server.URL + "/site.webmanifest"
actualURL, err := client.FindManifest(server.URL)
if err != nil {
t.Fatalf("FindManifest should find fallback site.webmanifest: %v", err)
}
if actualURL != expectedURL {
t.Errorf("Expected manifest URL %s, but got %s", expectedURL, actualURL)
}
})
}
// --- Test Cases for DownloadAndPackagePWA ---
@ -189,6 +246,268 @@ func TestResolveURL_Bad(t *testing.T) {
}
}
// --- Test Cases for extractAssetsFromHTML ---
func TestExtractAssetsFromHTML(t *testing.T) {
client := NewPWAClient().(*pwaClient)
t.Run("extracts stylesheets", func(t *testing.T) {
html := []byte(`<html><head><link rel="stylesheet" href="style.css"></head></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 1 || assets[0] != "http://example.com/style.css" {
t.Errorf("Expected [http://example.com/style.css], got %v", assets)
}
})
t.Run("extracts scripts", func(t *testing.T) {
html := []byte(`<html><body><script src="app.js"></script></body></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 1 || assets[0] != "http://example.com/app.js" {
t.Errorf("Expected [http://example.com/app.js], got %v", assets)
}
})
t.Run("extracts images", func(t *testing.T) {
html := []byte(`<html><body><img src="logo.png"></body></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 1 || assets[0] != "http://example.com/logo.png" {
t.Errorf("Expected [http://example.com/logo.png], got %v", assets)
}
})
t.Run("extracts icons", func(t *testing.T) {
html := []byte(`<html><head><link rel="icon" href="favicon.ico"></head></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 1 || assets[0] != "http://example.com/favicon.ico" {
t.Errorf("Expected [http://example.com/favicon.ico], got %v", assets)
}
})
t.Run("extracts apple-touch-icon", func(t *testing.T) {
html := []byte(`<html><head><link rel="apple-touch-icon" href="apple-icon.png"></head></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 1 || assets[0] != "http://example.com/apple-icon.png" {
t.Errorf("Expected [http://example.com/apple-icon.png], got %v", assets)
}
})
t.Run("ignores data URIs", func(t *testing.T) {
html := []byte(`<html><body><img src="data:image/png;base64,abc123"></body></html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 0 {
t.Errorf("Expected no assets for data URI, got %v", assets)
}
})
t.Run("handles multiple assets", func(t *testing.T) {
html := []byte(`<html>
<head>
<link rel="stylesheet" href="style.css">
<link rel="icon" href="favicon.ico">
</head>
<body>
<script src="app.js"></script>
<img src="logo.png">
</body>
</html>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
if len(assets) != 4 {
t.Errorf("Expected 4 assets, got %d: %v", len(assets), assets)
}
})
t.Run("handles invalid HTML gracefully", func(t *testing.T) {
html := []byte(`not valid html at all <<<>>>`)
assets := client.extractAssetsFromHTML("http://example.com/", html)
// Should not panic, may return empty or partial results
_ = assets
})
}
// --- Test Cases for isHTMLContent ---
func TestIsHTMLContent(t *testing.T) {
t.Run("detects text/html content-type", func(t *testing.T) {
if !isHTMLContent("text/html; charset=utf-8", []byte("anything")) {
t.Error("Should detect text/html content type")
}
})
t.Run("detects doctype", func(t *testing.T) {
if !isHTMLContent("", []byte("<!DOCTYPE html><html></html>")) {
t.Error("Should detect HTML by doctype")
}
})
t.Run("detects html tag", func(t *testing.T) {
if !isHTMLContent("", []byte("<html><body>test</body></html>")) {
t.Error("Should detect HTML by html tag")
}
})
t.Run("rejects non-html", func(t *testing.T) {
if isHTMLContent("application/json", []byte(`{"key": "value"}`)) {
t.Error("Should not detect JSON as HTML")
}
})
}
// --- Test Cases for MockPWAClient ---
func TestMockPWAClient(t *testing.T) {
t.Run("FindManifest returns configured value", func(t *testing.T) {
mock := NewMockPWAClient("http://example.com/manifest.json", nil, nil)
url, err := mock.FindManifest("http://example.com")
if err != nil {
t.Fatalf("FindManifest error = %v", err)
}
if url != "http://example.com/manifest.json" {
t.Errorf("FindManifest = %q, want %q", url, "http://example.com/manifest.json")
}
})
t.Run("FindManifest returns configured error", func(t *testing.T) {
mock := NewMockPWAClient("", nil, fmt.Errorf("test error"))
_, err := mock.FindManifest("http://example.com")
if err == nil || err.Error() != "test error" {
t.Errorf("FindManifest error = %v, want 'test error'", err)
}
})
t.Run("DownloadAndPackagePWA returns configured datanode", func(t *testing.T) {
mock := NewMockPWAClient("", nil, nil)
dn, err := mock.DownloadAndPackagePWA("http://example.com", "http://example.com/manifest.json", nil)
if err != nil {
t.Fatalf("DownloadAndPackagePWA error = %v", err)
}
if dn != nil {
t.Error("Expected nil datanode from mock")
}
})
}
// --- Test Cases for full manifest parsing ---
func TestDownloadAndPackagePWA_FullManifest(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/manifest.json":
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{
"name": "Full PWA",
"start_url": "index.html",
"icons": [{"src": "icon.png"}],
"screenshots": [{"src": "screenshot.png"}],
"shortcuts": [
{
"name": "Action",
"url": "action.html",
"icons": [{"src": "action-icon.png"}]
}
]
}`)
case "/index.html":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<!DOCTYPE html><html><head><link rel="stylesheet" href="style.css"></head><body><script src="app.js"></script></body></html>`)
case "/icon.png", "/screenshot.png", "/action-icon.png":
w.Header().Set("Content-Type", "image/png")
fmt.Fprint(w, "fake image")
case "/action.html":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, "<html></html>")
case "/style.css":
w.Header().Set("Content-Type", "text/css")
fmt.Fprint(w, "body { color: red; }")
case "/app.js":
w.Header().Set("Content-Type", "application/javascript")
fmt.Fprint(w, "console.log('hello');")
default:
http.NotFound(w, r)
}
}))
defer server.Close()
client := NewPWAClient()
dn, err := client.DownloadAndPackagePWA(server.URL, server.URL+"/manifest.json", nil)
if err != nil {
t.Fatalf("DownloadAndPackagePWA failed: %v", err)
}
// Check manifest
exists, _ := dn.Exists("manifest.json")
if !exists {
t.Error("Expected manifest.json")
}
// Check icons
exists, _ = dn.Exists("icon.png")
if !exists {
t.Error("Expected icon.png")
}
// Check screenshots
exists, _ = dn.Exists("screenshot.png")
if !exists {
t.Error("Expected screenshot.png")
}
// Check shortcut page
exists, _ = dn.Exists("action.html")
if !exists {
t.Error("Expected action.html")
}
// Check shortcut icon
exists, _ = dn.Exists("action-icon.png")
if !exists {
t.Error("Expected action-icon.png")
}
// Check HTML-extracted assets
exists, _ = dn.Exists("style.css")
if !exists {
t.Error("Expected style.css (extracted from HTML)")
}
exists, _ = dn.Exists("app.js")
if !exists {
t.Error("Expected app.js (extracted from HTML)")
}
}
// --- Test Cases for service worker detection ---
func TestDownloadAndPackagePWA_ServiceWorker(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/manifest.json":
w.Header().Set("Content-Type", "application/json")
fmt.Fprint(w, `{"name": "SW PWA", "start_url": "index.html"}`)
case "/index.html":
w.Header().Set("Content-Type", "text/html")
fmt.Fprint(w, `<!DOCTYPE html><html><body><script>navigator.serviceWorker.register('/sw.js');</script></body></html>`)
case "/sw.js":
w.Header().Set("Content-Type", "application/javascript")
fmt.Fprint(w, "self.addEventListener('fetch', e => {});")
default:
http.NotFound(w, r)
}
}))
defer server.Close()
client := NewPWAClient()
dn, err := client.DownloadAndPackagePWA(server.URL, server.URL+"/manifest.json", nil)
if err != nil {
t.Fatalf("DownloadAndPackagePWA failed: %v", err)
}
// Service worker should be detected and downloaded
exists, _ := dn.Exists("sw.js")
if !exists {
t.Error("Expected sw.js (service worker detected from script)")
}
}
// --- Helpers ---
// newPWATestServer creates a test server for a simple PWA.

314
pkg/tarfs/tarfs_test.go Normal file
View file

@ -0,0 +1,314 @@
package tarfs
import (
"archive/tar"
"bytes"
"io"
"os"
"testing"
"time"
)
// createTestTar creates a tar archive with the given files in rootfs/ prefix
func createTestTar(files map[string][]byte) ([]byte, error) {
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
for name, content := range files {
hdr := &tar.Header{
Name: "rootfs/" + name,
Mode: 0644,
Size: int64(len(content)),
ModTime: time.Now(),
}
if err := tw.WriteHeader(hdr); err != nil {
return nil, err
}
if _, err := tw.Write(content); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
func TestNew(t *testing.T) {
t.Run("valid tar", func(t *testing.T) {
files := map[string][]byte{
"test.txt": []byte("Hello, World!"),
"subdir/file.txt": []byte("Nested file"),
}
tarData, err := createTestTar(files)
if err != nil {
t.Fatalf("Failed to create test tar: %v", err)
}
fs, err := New(tarData)
if err != nil {
t.Fatalf("New() error = %v", err)
}
if fs == nil {
t.Fatal("New() returned nil")
}
if len(fs.files) != 2 {
t.Errorf("Expected 2 files, got %d", len(fs.files))
}
})
t.Run("empty tar", func(t *testing.T) {
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
tw.Close()
fs, err := New(buf.Bytes())
if err != nil {
t.Fatalf("New() error = %v", err)
}
if len(fs.files) != 0 {
t.Errorf("Expected 0 files, got %d", len(fs.files))
}
})
t.Run("invalid tar", func(t *testing.T) {
_, err := New([]byte("not a tar file"))
if err == nil {
t.Error("Expected error for invalid tar data")
}
})
t.Run("files without rootfs prefix are ignored", func(t *testing.T) {
buf := new(bytes.Buffer)
tw := tar.NewWriter(buf)
// Add file without rootfs/ prefix
hdr := &tar.Header{
Name: "outside.txt",
Mode: 0644,
Size: 5,
}
tw.WriteHeader(hdr)
tw.Write([]byte("hello"))
// Add file with rootfs/ prefix
hdr = &tar.Header{
Name: "rootfs/inside.txt",
Mode: 0644,
Size: 5,
}
tw.WriteHeader(hdr)
tw.Write([]byte("world"))
tw.Close()
fs, err := New(buf.Bytes())
if err != nil {
t.Fatalf("New() error = %v", err)
}
// Only the rootfs file should be included
if len(fs.files) != 1 {
t.Errorf("Expected 1 file, got %d", len(fs.files))
}
if _, ok := fs.files["inside.txt"]; !ok {
t.Error("Expected 'inside.txt' to be in files")
}
})
}
func TestTarFS_Open(t *testing.T) {
files := map[string][]byte{
"test.txt": []byte("Hello, World!"),
"subdir/file.txt": []byte("Nested file"),
}
tarData, err := createTestTar(files)
if err != nil {
t.Fatalf("Failed to create test tar: %v", err)
}
fs, err := New(tarData)
if err != nil {
t.Fatalf("New() error = %v", err)
}
t.Run("existing file", func(t *testing.T) {
file, err := fs.Open("test.txt")
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
t.Fatalf("ReadAll() error = %v", err)
}
if string(content) != "Hello, World!" {
t.Errorf("Got %q, want %q", string(content), "Hello, World!")
}
})
t.Run("existing file with leading slash", func(t *testing.T) {
file, err := fs.Open("/test.txt")
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
t.Fatalf("ReadAll() error = %v", err)
}
if string(content) != "Hello, World!" {
t.Errorf("Got %q, want %q", string(content), "Hello, World!")
}
})
t.Run("nested file", func(t *testing.T) {
file, err := fs.Open("subdir/file.txt")
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer file.Close()
content, err := io.ReadAll(file)
if err != nil {
t.Fatalf("ReadAll() error = %v", err)
}
if string(content) != "Nested file" {
t.Errorf("Got %q, want %q", string(content), "Nested file")
}
})
t.Run("non-existing file", func(t *testing.T) {
_, err := fs.Open("nonexistent.txt")
if err != os.ErrNotExist {
t.Errorf("Expected os.ErrNotExist, got %v", err)
}
})
t.Run("multiple reads reset position", func(t *testing.T) {
// First read
file1, _ := fs.Open("test.txt")
content1, _ := io.ReadAll(file1)
file1.Close()
// Second read should work too
file2, _ := fs.Open("test.txt")
content2, _ := io.ReadAll(file2)
file2.Close()
if string(content1) != string(content2) {
t.Errorf("Multiple reads returned different content")
}
})
}
func TestTarFile_Methods(t *testing.T) {
files := map[string][]byte{
"test.txt": []byte("Hello, World!"),
}
tarData, err := createTestTar(files)
if err != nil {
t.Fatalf("Failed to create test tar: %v", err)
}
fs, err := New(tarData)
if err != nil {
t.Fatalf("New() error = %v", err)
}
file, err := fs.Open("test.txt")
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer file.Close()
t.Run("Read", func(t *testing.T) {
buf := make([]byte, 5)
n, err := file.Read(buf)
if err != nil {
t.Fatalf("Read() error = %v", err)
}
if n != 5 {
t.Errorf("Read() returned %d bytes, want 5", n)
}
if string(buf) != "Hello" {
t.Errorf("Got %q, want %q", string(buf), "Hello")
}
})
t.Run("Seek", func(t *testing.T) {
pos, err := file.Seek(0, io.SeekStart)
if err != nil {
t.Fatalf("Seek() error = %v", err)
}
if pos != 0 {
t.Errorf("Seek() returned position %d, want 0", pos)
}
pos, err = file.Seek(7, io.SeekStart)
if err != nil {
t.Fatalf("Seek() error = %v", err)
}
if pos != 7 {
t.Errorf("Seek() returned position %d, want 7", pos)
}
buf := make([]byte, 6)
file.Read(buf)
if string(buf) != "World!" {
t.Errorf("After seek, got %q, want %q", string(buf), "World!")
}
})
t.Run("Readdir", func(t *testing.T) {
_, err := file.Readdir(0)
if err != os.ErrInvalid {
t.Errorf("Readdir() should return os.ErrInvalid, got %v", err)
}
})
t.Run("Stat", func(t *testing.T) {
info, err := file.Stat()
if err != nil {
t.Fatalf("Stat() error = %v", err)
}
if info.Name() != "test.txt" {
t.Errorf("Name() = %q, want %q", info.Name(), "test.txt")
}
if info.Size() != 13 {
t.Errorf("Size() = %d, want 13", info.Size())
}
if info.Mode() != 0444 {
t.Errorf("Mode() = %v, want 0444", info.Mode())
}
if info.IsDir() {
t.Error("IsDir() should be false")
}
if info.Sys() != nil {
t.Error("Sys() should return nil")
}
})
t.Run("Close", func(t *testing.T) {
err := file.Close()
if err != nil {
t.Errorf("Close() error = %v", err)
}
})
}

91
pkg/tim/cache.go Normal file
View file

@ -0,0 +1,91 @@
package tim
import (
"os"
"path/filepath"
"strings"
)
// Cache provides encrypted storage for TIM containers.
// It stores TIMs as .stim files in a directory, encrypted with
// ChaCha20-Poly1305 using a shared password.
type Cache struct {
Dir string
Password string
}
// NewCache creates a cache in the given directory.
// The directory will be created if it doesn't exist.
func NewCache(dir, password string) (*Cache, error) {
if password == "" {
return nil, ErrPasswordRequired
}
if err := os.MkdirAll(dir, 0700); err != nil {
return nil, err
}
return &Cache{Dir: dir, Password: password}, nil
}
// Store encrypts and saves a TIM to the cache.
func (c *Cache) Store(name string, m *TerminalIsolationMatrix) error {
data, err := m.ToSigil(c.Password)
if err != nil {
return err
}
path := filepath.Join(c.Dir, name+".stim")
return os.WriteFile(path, data, 0600)
}
// Load retrieves and decrypts a TIM from the cache.
func (c *Cache) Load(name string) (*TerminalIsolationMatrix, error) {
path := filepath.Join(c.Dir, name+".stim")
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
return FromSigil(data, c.Password)
}
// Delete removes a TIM from the cache.
func (c *Cache) Delete(name string) error {
path := filepath.Join(c.Dir, name+".stim")
return os.Remove(path)
}
// Exists checks if a TIM exists in the cache.
func (c *Cache) Exists(name string) bool {
path := filepath.Join(c.Dir, name+".stim")
_, err := os.Stat(path)
return err == nil
}
// List returns all cached TIM names.
func (c *Cache) List() ([]string, error) {
entries, err := os.ReadDir(c.Dir)
if err != nil {
return nil, err
}
var names []string
for _, e := range entries {
if !e.IsDir() && strings.HasSuffix(e.Name(), ".stim") {
names = append(names, strings.TrimSuffix(e.Name(), ".stim"))
}
}
return names, nil
}
// Run loads and executes a TIM from the cache using runc.
func (c *Cache) Run(name string) error {
path := filepath.Join(c.Dir, name+".stim")
return RunEncrypted(path, c.Password)
}
// Size returns the size of a cached TIM in bytes.
func (c *Cache) Size(name string) (int64, error) {
path := filepath.Join(c.Dir, name+".stim")
info, err := os.Stat(path)
if err != nil {
return 0, err
}
return info.Size(), nil
}

61
pkg/tim/run.go
View file

@ -4,6 +4,7 @@ import (
"archive/tar"
"fmt"
"io"
"io/fs"
"os"
"os/exec"
"path/filepath"
@ -71,3 +72,63 @@ func Run(timPath string) error {
cmd.Stderr = os.Stderr
return cmd.Run()
}
// RunEncrypted runs an encrypted .stim file.
// It decrypts the file, extracts the contents to a temporary directory,
// and runs the container using runc.
func RunEncrypted(stimPath, password string) error {
// Read the encrypted file
data, err := os.ReadFile(stimPath)
if err != nil {
return fmt.Errorf("failed to read stim file: %w", err)
}
// Decrypt and parse
m, err := FromSigil(data, password)
if err != nil {
return fmt.Errorf("failed to decrypt stim: %w", err)
}
// Create temp directory
tempDir, err := os.MkdirTemp("", "borg-run-*")
if err != nil {
return fmt.Errorf("failed to create temporary directory: %w", err)
}
defer os.RemoveAll(tempDir)
// Write config.json
configPath := filepath.Join(tempDir, "config.json")
if err := os.WriteFile(configPath, m.Config, 0600); err != nil {
return fmt.Errorf("failed to write config: %w", err)
}
// Extract rootfs
rootfsPath := filepath.Join(tempDir, "rootfs")
if err := os.MkdirAll(rootfsPath, 0755); err != nil {
return fmt.Errorf("failed to create rootfs dir: %w", err)
}
// Walk DataNode and extract files
err = m.RootFS.Walk(".", func(path string, d fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if d.IsDir() {
return nil
}
target := filepath.Join(rootfsPath, path)
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return err
}
return m.RootFS.CopyFile(path, target, 0600)
})
if err != nil {
return fmt.Errorf("failed to extract rootfs: %w", err)
}
// Run with runc
cmd := ExecCommand("runc", "run", "-b", tempDir, "borg-container")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
return cmd.Run()
}

441
pkg/tim/sigil_test.go Normal file
View file

@ -0,0 +1,441 @@
package tim
import (
"os"
"path/filepath"
"testing"
"github.com/Snider/Borg/pkg/trix"
)
func TestToFromSigil(t *testing.T) {
// Create a TIM with some data
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
m.RootFS.AddData("hello.txt", []byte("Hello, World!"))
m.RootFS.AddData("subdir/nested.txt", []byte("Nested content"))
password := "testpassword123"
// Encrypt
stim, err := m.ToSigil(password)
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
// Verify magic number
if len(stim) < 4 || string(stim[:4]) != "STIM" {
t.Errorf("Expected magic 'STIM', got '%s'", string(stim[:4]))
}
// Decrypt
restored, err := FromSigil(stim, password)
if err != nil {
t.Fatalf("FromSigil() error = %v", err)
}
// Verify config matches
if string(restored.Config) != string(m.Config) {
t.Error("Config mismatch after round-trip")
}
// Verify rootfs file exists
file, err := restored.RootFS.Open("hello.txt")
if err != nil {
t.Fatalf("Failed to open hello.txt: %v", err)
}
defer file.Close()
}
func TestFromSigilWrongPassword(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
stim, err := m.ToSigil("correct")
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
_, err = FromSigil(stim, "wrong")
if err == nil {
t.Error("Expected error with wrong password")
}
}
func TestToSigilEmptyPassword(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
_, err = m.ToSigil("")
if err != ErrPasswordRequired {
t.Errorf("Expected ErrPasswordRequired, got %v", err)
}
}
func TestFromSigilEmptyPassword(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
stim, err := m.ToSigil("password")
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
_, err = FromSigil(stim, "")
if err != ErrPasswordRequired {
t.Errorf("Expected ErrPasswordRequired, got %v", err)
}
}
func TestCache(t *testing.T) {
// Create a temporary directory for the cache
tempDir, err := os.MkdirTemp("", "borg-cache-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tempDir)
password := "cachepassword"
cache, err := NewCache(tempDir, password)
if err != nil {
t.Fatalf("NewCache() error = %v", err)
}
// Create a TIM
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
m.RootFS.AddData("test.txt", []byte("Cache test content"))
// Store
if err := cache.Store("mytim", m); err != nil {
t.Fatalf("Store() error = %v", err)
}
// Verify file exists
if !cache.Exists("mytim") {
t.Error("Exists() returned false for stored TIM")
}
// List
names, err := cache.List()
if err != nil {
t.Fatalf("List() error = %v", err)
}
if len(names) != 1 || names[0] != "mytim" {
t.Errorf("List() = %v, want [mytim]", names)
}
// Load
loaded, err := cache.Load("mytim")
if err != nil {
t.Fatalf("Load() error = %v", err)
}
// Verify content
file, err := loaded.RootFS.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open test.txt: %v", err)
}
defer file.Close()
// Delete
if err := cache.Delete("mytim"); err != nil {
t.Fatalf("Delete() error = %v", err)
}
if cache.Exists("mytim") {
t.Error("Exists() returned true after Delete()")
}
}
func TestCacheEmptyPassword(t *testing.T) {
tempDir, err := os.MkdirTemp("", "borg-cache-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tempDir)
_, err = NewCache(tempDir, "")
if err != ErrPasswordRequired {
t.Errorf("Expected ErrPasswordRequired, got %v", err)
}
}
func TestDeriveKey(t *testing.T) {
key := trix.DeriveKey("password")
if len(key) != 32 {
t.Errorf("DeriveKey() returned key of length %d, want 32", len(key))
}
// Same password should produce same key
key2 := trix.DeriveKey("password")
for i := range key {
if key[i] != key2[i] {
t.Error("DeriveKey() not deterministic")
break
}
}
// Different password should produce different key
key3 := trix.DeriveKey("different")
same := true
for i := range key {
if key[i] != key3[i] {
same = false
break
}
}
if same {
t.Error("DeriveKey() produced same key for different passwords")
}
}
func TestToSigilWithLargeData(t *testing.T) {
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
// Add large file
largeData := make([]byte, 1024*1024) // 1MB
for i := range largeData {
largeData[i] = byte(i % 256)
}
m.RootFS.AddData("large.bin", largeData)
password := "largetest"
// Encrypt
stim, err := m.ToSigil(password)
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
// Decrypt
restored, err := FromSigil(stim, password)
if err != nil {
t.Fatalf("FromSigil() error = %v", err)
}
// Verify file exists
_, err = restored.RootFS.Open("large.bin")
if err != nil {
t.Fatalf("Failed to open large.bin: %v", err)
}
}
func TestRunEncryptedFileNotFound(t *testing.T) {
err := RunEncrypted("/nonexistent/path.stim", "password")
if err == nil {
t.Error("Expected error for nonexistent file")
}
}
func TestRunEncryptedWithTempFile(t *testing.T) {
// Create a TIM
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
m.RootFS.AddData("test.txt", []byte("test"))
// Encrypt
password := "runtest"
stim, err := m.ToSigil(password)
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
// Write to temp file
tempFile := filepath.Join(t.TempDir(), "test.stim")
if err := os.WriteFile(tempFile, stim, 0600); err != nil {
t.Fatalf("Failed to write temp file: %v", err)
}
// RunEncrypted will fail because runc is not available in test,
// but it should at least decrypt successfully
err = RunEncrypted(tempFile, password)
// We expect an error about runc not being found, not about decryption
if err != nil && err.Error() != "" {
// Check it's not a decryption error
if err.Error() == ErrDecryptionFailed.Error() {
t.Errorf("Unexpected decryption error: %v", err)
}
}
}
func TestCacheSize(t *testing.T) {
tempDir, err := os.MkdirTemp("", "borg-cache-size-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tempDir)
password := "sizetest"
cache, err := NewCache(tempDir, password)
if err != nil {
t.Fatalf("NewCache() error = %v", err)
}
// Create and store a TIM
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
m.RootFS.AddData("test.txt", []byte("Size test content"))
if err := cache.Store("sizetest", m); err != nil {
t.Fatalf("Store() error = %v", err)
}
// Get size
size, err := cache.Size("sizetest")
if err != nil {
t.Fatalf("Size() error = %v", err)
}
if size <= 0 {
t.Errorf("Size() = %d, want > 0", size)
}
}
func TestCacheSizeNotFound(t *testing.T) {
tempDir, err := os.MkdirTemp("", "borg-cache-size-test-*")
if err != nil {
t.Fatalf("Failed to create temp dir: %v", err)
}
defer os.RemoveAll(tempDir)
cache, err := NewCache(tempDir, "password")
if err != nil {
t.Fatalf("NewCache() error = %v", err)
}
_, err = cache.Size("nonexistent")
if err == nil {
t.Error("Expected error for nonexistent TIM")
}
}
func TestFromTar(t *testing.T) {
// Create a TIM and convert to tar
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
m.RootFS.AddData("test.txt", []byte("Test content"))
m.RootFS.AddData("subdir/nested.txt", []byte("Nested content"))
tarData, err := m.ToTar()
if err != nil {
t.Fatalf("ToTar() error = %v", err)
}
// Parse the tar back to a TIM
restored, err := FromTar(tarData)
if err != nil {
t.Fatalf("FromTar() error = %v", err)
}
// Verify config
if string(restored.Config) != string(m.Config) {
t.Error("Config mismatch after FromTar")
}
// Verify files
file, err := restored.RootFS.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open test.txt: %v", err)
}
file.Close()
file, err = restored.RootFS.Open("subdir/nested.txt")
if err != nil {
t.Fatalf("Failed to open subdir/nested.txt: %v", err)
}
file.Close()
}
func TestFromTarInvalidData(t *testing.T) {
_, err := FromTar([]byte("not a tar file"))
if err == nil {
t.Error("Expected error for invalid tar data")
}
}
func TestFromTarNoConfig(t *testing.T) {
// Create tar without config.json
_, err := FromTar([]byte{})
if err == nil {
t.Error("Expected error for tar without config.json")
}
}
func TestToTarNilConfig(t *testing.T) {
m := &TerminalIsolationMatrix{
Config: nil,
RootFS: nil,
}
_, err := m.ToTar()
if err != ErrConfigIsNil {
t.Errorf("Expected ErrConfigIsNil, got %v", err)
}
}
func TestToSigilNilConfig(t *testing.T) {
m := &TerminalIsolationMatrix{
Config: nil,
RootFS: nil,
}
_, err := m.ToSigil("password")
if err != ErrConfigIsNil {
t.Errorf("Expected ErrConfigIsNil, got %v", err)
}
}
func TestFromSigilInvalidData(t *testing.T) {
_, err := FromSigil([]byte("invalid stim data"), "password")
if err == nil {
t.Error("Expected error for invalid stim data")
}
}
func TestFromSigilTruncatedPayload(t *testing.T) {
// Create valid STIM but truncate the payload
m, err := New()
if err != nil {
t.Fatalf("New() error = %v", err)
}
stim, err := m.ToSigil("password")
if err != nil {
t.Fatalf("ToSigil() error = %v", err)
}
// Truncate to just magic + partial header
if len(stim) > 20 {
_, err = FromSigil(stim[:20], "password")
if err == nil {
t.Error("Expected error for truncated payload")
}
}
}
func TestFromDataNodeNil(t *testing.T) {
_, err := FromDataNode(nil)
if err != ErrDataNodeRequired {
t.Errorf("Expected ErrDataNodeRequired, got %v", err)
}
}

173
pkg/tim/tim.go
View file

@ -3,16 +3,26 @@ package tim
import (
"archive/tar"
"bytes"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"strings"
"github.com/Snider/Borg/pkg/datanode"
borgtrix "github.com/Snider/Borg/pkg/trix"
"github.com/Snider/Enchantrix/pkg/enchantrix"
"github.com/Snider/Enchantrix/pkg/trix"
)
var (
ErrDataNodeRequired = errors.New("datanode is required")
ErrConfigIsNil = errors.New("config is nil")
ErrDataNodeRequired = errors.New("datanode is required")
ErrConfigIsNil = errors.New("config is nil")
ErrPasswordRequired = errors.New("password is required for encryption")
ErrInvalidStimPayload = errors.New("invalid stim payload")
ErrDecryptionFailed = errors.New("decryption failed (wrong password?)")
)
// TerminalIsolationMatrix represents a runc bundle.
@ -54,6 +64,49 @@ func FromDataNode(dn *datanode.DataNode) (*TerminalIsolationMatrix, error) {
return m, nil
}
// FromTar creates a TerminalIsolationMatrix from a tarball.
// The tarball must contain config.json and a rootfs/ directory.
func FromTar(data []byte) (*TerminalIsolationMatrix, error) {
tr := tar.NewReader(bytes.NewReader(data))
var config []byte
rootfs := datanode.New()
for {
hdr, err := tr.Next()
if err != nil {
break
}
if hdr.Name == "config.json" {
config, err = io.ReadAll(tr)
if err != nil {
return nil, fmt.Errorf("failed to read config.json: %w", err)
}
} else if strings.HasPrefix(hdr.Name, "rootfs/") && hdr.Typeflag == tar.TypeReg {
// Strip "rootfs/" prefix
name := strings.TrimPrefix(hdr.Name, "rootfs/")
if name == "" {
continue
}
content, err := io.ReadAll(tr)
if err != nil {
return nil, fmt.Errorf("failed to read %s: %w", hdr.Name, err)
}
rootfs.AddData(name, content)
}
}
if config == nil {
return nil, ErrConfigIsNil
}
return &TerminalIsolationMatrix{
Config: config,
RootFS: rootfs,
}, nil
}
// ToTar serializes the TerminalIsolationMatrix to a tarball.
func (m *TerminalIsolationMatrix) ToTar() ([]byte, error) {
if m.Config == nil {
@ -141,7 +194,115 @@ func (m *TerminalIsolationMatrix) ToTar() ([]byte, error) {
return buf.Bytes(), nil
}
// ToTrix is not yet implemented.
// func (m *TerminalIsolationMatrix) ToTrix(password string) ([]byte, error) {
// return nil, errors.New("not implemented")
// }
// ToSigil serializes and encrypts the TIM to .stim format using ChaChaPolySigil.
// Config and RootFS are encrypted separately.
// The output format is a Trix container with "STIM" magic containing:
// - Header: {"encryption_algorithm": "chacha20poly1305", "tim": true}
// - Payload: [config_size(4 bytes)][encrypted_config][encrypted_rootfs]
func (m *TerminalIsolationMatrix) ToSigil(password string) ([]byte, error) {
if password == "" {
return nil, ErrPasswordRequired
}
if m.Config == nil {
return nil, ErrConfigIsNil
}
key := borgtrix.DeriveKey(password)
sigil, err := enchantrix.NewChaChaPolySigil(key)
if err != nil {
return nil, fmt.Errorf("failed to create sigil: %w", err)
}
// Encrypt config
encConfig, err := sigil.In(m.Config)
if err != nil {
return nil, fmt.Errorf("failed to encrypt config: %w", err)
}
// Get rootfs as tar
rootfsTar, err := m.RootFS.ToTar()
if err != nil {
return nil, fmt.Errorf("failed to serialize rootfs: %w", err)
}
// Encrypt rootfs
encRootFS, err := sigil.In(rootfsTar)
if err != nil {
return nil, fmt.Errorf("failed to encrypt rootfs: %w", err)
}
// Build payload: [config_size(4 bytes)][encrypted_config][encrypted_rootfs]
payload := make([]byte, 4+len(encConfig)+len(encRootFS))
binary.BigEndian.PutUint32(payload[:4], uint32(len(encConfig)))
copy(payload[4:4+len(encConfig)], encConfig)
copy(payload[4+len(encConfig):], encRootFS)
// Create trix container
t := &trix.Trix{
Header: map[string]interface{}{
"encryption_algorithm": "chacha20poly1305",
"tim": true,
"config_size": len(encConfig),
"rootfs_size": len(encRootFS),
"version": "1.0",
},
Payload: payload,
}
return trix.Encode(t, "STIM", nil)
}
// FromSigil decrypts and deserializes a .stim file into a TerminalIsolationMatrix.
func FromSigil(data []byte, password string) (*TerminalIsolationMatrix, error) {
if password == "" {
return nil, ErrPasswordRequired
}
// Decode the trix container
t, err := trix.Decode(data, "STIM", nil)
if err != nil {
return nil, fmt.Errorf("failed to decode stim: %w", err)
}
key := borgtrix.DeriveKey(password)
sigil, err := enchantrix.NewChaChaPolySigil(key)
if err != nil {
return nil, fmt.Errorf("failed to create sigil: %w", err)
}
// Parse payload structure
if len(t.Payload) < 4 {
return nil, ErrInvalidStimPayload
}
configSize := binary.BigEndian.Uint32(t.Payload[:4])
if len(t.Payload) < int(4+configSize) {
return nil, ErrInvalidStimPayload
}
encConfig := t.Payload[4 : 4+configSize]
encRootFS := t.Payload[4+configSize:]
// Decrypt config
config, err := sigil.Out(encConfig)
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err)
}
// Decrypt rootfs
rootfsTar, err := sigil.Out(encRootFS)
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err)
}
// Reconstruct DataNode from tar
rootfs, err := datanode.FromTar(rootfsTar)
if err != nil {
return nil, fmt.Errorf("failed to parse rootfs: %w", err)
}
return &TerminalIsolationMatrix{
Config: config,
RootFS: rootfs,
}, nil
}

80
pkg/trix/trix.go
View file

@ -1,12 +1,21 @@
package trix
import (
"crypto/sha256"
"errors"
"fmt"
"github.com/Snider/Borg/pkg/datanode"
"github.com/Snider/Enchantrix/pkg/crypt"
"github.com/Snider/Enchantrix/pkg/enchantrix"
"github.com/Snider/Enchantrix/pkg/trix"
)
var (
ErrPasswordRequired = errors.New("password is required for encryption")
ErrDecryptionFailed = errors.New("decryption failed (wrong password?)")
)
// ToTrix converts a DataNode to the Trix format.
func ToTrix(dn *datanode.DataNode, password string) ([]byte, error) {
// Convert the DataNode to a tarball.
@ -49,3 +58,74 @@ func FromTrix(data []byte, password string) (*datanode.DataNode, error) {
// Convert the tarball back to a DataNode.
return datanode.FromTar(t.Payload)
}
// DeriveKey derives a 32-byte key from a password using SHA-256.
// This is used for ChaCha20-Poly1305 encryption which requires a 32-byte key.
func DeriveKey(password string) []byte {
hash := sha256.Sum256([]byte(password))
return hash[:]
}
// ToTrixChaCha converts a DataNode to encrypted Trix format using ChaCha20-Poly1305.
func ToTrixChaCha(dn *datanode.DataNode, password string) ([]byte, error) {
if password == "" {
return nil, ErrPasswordRequired
}
// Convert the DataNode to a tarball.
tarball, err := dn.ToTar()
if err != nil {
return nil, err
}
// Create sigil and encrypt
key := DeriveKey(password)
sigil, err := enchantrix.NewChaChaPolySigil(key)
if err != nil {
return nil, fmt.Errorf("failed to create sigil: %w", err)
}
encrypted, err := sigil.In(tarball)
if err != nil {
return nil, fmt.Errorf("failed to encrypt: %w", err)
}
// Create a Trix struct with encryption metadata.
t := &trix.Trix{
Header: map[string]interface{}{
"encryption_algorithm": "chacha20poly1305",
},
Payload: encrypted,
}
// Encode the Trix struct.
return trix.Encode(t, "TRIX", nil)
}
// FromTrixChaCha decrypts a ChaCha-encrypted Trix byte slice back to a DataNode.
func FromTrixChaCha(data []byte, password string) (*datanode.DataNode, error) {
if password == "" {
return nil, ErrPasswordRequired
}
// Decode the Trix byte slice.
t, err := trix.Decode(data, "TRIX", nil)
if err != nil {
return nil, err
}
// Create sigil and decrypt
key := DeriveKey(password)
sigil, err := enchantrix.NewChaChaPolySigil(key)
if err != nil {
return nil, fmt.Errorf("failed to create sigil: %w", err)
}
decrypted, err := sigil.Out(t.Payload)
if err != nil {
return nil, fmt.Errorf("%w: %v", ErrDecryptionFailed, err)
}
// Convert the tarball back to a DataNode.
return datanode.FromTar(decrypted)
}

238
pkg/trix/trix_test.go Normal file
View file

@ -0,0 +1,238 @@
package trix
import (
"testing"
"github.com/Snider/Borg/pkg/datanode"
)
func TestDeriveKey(t *testing.T) {
// Test key length
key := DeriveKey("password")
if len(key) != 32 {
t.Errorf("DeriveKey() returned key of length %d, want 32", len(key))
}
// Same password should produce same key
key2 := DeriveKey("password")
for i := range key {
if key[i] != key2[i] {
t.Error("DeriveKey() not deterministic")
break
}
}
// Different password should produce different key
key3 := DeriveKey("different")
same := true
for i := range key {
if key[i] != key3[i] {
same = false
break
}
}
if same {
t.Error("DeriveKey() produced same key for different passwords")
}
}
func TestToTrix(t *testing.T) {
t.Run("without password", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrix(dn, "")
if err != nil {
t.Fatalf("ToTrix() error = %v", err)
}
// Verify magic number
if len(data) < 4 || string(data[:4]) != "TRIX" {
t.Errorf("Expected magic 'TRIX', got '%s'", string(data[:4]))
}
})
t.Run("with password", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrix(dn, "secret")
if err != nil {
t.Fatalf("ToTrix() error = %v", err)
}
// Verify magic number
if len(data) < 4 || string(data[:4]) != "TRIX" {
t.Errorf("Expected magic 'TRIX', got '%s'", string(data[:4]))
}
})
}
func TestFromTrix(t *testing.T) {
t.Run("without password round-trip", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrix(dn, "")
if err != nil {
t.Fatalf("ToTrix() error = %v", err)
}
restored, err := FromTrix(data, "")
if err != nil {
t.Fatalf("FromTrix() error = %v", err)
}
// Verify file exists
file, err := restored.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open test.txt: %v", err)
}
defer file.Close()
})
t.Run("with password returns error", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrix(dn, "")
if err != nil {
t.Fatalf("ToTrix() error = %v", err)
}
// FromTrix with password should return error (decryption disabled)
_, err = FromTrix(data, "password")
if err == nil {
t.Error("Expected error when providing password to FromTrix")
}
})
t.Run("invalid data", func(t *testing.T) {
_, err := FromTrix([]byte("invalid"), "")
if err == nil {
t.Error("Expected error for invalid data")
}
})
}
func TestToTrixChaCha(t *testing.T) {
t.Run("success", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrixChaCha(dn, "password")
if err != nil {
t.Fatalf("ToTrixChaCha() error = %v", err)
}
// Verify magic number
if len(data) < 4 || string(data[:4]) != "TRIX" {
t.Errorf("Expected magic 'TRIX', got '%s'", string(data[:4]))
}
})
t.Run("empty password", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
_, err := ToTrixChaCha(dn, "")
if err != ErrPasswordRequired {
t.Errorf("Expected ErrPasswordRequired, got %v", err)
}
})
}
func TestFromTrixChaCha(t *testing.T) {
t.Run("round-trip", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
dn.AddData("subdir/nested.txt", []byte("Nested content"))
password := "testpassword123"
// Encrypt
data, err := ToTrixChaCha(dn, password)
if err != nil {
t.Fatalf("ToTrixChaCha() error = %v", err)
}
// Decrypt
restored, err := FromTrixChaCha(data, password)
if err != nil {
t.Fatalf("FromTrixChaCha() error = %v", err)
}
// Verify files exist
file, err := restored.Open("test.txt")
if err != nil {
t.Fatalf("Failed to open test.txt: %v", err)
}
file.Close()
file, err = restored.Open("subdir/nested.txt")
if err != nil {
t.Fatalf("Failed to open subdir/nested.txt: %v", err)
}
file.Close()
})
t.Run("empty password", func(t *testing.T) {
_, err := FromTrixChaCha([]byte("data"), "")
if err != ErrPasswordRequired {
t.Errorf("Expected ErrPasswordRequired, got %v", err)
}
})
t.Run("wrong password", func(t *testing.T) {
dn := datanode.New()
dn.AddData("test.txt", []byte("Hello, World!"))
data, err := ToTrixChaCha(dn, "correct")
if err != nil {
t.Fatalf("ToTrixChaCha() error = %v", err)
}
_, err = FromTrixChaCha(data, "wrong")
if err == nil {
t.Error("Expected error with wrong password")
}
})
t.Run("invalid data", func(t *testing.T) {
_, err := FromTrixChaCha([]byte("invalid"), "password")
if err == nil {
t.Error("Expected error for invalid data")
}
})
}
func TestToTrixChaChaWithLargeData(t *testing.T) {
dn := datanode.New()
// Add large file
largeData := make([]byte, 1024*1024) // 1MB
for i := range largeData {
largeData[i] = byte(i % 256)
}
dn.AddData("large.bin", largeData)
password := "largetest"
// Encrypt
data, err := ToTrixChaCha(dn, password)
if err != nil {
t.Fatalf("ToTrixChaCha() error = %v", err)
}
// Decrypt
restored, err := FromTrixChaCha(data, password)
if err != nil {
t.Fatalf("FromTrixChaCha() error = %v", err)
}
// Verify file exists
_, err = restored.Open("large.bin")
if err != nil {
t.Fatalf("Failed to open large.bin: %v", err)
}
}