61e01bfdf1
Consolidates three codebases into a single agent orchestration repo: - agentci (from go-scm): Clotho dual-run verification, agent config, SSH security (sanitisation, secure commands, token masking) - jobrunner (from go-scm): Poll-dispatch-report pipeline with 7 handlers (dispatch, completion, auto-merge, publish draft, dismiss reviews, send fix command, tick parent epic) - plugins marketplace (from agentic/plugins): 27 Claude/Codex/Gemini plugins with shared MCP server All 150+ tests passing across 6 packages. Co-Authored-By: Virgil <virgil@lethean.io>
1.3 KiB
1.3 KiB
| name | description | args |
|---|---|---|
| website | Crawl and collect a website using Borg | <url> [--depth N] [--format stim|tim|tar] [-o output] |
Website Collection
Crawl and collect websites using Borg.
Usage
/collect:website https://getmasari.org
/collect:website https://docs.lethean.io --depth 3
/collect:website https://graft.network --format stim -o graft-site.stim
Action
Run Borg to crawl the website:
borg collect website <url> [--depth <N>] [--format <format>] [-o <output>]
Default depth is 2 levels.
Options
| Option | Default | Description |
|---|---|---|
--depth |
2 | How many levels deep to crawl |
--format |
tar | Output format (tar, tim, stim) |
-o |
auto | Output filename |
Examples
# Basic crawl
borg collect website https://getmasari.org
# Deep crawl with encryption
borg collect website https://docs.lethean.io --depth 5 --format stim -o lethean-docs.stim
# Wayback Machine archive
borg collect website "https://web.archive.org/web/*/graft.network" --depth 3
Use Cases
- Project Documentation - Archive docs before they go offline
- Wayback Snapshots - Collect historical versions
- Forum Threads - Archive discussion pages
- PWA Collection - Use
borg collect pwafor progressive web apps