e90a84eaa0
Combines three repositories into a single workspace: - go-agent → pkg/orchestrator (Clotho), pkg/jobrunner, pkg/loop, cmd/ - go-agentic → pkg/lifecycle (allowance, sessions, plans, dispatch) - php-devops → repos.yaml, setup.sh, scripts/, .core/ Module path: forge.lthn.ai/core/agent All packages build, all tests pass. Co-Authored-By: Virgil <virgil@lethean.io>
1.3 KiB
1.3 KiB
| name | description | args |
|---|---|---|
| website | Crawl and collect a website using Borg | <url> [--depth N] [--format stim|tim|tar] [-o output] |
Website Collection
Crawl and collect websites using Borg.
Usage
/collect:website https://getmasari.org
/collect:website https://docs.lethean.io --depth 3
/collect:website https://graft.network --format stim -o graft-site.stim
Action
Run Borg to crawl the website:
borg collect website <url> [--depth <N>] [--format <format>] [-o <output>]
Default depth is 2 levels.
Options
| Option | Default | Description |
|---|---|---|
--depth |
2 | How many levels deep to crawl |
--format |
tar | Output format (tar, tim, stim) |
-o |
auto | Output filename |
Examples
# Basic crawl
borg collect website https://getmasari.org
# Deep crawl with encryption
borg collect website https://docs.lethean.io --depth 5 --format stim -o lethean-docs.stim
# Wayback Machine archive
borg collect website "https://web.archive.org/web/*/graft.network" --depth 3
Use Cases
- Project Documentation - Archive docs before they go offline
- Wayback Snapshots - Collect historical versions
- Forum Threads - Archive discussion pages
- PWA Collection - Use
borg collect pwafor progressive web apps