diff --git a/Node-Architecture.-.md b/Architecture.md similarity index 100% rename from Node-Architecture.-.md rename to Architecture.md diff --git a/Fleet-Context.md b/Fleet-Context.md new file mode 100644 index 0000000..4285a27 --- /dev/null +++ b/Fleet-Context.md @@ -0,0 +1,53 @@ +# Fleet Context + +How this repo fits in the wider Core Go agent fleet. + +## Your Role + +You are the **dedicated go-p2p domain expert**. You own this repo end-to-end. + +## The Contract + +| File | You Read | You Write | +|------|----------|-----------| +| `CLAUDE.md` | Yes — build commands, architecture, standards | No (Charon maintains) | +| `TODO.md` | Yes — pick up tasks in phase order | Yes — mark `[x]` when done | +| `FINDINGS.md` | Yes — previous discoveries | Yes — add new findings | + +## Who's Who + +| Agent | Where | What They Do | +|-------|-------|-------------| +| **Virgil** | M3 Ultra (macOS) | Framework orchestrator, owns core/go | +| **Charon** | snider-linux (this machine) | Linux orchestrator, environment setup, plan review | +| **go-rocm Claude** | snider-linux | AMD GPU inference — Phase 1-4 complete | +| **go-rag Claude** | snider-linux | RAG with Qdrant + Ollama (pending) | +| **You** | snider-linux | go-p2p networking layer | + +## Dependencies + +``` +go-p2p (you implement/test this) + ↓ uses +Borg (STMF crypto, SMSG encryption) +Poindexter (KD-tree peer scoring) + ↑ consumed by +go-ai (future: P2P MCP transport?) +core/cli (future: CLI commands for node management) +``` + +go-p2p is currently standalone — it doesn't depend on other `core/go-*` packages. Its consumers will come later. + +## Communication + +- **New findings**: Write to `FINDINGS.md` in this repo +- **Cross-repo blockers**: Note in `TODO.md` with "Blocked on [package] [phase]" +- **Questions for Charon**: Note in `FINDINGS.md` with "QUESTION:" prefix + +## What You Don't Need to Know + +- How go-rocm works (different domain entirely) +- Infrastructure details (Charon handles that) +- How the fleet delegation pattern works (just follow TODO.md) + +Focus on implementing the TODO.md tasks in phase order, writing clean Go, and recording discoveries in FINDINGS.md. diff --git a/Home.md b/Home.md index 7e06e88..9ad611b 100644 --- a/Home.md +++ b/Home.md @@ -1,67 +1,50 @@ # go-p2p -> P2P node identity and message protocol for multi-node mining and compute orchestration. +P2P networking layer for the Lethean network. Encrypted WebSocket mesh with UEPS wire protocol. -**Module**: `forge.lthn.ai/core/go-p2p` -**Go**: 1.25.5 +**Module**: `forge.lthn.ai/core/go-p2p` +**Go**: 1.25.5 **Licence**: EUPL-1.2 -## Overview - -go-p2p provides the networking layer for Lethean's distributed compute mesh. It handles node identity (X25519 keypair-derived), WebSocket-based messaging, challenge-response authentication, and peer discovery with latency/geo scoring via Poindexter. - ## Package Structure -| Package | Purpose | -|---------|---------| -| `node/` | Core types — peers, messages, identity, protocol handlers | -| `logging/` | Structured logging with configurable levels | -| `ueps/` | UEPS protocol packet encoding and stream reader | +| Package | Purpose | Coverage | +|---------|---------|----------| +| `node/` | P2P mesh: identity, transport, peers, protocol, worker, controller | 42% | +| `ueps/` | UEPS wire protocol: TLV encoding, HMAC signing, stream parsing | **0%** | +| `logging/` | Structured levelled logger | 86% | -## Quick Start +## Phase Status -```go -import "forge.lthn.ai/core/go-p2p/node" +| Phase | Status | Notes | +|-------|--------|-------| +| **Extraction** | Done | Extracted from Mining repo (commit `8f94639`) | +| **Test Coverage** | **Active** | High priority: ueps (0%), transport (0 tests), controller (0 tests) | +| **Dispatcher** | Not started | Stub exists, needs UEPS routing implementation | +| **Integration** | Not started | Node-to-node handshake over localhost | -// Create a controller identity -identity := node.NewNodeIdentity("my-node", node.RoleController) +## Critical Gaps -// Generate a challenge for peer authentication -challenge := node.GenerateChallenge() -response := node.SignChallenge(challenge, sharedSecret) -ok := node.VerifyChallenge(challenge, response, sharedSecret) -``` - -## Protocol - -- **Version**: `1.0` -- **Key Exchange**: X25519 ECDH -- **Authentication**: HMAC-SHA256 challenge-response -- **Transport**: WebSocket (gorilla/websocket) - -## API Overview - -| Type | Description | -|------|-------------| -| `NodeIdentity` | Node identity derived from X25519 public key | -| `Peer` | Remote peer with address, role, latency, and scoring | -| `Message` | Protocol message with typed payload (JSON) | -| `MessageType` | Enumeration of all protocol message types | -| `ProtocolError` | Structured error with code and message | -| `ResponseHandler` | Helpers for validating and parsing responses | +1. **UEPS has ZERO tests** — 262 lines of crypto wire protocol completely untested +2. **Transport (934 lines)** — WebSocket+SMSG encryption, handshake, keepalive, rate limiting — all untested +3. **Controller (327 lines)** — Request-response correlation, auto-connect, concurrent stats — all untested ## Dependencies | Module | Purpose | |--------|---------| -| `github.com/Snider/Borg` | Secure blob and pointer storage (stmf) | -| `github.com/Snider/Poindexter` | Latency, hops, geo-distance, reliability metrics | -| `github.com/google/uuid` | Message ID generation | +| `github.com/Snider/Borg` | STMF crypto, SMSG encryption, TIM bundles | +| `github.com/Snider/Poindexter` | KD-tree peer scoring | +| `github.com/google/uuid` | Message/peer ID generation | | `github.com/gorilla/websocket` | WebSocket transport | | `github.com/adrg/xdg` | XDG base directory paths | ## Pages -- [[Node-Identity]] — Identity creation, roles, and authentication -- [[Protocol-Messages]] — Message types, format, and payload handling -- [[Peer-Discovery]] — Peer management, scoring, and Poindexter integration +- [Architecture](Architecture) — Component layers and data flow +- [Test-Gaps](Test-Gaps) — Detailed coverage analysis with test suggestions +- [UEPS-Protocol](UEPS-Protocol) — Wire protocol specification +- [Node-Identity](Node-Identity) — Identity, roles, authentication +- [Protocol-Messages](Protocol-Messages) — Message types and payload handling +- [Peer-Discovery](Peer-Discovery) — Peer management and scoring +- [Fleet-Context](Fleet-Context) — How this fits in the Core Go ecosystem diff --git a/Node-Identity.-.md b/Node-Identity.md similarity index 100% rename from Node-Identity.-.md rename to Node-Identity.md diff --git a/Peer-Discovery.-.md b/Peer-Discovery.md similarity index 100% rename from Peer-Discovery.-.md rename to Peer-Discovery.md diff --git a/Protocol-Messages.-.md b/Protocol-Messages.md similarity index 100% rename from Protocol-Messages.-.md rename to Protocol-Messages.md diff --git a/Test-Gaps.md b/Test-Gaps.md new file mode 100644 index 0000000..b255dcb --- /dev/null +++ b/Test-Gaps.md @@ -0,0 +1,113 @@ +# Test Gaps + +Detailed analysis of untested code to guide the dedicated domain expert. + +## Current State + +``` +logging/ 86.2% coverage — 12 tests (good) +node/ 42.0% coverage — 76 tests (critical gaps below) +ueps/ 0.0% coverage — 0 tests (CRITICAL) +``` + +## Priority 1: UEPS Wire Protocol (0 tests, 262 lines) + +### packet.go — PacketBuilder + MarshalAndSign + +| Test | What to verify | +|------|---------------| +| `TestNewBuilder_DefaultFields` | Version=0x09, CurrentLayer=5, TargetLayer=5, ThreatScore=0 | +| `TestMarshalAndSign_Roundtrip` | Build packet, marshal, then ReadAndVerify, assert fields match | +| `TestMarshalAndSign_HMACCoversHeadersAndPayload` | Tamper with payload after signing, verify HMAC rejects | +| `TestMarshalAndSign_EmptyPayload` | Payload=nil, should still produce valid signed packet | +| `TestMarshalAndSign_MaxThreatScore` | ThreatScore=65535, verify uint16 boundary | +| `TestWriteTLV_ValueTooLarge` | Value >255 bytes should error (1-byte length limit) | + +### reader.go — ReadAndVerify + +| Test | What to verify | +|------|---------------| +| `TestReadAndVerify_ValidPacket` | Full roundtrip from MarshalAndSign | +| `TestReadAndVerify_TamperedPayload` | Flip a payload byte, expect HMAC mismatch error | +| `TestReadAndVerify_TamperedHeader` | Change header byte after signing, expect HMAC mismatch | +| `TestReadAndVerify_MissingHMAC` | Packet with no 0x06 tag, expect "missing HMAC" error | +| `TestReadAndVerify_WrongSecret` | Sign with key A, verify with key B, expect mismatch | +| `TestReadAndVerify_TruncatedPacket` | Short read / EOF mid-TLV, expect io error | +| `TestReadAndVerify_UnknownTag` | Unknown TLV tag between header and HMAC, verify it's skipped but included in signature | + +### Key Implementation Note + +The 0xFF payload tag has **no length prefix** — reader.go calls `io.ReadAll(r)` which reads until EOF. This means the caller must frame the packet externally (e.g. TCP length prefix). Test with `bytes.NewReader(data)` which naturally provides EOF framing. + +## Priority 2: Transport (0 tests, 934 lines) + +Testing transport requires spinning up real WebSocket connections. Use `httptest.NewServer` with the transport's `handleWSUpgrade` handler. + +### Challenges + +- **SMSG encryption**: `encryptMessage`/`decryptMessage` use `github.com/Snider/Borg/pkg/smsg`. Tests need a real shared secret derived from two X25519 keypairs. +- **Handshake**: Requires two `NodeManager` instances with valid identities. Each side does challenge-response auth. + +### Suggested Test Approach + +```go +func setupTestPair(t *testing.T) (serverTransport *Transport, clientTransport *Transport) { + // Create two identities + serverNode := NewNodeManager(NewNodeIdentity("server", RoleWorker)) + clientNode := NewNodeManager(NewNodeIdentity("client", RoleController)) + + // Create registries (open auth for tests) + serverRegistry := NewPeerRegistry(WithAuthMode(PeerAuthOpen)) + clientRegistry := NewPeerRegistry(WithAuthMode(PeerAuthOpen)) + + // Create transports on different ports + serverCfg := DefaultTransportConfig() + serverCfg.ListenAddr = ":0" // kernel-assigned + // ... +} +``` + +### Key Tests Needed + +| Test | What to verify | +|------|---------------| +| `TestTransport_ConnectAndHandshake` | Full handshake, shared secret derived, challenge verified | +| `TestTransport_HandshakeRejectWrongVersion` | Peer with incompatible protocol version rejected | +| `TestTransport_AllowlistReject` | Peer not in allowlist gets rejection message | +| `TestTransport_SendEncrypted` | Send message, verify SMSG decrypt on receiver | +| `TestTransport_Deduplication` | Send same message ID twice, second is dropped | +| `TestTransport_RateLimiting` | Burst >100 messages, verify drops after limit | +| `TestTransport_MaxConns` | Hit MaxConns limit, verify 503 response | +| `TestTransport_KeepaliveTimeout` | No activity beyond PingInterval+PongTimeout, verify disconnect | +| `TestTransport_GracefulClose` | GracefulClose sends disconnect message before closing | +| `TestTransport_ConcurrentSend` | Multiple goroutines sending, no races (writeMu) | + +## Priority 3: Controller (0 tests, 327 lines) + +Controller wraps transport for request-response patterns. Test by mocking or using a real transport pair. + +| Test | What to verify | +|------|---------------| +| `TestController_SendRequest` | Send message, get correlated response via ReplyTo | +| `TestController_Timeout` | No response within deadline, returns timeout error | +| `TestController_AutoConnect` | Peer not connected, controller auto-connects before sending | +| `TestController_GetAllStats` | Multiple peers, parallel stat collection | +| `TestController_PingPeer` | RTT calculated correctly | +| `TestController_ConcurrentRequests` | Multiple requests in flight, correct correlation | + +## Priority 4: Dispatcher Implementation + +Currently a commented-out stub. Once UEPS tests exist and transport tests pass, implement: + +1. Threat circuit breaker (ThreatScore > 50000 = drop) +2. Intent-based routing (0x01 handshake, 0x20 compute, 0x30 rehab, 0xFF custom) +3. Integration with Transport for incoming UEPS packets + +## What's Already Well-Tested + +- `identity.go` — 5 tests, good coverage +- `peer.go` — 19 tests, good coverage (scoring, allowlist, persistence) +- `message.go` — 8 tests, good coverage (all message types, serialisation) +- `worker.go` — 10 tests, good coverage (command handlers with mock managers) +- `bundle.go` — 9 tests, good coverage (TIM encryption, Zip Slip defence) +- `protocol.go` — 5 tests, good coverage (response validation) diff --git a/UEPS-Protocol.-.md b/UEPS-Protocol.md similarity index 100% rename from UEPS-Protocol.-.md rename to UEPS-Protocol.md