php-uptelligence/docs/architecture.md

title	description	updated
Architecture	Technical architecture of core-uptelligence	2026-01-29

Architecture

The core-uptelligence package provides upstream vendor tracking and dependency intelligence for the Host UK platform. It monitors software vendors, analyses version differences, generates porting tasks, and manages notification digests.

Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                           External Systems                                   │
├─────────────┬─────────────┬─────────────┬─────────────┬────────────────────┤
│   GitHub    │   GitLab    │     npm     │  Packagist  │   AI Providers     │
│  Releases   │  Releases   │   Publish   │   Updates   │ (Anthropic/OpenAI) │
└──────┬──────┴──────┬──────┴──────┬──────┴──────┬──────┴─────────┬──────────┘
       │             │             │             │                 │
       └─────────────┴─────────────┴─────────────┘                 │
                           │                                       │
                    ┌──────▼──────┐                                │
                    │   Webhook   │                                │
                    │  Controller │                                │
                    └──────┬──────┘                                │
                           │                                       │
                    ┌──────▼──────┐                         ┌──────▼──────┐
                    │   Webhook   │                         │     AI      │
                    │  Receiver   │                         │  Analyzer   │
                    │   Service   │                         │   Service   │
                    └──────┬──────┘                         └──────┬──────┘
                           │                                       │
       ┌───────────────────┼───────────────────────────────────────┤
       │                   │                                       │
┌──────▼──────┐     ┌──────▼──────┐     ┌──────────────┐    ┌──────▼──────┐
│   Vendor    │     │   Version   │     │     Diff     │    │   Upstream  │
│   Model     │◄────┤   Release   │────►│   Analyzer   │───►│    Todo     │
└─────────────┘     └─────────────┘     │   Service    │    └─────────────┘
                                        └──────────────┘           │
                                                                   │
                    ┌──────────────┐                        ┌──────▼──────┐
                    │    Issue     │◄───────────────────────┤    Issue    │
                    │   (GitHub/   │                        │  Generator  │
                    │    Gitea)    │                        │   Service   │
                    └──────────────┘                        └─────────────┘

Module Registration

The package registers as a Laravel service provider via Boot.php and uses the event-driven module loading pattern from core-php.

Event Listeners

public static array $listens = [
    AdminPanelBooting::class => 'onAdminPanel',
    ApiRoutesRegistering::class => 'onApiRoutes',
    ConsoleBooting::class => 'onConsole',
];

Registered Services

All services are registered as singletons:

Service	Purpose
VendorStorageService	Local and S3 file storage for vendor archives
VendorUpdateCheckerService	Polls registries (GitHub, Packagist, npm) for updates
DiffAnalyzerService	Generates file diffs between versions
AIAnalyzerService	Uses AI to analyse diffs and categorise changes
IssueGeneratorService	Creates GitHub/Gitea issues from todos
UptelligenceDigestService	Compiles and sends digest notifications
WebhookReceiverService	Processes incoming vendor release webhooks
AssetTrackerService	Tracks package dependencies (Composer, npm)

Data Model

Core Entities

┌─────────────┐     ┌─────────────────┐     ┌─────────────┐
│   Vendor    │────►│ VersionRelease  │────►│  DiffCache  │
└─────────────┘     └─────────────────┘     └─────────────┘
      │                     │
      │                     │
      ▼                     ▼
┌─────────────┐     ┌─────────────────┐
│UpstreamTodo │     │  AnalysisLog    │
└─────────────┘     └─────────────────┘

Vendor

Represents an upstream software source (licensed, OSS, or plugin).

Key attributes:

• slug - Unique identifier
• source_type - licensed, oss, or plugin
• git_repo_url - Repository URL for OSS vendors
• path_mapping - Maps upstream paths to target paths
• ignored_paths - Patterns to skip during analysis
• priority_paths - High-importance file patterns
• target_repo - GitHub/Gitea repo for issue creation

VersionRelease

Tracks a specific version of a vendor's software.

Key attributes:

• version / previous_version - Version comparison
• storage_disk - local or s3
• s3_key - Archive location in cold storage
• file_hash - SHA-256 for integrity verification
• summary - AI-generated release summary

UpstreamTodo

A porting task generated from version analysis.

Key attributes:

• type - feature, bugfix, security, ui, block, api, refactor, dependency
• status - pending, in_progress, ported, skipped, wont_port
• priority - 1-10 scale
• effort - low, medium, high
• ai_analysis - Raw AI analysis data
• github_issue_number - Linked issue

DiffCache

Stores individual file changes for a version release.

Key attributes:

• change_type - added, modified, removed
• category - Auto-detected: controller, model, view, security, etc.
• diff_content - Unified diff for modified files

Webhook System

Architecture

                          ┌─────────────────┐
  GitHub/GitLab/npm ─────►│WebhookController│
                          └────────┬────────┘
                                   │
                          ┌────────▼────────┐
                          │    Validate     │
                          │   Signature     │
                          └────────┬────────┘
                                   │
                          ┌────────▼────────┐
                          │    Create       │
                          │   Delivery      │
                          └────────┬────────┘
                                   │
                          ┌────────▼────────┐
                          │   Dispatch      │
                          │     Job         │
                          └────────┬────────┘
                                   │
                          ┌────────▼────────┐
                          │   Process       │
                          │   Webhook       │
                          │     Job         │
                          └────────┬────────┘
                                   │
                    ┌──────────────┼──────────────┐
                    │              │              │
             ┌──────▼───┐   ┌──────▼───┐   ┌──────▼───┐
             │  Parse   │   │  Create  │   │  Notify  │
             │ Payload  │   │ Release  │   │  Users   │
             └──────────┘   └──────────┘   └──────────┘

Supported Providers

Provider	Signature Method	Event Types
GitHub	HMAC-SHA256 (X-Hub-Signature-256)	release.published, release.created
GitLab	Token (X-Gitlab-Token)	release.create, tag_push
npm	HMAC-SHA256	package:publish
Packagist	HMAC-SHA1	package.update
Custom	HMAC-SHA256	Flexible

Secret Rotation

Webhooks support secret rotation with a configurable grace period (default 24 hours) where both old and new secrets are accepted.

Circuit Breaker

After 10 consecutive failures, a webhook endpoint is automatically disabled to prevent continued processing failures.

Storage Architecture

Dual Storage Mode

The package supports both local and S3 storage for vendor archives.

┌─────────────────────────────────────────────────────────┐
│                    Storage Flow                          │
├─────────────────────────────────────────────────────────┤
│                                                          │
│   Upload ─► Local Storage ─► Create Archive ─► S3       │
│                                    │                     │
│                                    ▼                     │
│                            Delete Local                  │
│                          (optional, based                │
│                           on retention)                  │
│                                                          │
│   Analysis ─► Check Local ─► Not Found ─► Download S3   │
│                   │                            │         │
│                   ▼                            ▼         │
│              Use Local                    Extract        │
│                                              │           │
│                                              ▼           │
│                                         Use Local        │
│                                                          │
└─────────────────────────────────────────────────────────┘

Retention Policy

• Keep N most recent versions locally (configurable, default 2)
• Never delete current or previous version locally
• Archive older versions to S3 automatically
• Verify file integrity on S3 download via SHA-256 hash

Path Structure

Local:

storage/app/vendors/{vendor-slug}/{version}/

S3:

{prefix}{vendor-slug}/{version}.tar.gz

AI Analysis Pipeline

Flow

VersionRelease
     │
     ▼
┌────────────┐
│   Group    │ ─► Related files grouped together
│   Diffs    │    (controller + view + route)
└─────┬──────┘
      │
      ▼
┌────────────┐
│   Build    │ ─► Construct context with file changes
│  Context   │
└─────┬──────┘
      │
      ▼
┌────────────┐
│   Call     │ ─► Rate-limited AI API call
│    AI      │    (10/minute default)
└─────┬──────┘
      │
      ▼
┌────────────┐
│   Parse    │ ─► Extract structured JSON
│ Response   │
└─────┬──────┘
      │
      ▼
┌────────────┐
│  Create    │ ─► Generate UpstreamTodo records
│   Todos    │
└────────────┘

AI Providers

Supports:

• Anthropic (default) - Claude models
• OpenAI - GPT models

Configuration via upstream.ai.provider and upstream.ai.model.

Rate Limiting

• AI API calls: 10/minute (configurable)
• Registry checks: 30/minute
• Issue creation: 10/minute
• Webhook ingestion: 60/minute per endpoint

Console Commands

Command	Purpose
upstream:check	Check vendors for updates, display status table
upstream:analyze	Analyse version diffs and generate todos
upstream:issues	Create GitHub/Gitea issues from pending todos
upstream:check-updates	Poll external registries for new versions
upstream:send-digests	Send scheduled digest emails

Admin UI Components

All components are Livewire-based and registered under the uptelligence.admin.* namespace:

Component	Purpose
Dashboard	Overview of vendors, todos, recent activity
VendorManager	CRUD for vendor configurations
TodoList	View and manage porting todos
DiffViewer	Browse file changes between versions
AssetManager	Track package dependencies
DigestPreferences	User notification settings
WebhookManager	Configure and monitor webhook endpoints

Configuration

See /Users/snider/Code/host-uk/core-uptelligence/config.php for all configuration options.

Key configuration sections:

• storage - Local and S3 storage settings
• source_types - Vendor type definitions
• detection_patterns - File categorisation patterns
• ai - AI provider and model settings
• github / gitea - Issue tracker integration
• update_checker - Auto-checking behaviour
• notifications - Slack, Discord, email settings
• default_vendors - Pre-configured vendor seeds

Dependencies

Required

• host-uk/core - Foundation framework

External Services

• Anthropic API or OpenAI API (for AI analysis)
• GitHub API (for releases and issues)
• Gitea API (for internal git server)
• Packagist API (for Composer package checks)
• npm Registry (for npm package checks)
• S3-compatible storage (for archives)