--- title: Architecture description: Technical architecture of core-uptelligence updated: 2026-01-29 --- # Architecture The `core-uptelligence` package provides upstream vendor tracking and dependency intelligence for the Host UK platform. It monitors software vendors, analyses version differences, generates porting tasks, and manages notification digests. ## Overview ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ External Systems │ ├─────────────┬─────────────┬─────────────┬─────────────┬────────────────────┤ │ GitHub │ GitLab │ npm │ Packagist │ AI Providers │ │ Releases │ Releases │ Publish │ Updates │ (Anthropic/OpenAI) │ └──────┬──────┴──────┬──────┴──────┬──────┴──────┬──────┴─────────┬──────────┘ │ │ │ │ │ └─────────────┴─────────────┴─────────────┘ │ │ │ ┌──────▼──────┐ │ │ Webhook │ │ │ Controller │ │ └──────┬──────┘ │ │ │ ┌──────▼──────┐ ┌──────▼──────┐ │ Webhook │ │ AI │ │ Receiver │ │ Analyzer │ │ Service │ │ Service │ └──────┬──────┘ └──────┬──────┘ │ │ ┌───────────────────┼───────────────────────────────────────┤ │ │ │ ┌──────▼──────┐ ┌──────▼──────┐ ┌──────────────┐ ┌──────▼──────┐ │ Vendor │ │ Version │ │ Diff │ │ Upstream │ │ Model │◄────┤ Release │────►│ Analyzer │───►│ Todo │ └─────────────┘ └─────────────┘ │ Service │ └─────────────┘ └──────────────┘ │ │ ┌──────────────┐ ┌──────▼──────┐ │ Issue │◄───────────────────────┤ Issue │ │ (GitHub/ │ │ Generator │ │ Gitea) │ │ Service │ └──────────────┘ └─────────────┘ ``` ## Module Registration The package registers as a Laravel service provider via `Boot.php` and uses the event-driven module loading pattern from `core-php`. ### Event Listeners ```php public static array $listens = [ AdminPanelBooting::class => 'onAdminPanel', ApiRoutesRegistering::class => 'onApiRoutes', ConsoleBooting::class => 'onConsole', ]; ``` ### Registered Services All services are registered as singletons: | Service | Purpose | |---------|---------| | `VendorStorageService` | Local and S3 file storage for vendor archives | | `VendorUpdateCheckerService` | Polls registries (GitHub, Packagist, npm) for updates | | `DiffAnalyzerService` | Generates file diffs between versions | | `AIAnalyzerService` | Uses AI to analyse diffs and categorise changes | | `IssueGeneratorService` | Creates GitHub/Gitea issues from todos | | `UptelligenceDigestService` | Compiles and sends digest notifications | | `WebhookReceiverService` | Processes incoming vendor release webhooks | | `AssetTrackerService` | Tracks package dependencies (Composer, npm) | ## Data Model ### Core Entities ``` ┌─────────────┐ ┌─────────────────┐ ┌─────────────┐ │ Vendor │────►│ VersionRelease │────►│ DiffCache │ └─────────────┘ └─────────────────┘ └─────────────┘ │ │ │ │ ▼ ▼ ┌─────────────┐ ┌─────────────────┐ │UpstreamTodo │ │ AnalysisLog │ └─────────────┘ └─────────────────┘ ``` ### Vendor Represents an upstream software source (licensed, OSS, or plugin). **Key attributes:** - `slug` - Unique identifier - `source_type` - `licensed`, `oss`, or `plugin` - `git_repo_url` - Repository URL for OSS vendors - `path_mapping` - Maps upstream paths to target paths - `ignored_paths` - Patterns to skip during analysis - `priority_paths` - High-importance file patterns - `target_repo` - GitHub/Gitea repo for issue creation ### VersionRelease Tracks a specific version of a vendor's software. **Key attributes:** - `version` / `previous_version` - Version comparison - `storage_disk` - `local` or `s3` - `s3_key` - Archive location in cold storage - `file_hash` - SHA-256 for integrity verification - `summary` - AI-generated release summary ### UpstreamTodo A porting task generated from version analysis. **Key attributes:** - `type` - `feature`, `bugfix`, `security`, `ui`, `block`, `api`, `refactor`, `dependency` - `status` - `pending`, `in_progress`, `ported`, `skipped`, `wont_port` - `priority` - 1-10 scale - `effort` - `low`, `medium`, `high` - `ai_analysis` - Raw AI analysis data - `github_issue_number` - Linked issue ### DiffCache Stores individual file changes for a version release. **Key attributes:** - `change_type` - `added`, `modified`, `removed` - `category` - Auto-detected: `controller`, `model`, `view`, `security`, etc. - `diff_content` - Unified diff for modified files ## Webhook System ### Architecture ``` ┌─────────────────┐ GitHub/GitLab/npm ─────►│WebhookController│ └────────┬────────┘ │ ┌────────▼────────┐ │ Validate │ │ Signature │ └────────┬────────┘ │ ┌────────▼────────┐ │ Create │ │ Delivery │ └────────┬────────┘ │ ┌────────▼────────┐ │ Dispatch │ │ Job │ └────────┬────────┘ │ ┌────────▼────────┐ │ Process │ │ Webhook │ │ Job │ └────────┬────────┘ │ ┌──────────────┼──────────────┐ │ │ │ ┌──────▼───┐ ┌──────▼───┐ ┌──────▼───┐ │ Parse │ │ Create │ │ Notify │ │ Payload │ │ Release │ │ Users │ └──────────┘ └──────────┘ └──────────┘ ``` ### Supported Providers | Provider | Signature Method | Event Types | |----------|-----------------|-------------| | GitHub | HMAC-SHA256 (`X-Hub-Signature-256`) | `release.published`, `release.created` | | GitLab | Token (`X-Gitlab-Token`) | `release.create`, `tag_push` | | npm | HMAC-SHA256 | `package:publish` | | Packagist | HMAC-SHA1 | `package.update` | | Custom | HMAC-SHA256 | Flexible | ### Secret Rotation Webhooks support secret rotation with a configurable grace period (default 24 hours) where both old and new secrets are accepted. ### Circuit Breaker After 10 consecutive failures, a webhook endpoint is automatically disabled to prevent continued processing failures. ## Storage Architecture ### Dual Storage Mode The package supports both local and S3 storage for vendor archives. ``` ┌─────────────────────────────────────────────────────────┐ │ Storage Flow │ ├─────────────────────────────────────────────────────────┤ │ │ │ Upload ─► Local Storage ─► Create Archive ─► S3 │ │ │ │ │ ▼ │ │ Delete Local │ │ (optional, based │ │ on retention) │ │ │ │ Analysis ─► Check Local ─► Not Found ─► Download S3 │ │ │ │ │ │ ▼ ▼ │ │ Use Local Extract │ │ │ │ │ ▼ │ │ Use Local │ │ │ └─────────────────────────────────────────────────────────┘ ``` ### Retention Policy - Keep N most recent versions locally (configurable, default 2) - Never delete current or previous version locally - Archive older versions to S3 automatically - Verify file integrity on S3 download via SHA-256 hash ### Path Structure **Local:** ``` storage/app/vendors/{vendor-slug}/{version}/ ``` **S3:** ``` {prefix}{vendor-slug}/{version}.tar.gz ``` ## AI Analysis Pipeline ### Flow ``` VersionRelease │ ▼ ┌────────────┐ │ Group │ ─► Related files grouped together │ Diffs │ (controller + view + route) └─────┬──────┘ │ ▼ ┌────────────┐ │ Build │ ─► Construct context with file changes │ Context │ └─────┬──────┘ │ ▼ ┌────────────┐ │ Call │ ─► Rate-limited AI API call │ AI │ (10/minute default) └─────┬──────┘ │ ▼ ┌────────────┐ │ Parse │ ─► Extract structured JSON │ Response │ └─────┬──────┘ │ ▼ ┌────────────┐ │ Create │ ─► Generate UpstreamTodo records │ Todos │ └────────────┘ ``` ### AI Providers Supports: - **Anthropic** (default) - Claude models - **OpenAI** - GPT models Configuration via `upstream.ai.provider` and `upstream.ai.model`. ### Rate Limiting - AI API calls: 10/minute (configurable) - Registry checks: 30/minute - Issue creation: 10/minute - Webhook ingestion: 60/minute per endpoint ## Console Commands | Command | Purpose | |---------|---------| | `upstream:check` | Check vendors for updates, display status table | | `upstream:analyze` | Analyse version diffs and generate todos | | `upstream:issues` | Create GitHub/Gitea issues from pending todos | | `upstream:check-updates` | Poll external registries for new versions | | `upstream:send-digests` | Send scheduled digest emails | ## Admin UI Components All components are Livewire-based and registered under the `uptelligence.admin.*` namespace: | Component | Purpose | |-----------|---------| | `Dashboard` | Overview of vendors, todos, recent activity | | `VendorManager` | CRUD for vendor configurations | | `TodoList` | View and manage porting todos | | `DiffViewer` | Browse file changes between versions | | `AssetManager` | Track package dependencies | | `DigestPreferences` | User notification settings | | `WebhookManager` | Configure and monitor webhook endpoints | ## Configuration See `/Users/snider/Code/host-uk/core-uptelligence/config.php` for all configuration options. Key configuration sections: - `storage` - Local and S3 storage settings - `source_types` - Vendor type definitions - `detection_patterns` - File categorisation patterns - `ai` - AI provider and model settings - `github` / `gitea` - Issue tracker integration - `update_checker` - Auto-checking behaviour - `notifications` - Slack, Discord, email settings - `default_vendors` - Pre-configured vendor seeds ## Dependencies ### Required - `host-uk/core` - Foundation framework ### External Services - Anthropic API or OpenAI API (for AI analysis) - GitHub API (for releases and issues) - Gitea API (for internal git server) - Packagist API (for Composer package checks) - npm Registry (for npm package checks) - S3-compatible storage (for archives)