docs: add human-friendly documentation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Snider
parent
f1aae0055f
commit
e876b62045
3 changed files with 732 additions and 0 deletions
320
docs/architecture.md
Normal file
320
docs/architecture.md
Normal file
|
|
@ -0,0 +1,320 @@
|
|||
---
|
||||
title: Architecture
|
||||
description: Internal design of core/lint -- types, data flow, and extension points
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
This document explains how `core/lint` works internally. It covers the core library (`pkg/lint`), the PHP quality pipeline (`pkg/php`), and the QA command layer (`cmd/qa`).
|
||||
|
||||
## Overview
|
||||
|
||||
The system is organised into three layers:
|
||||
|
||||
```
|
||||
cmd/core-lint CLI entry point (lint check, lint catalog)
|
||||
cmd/qa QA workflow commands (watch, review, health, issues, PHP tools)
|
||||
|
|
||||
pkg/lint Core library: rules, catalog, matcher, scanner, reporting
|
||||
pkg/php PHP tool wrappers: format, analyse, audit, security, test
|
||||
pkg/detect Project type detection
|
||||
|
|
||||
catalog/*.yaml Embedded rule definitions
|
||||
```
|
||||
|
||||
The root `lint.go` file ties the catalog layer to the library:
|
||||
|
||||
```go
|
||||
//go:embed catalog/*.yaml
|
||||
var catalogFS embed.FS
|
||||
|
||||
func LoadEmbeddedCatalog() (*lintpkg.Catalog, error) {
|
||||
return lintpkg.LoadFS(catalogFS, "catalog")
|
||||
}
|
||||
```
|
||||
|
||||
This means all YAML rules are baked into the binary at compile time. There are no runtime file lookups.
|
||||
|
||||
## Core Types (pkg/lint)
|
||||
|
||||
### Rule
|
||||
|
||||
A `Rule` represents a single lint check loaded from YAML. Key fields:
|
||||
|
||||
```go
|
||||
type Rule struct {
|
||||
ID string `yaml:"id"`
|
||||
Title string `yaml:"title"`
|
||||
Severity string `yaml:"severity"` // info, low, medium, high, critical
|
||||
Languages []string `yaml:"languages"` // e.g. ["go"], ["go", "php"]
|
||||
Tags []string `yaml:"tags"` // e.g. ["security", "injection"]
|
||||
Pattern string `yaml:"pattern"` // Regex pattern to match
|
||||
ExcludePattern string `yaml:"exclude_pattern"` // Regex to suppress false positives
|
||||
Fix string `yaml:"fix"` // Human-readable remediation
|
||||
Detection string `yaml:"detection"` // "regex" (extensible to other types)
|
||||
AutoFixable bool `yaml:"auto_fixable"`
|
||||
ExampleBad string `yaml:"example_bad"`
|
||||
ExampleGood string `yaml:"example_good"`
|
||||
FoundIn []string `yaml:"found_in"` // Repos where pattern was observed
|
||||
FirstSeen string `yaml:"first_seen"`
|
||||
}
|
||||
```
|
||||
|
||||
Each rule validates itself via `Validate()`, which checks required fields and compiles regex patterns. Severity is constrained to five levels: `info`, `low`, `medium`, `high`, `critical`.
|
||||
|
||||
### Catalog
|
||||
|
||||
A `Catalog` is a flat collection of rules with query methods:
|
||||
|
||||
- `ForLanguage(lang)` -- returns rules targeting a specific language
|
||||
- `AtSeverity(threshold)` -- returns rules at or above a severity level
|
||||
- `ByID(id)` -- looks up a single rule
|
||||
|
||||
Loading is done via `LoadDir(dir)` for filesystem paths or `LoadFS(fsys, dir)` for embedded filesystems. Both read all `.yaml` files in the directory and parse them into `[]Rule`.
|
||||
|
||||
### Matcher
|
||||
|
||||
The `Matcher` is the regex execution engine. It pre-compiles all regex-detection rules into `compiledRule` structs:
|
||||
|
||||
```go
|
||||
type compiledRule struct {
|
||||
rule Rule
|
||||
pattern *regexp.Regexp
|
||||
exclude *regexp.Regexp
|
||||
}
|
||||
```
|
||||
|
||||
`NewMatcher(rules)` compiles patterns once. `Match(filename, content)` then scans line by line:
|
||||
|
||||
1. For each compiled rule, check if the filename itself matches the exclude pattern (e.g., skip `_test.go` files).
|
||||
2. For each line, test against the rule's pattern.
|
||||
3. If the line matches, check the exclude pattern to suppress false positives.
|
||||
4. Emit a `Finding` with file, line number, matched text, and remediation advice.
|
||||
|
||||
Non-regex detection types are silently skipped, allowing the catalog schema to support future detection mechanisms (AST, semantic) without breaking the matcher.
|
||||
|
||||
### Scanner
|
||||
|
||||
The `Scanner` orchestrates directory walking and language-aware matching:
|
||||
|
||||
1. Walk the directory tree, skipping excluded directories (`vendor`, `node_modules`, `.git`, `testdata`, `.core`).
|
||||
2. For each file, detect its language from the file extension using `DetectLanguage()`.
|
||||
3. Filter the rule set to only rules targeting that language.
|
||||
4. Build a language-scoped `Matcher` and run it against the file content.
|
||||
|
||||
Supported language extensions:
|
||||
|
||||
| Extension | Language |
|
||||
|-----------|----------|
|
||||
| `.go` | go |
|
||||
| `.php` | php |
|
||||
| `.ts`, `.tsx` | ts |
|
||||
| `.js`, `.jsx` | js |
|
||||
| `.cpp`, `.cc`, `.c`, `.h` | cpp |
|
||||
| `.py` | py |
|
||||
|
||||
### Finding
|
||||
|
||||
A `Finding` is the output of a match:
|
||||
|
||||
```go
|
||||
type Finding struct {
|
||||
RuleID string `json:"rule_id"`
|
||||
Title string `json:"title"`
|
||||
Severity string `json:"severity"`
|
||||
File string `json:"file"`
|
||||
Line int `json:"line"`
|
||||
Match string `json:"match"`
|
||||
Fix string `json:"fix"`
|
||||
Repo string `json:"repo,omitempty"`
|
||||
}
|
||||
```
|
||||
|
||||
### Report
|
||||
|
||||
The `report.go` file provides three output formats:
|
||||
|
||||
- `WriteText(w, findings)` -- human-readable: `file:line [severity] title (rule-id)`
|
||||
- `WriteJSON(w, findings)` -- pretty-printed JSON array
|
||||
- `WriteJSONL(w, findings)` -- newline-delimited JSON (one object per line)
|
||||
|
||||
`Summarise(findings)` aggregates counts by severity.
|
||||
|
||||
## Data Flow
|
||||
|
||||
A typical scan follows this path:
|
||||
|
||||
```
|
||||
YAML files ──> LoadFS() ──> Catalog{Rules}
|
||||
|
|
||||
ForLanguage() / AtSeverity()
|
||||
|
|
||||
[]Rule (filtered)
|
||||
|
|
||||
NewScanner(rules)
|
||||
|
|
||||
ScanDir(root) / ScanFile(path)
|
||||
|
|
||||
┌───────────────┼───────────────┐
|
||||
│ Walk tree │ Detect lang │
|
||||
│ Skip dirs │ Filter rules │
|
||||
│ │ NewMatcher() │
|
||||
│ │ Match() │
|
||||
└───────────────┴───────────────┘
|
||||
|
|
||||
[]Finding
|
||||
|
|
||||
WriteText() / WriteJSON() / WriteJSONL()
|
||||
```
|
||||
|
||||
## Cyclomatic Complexity Analysis (pkg/lint/complexity.go)
|
||||
|
||||
The module includes a native Go AST-based cyclomatic complexity analyser. It uses `go/parser` and `go/ast` -- no external tools required.
|
||||
|
||||
```go
|
||||
results, err := lint.AnalyseComplexity(lint.ComplexityConfig{
|
||||
Threshold: 15,
|
||||
Path: "./pkg/...",
|
||||
})
|
||||
```
|
||||
|
||||
Complexity is calculated by starting at 1 and incrementing for each branching construct:
|
||||
- `if`, `for`, `range`, `case` (non-default), `comm` (non-default)
|
||||
- `&&`, `||` binary expressions
|
||||
- `type switch`, `select`
|
||||
|
||||
There is also `AnalyseComplexitySource(src, filename, threshold)` for testing without file I/O.
|
||||
|
||||
## Coverage Tracking (pkg/lint/coverage.go)
|
||||
|
||||
The coverage subsystem supports:
|
||||
|
||||
- **Parsing** Go coverage output (`ParseCoverProfile` for `-coverprofile` format, `ParseCoverOutput` for `-cover` output)
|
||||
- **Snapshotting** via `CoverageSnapshot` (timestamp, per-package percentages, metadata)
|
||||
- **Persistence** via `CoverageStore` (JSON file-backed append-only store)
|
||||
- **Regression detection** via `CompareCoverage(previous, current)` which returns a `CoverageComparison` with regressions, improvements, new packages, and removed packages
|
||||
|
||||
## Vulnerability Checking (pkg/lint/vulncheck.go)
|
||||
|
||||
`VulnCheck` wraps `govulncheck -json` and parses its newline-delimited JSON output into structured `VulnFinding` objects. The parser handles three message types from govulncheck's wire format:
|
||||
|
||||
- `config` -- extracts the module path
|
||||
- `osv` -- stores vulnerability metadata (ID, aliases, summary, affected ranges)
|
||||
- `finding` -- maps OSV IDs to call traces and affected packages
|
||||
|
||||
## Toolkit (pkg/lint/tools.go)
|
||||
|
||||
The `Toolkit` struct wraps common developer commands into structured Go APIs. It executes subprocesses and parses their output:
|
||||
|
||||
| Method | Wraps | Returns |
|
||||
|--------|-------|---------|
|
||||
| `FindTODOs(dir)` | `git grep` | `[]TODO` |
|
||||
| `Lint(pkg)` | `go vet` | `[]ToolFinding` |
|
||||
| `Coverage(pkg)` | `go test -cover` | `[]CoverageReport` |
|
||||
| `RaceDetect(pkg)` | `go test -race` | `[]RaceCondition` |
|
||||
| `AuditDeps()` | `govulncheck` (text) | `[]Vulnerability` |
|
||||
| `ScanSecrets(dir)` | `gitleaks` | `[]SecretLeak` |
|
||||
| `GocycloComplexity(threshold)` | `gocyclo` | `[]ComplexFunc` |
|
||||
| `DepGraph(pkg)` | `go mod graph` | `*Graph` |
|
||||
| `GitLog(n)` | `git log` | `[]Commit` |
|
||||
| `DiffStat()` | `git diff --stat` | `DiffSummary` |
|
||||
| `UncommittedFiles()` | `git status` | `[]string` |
|
||||
| `Build(targets...)` | `go build` | `[]BuildResult` |
|
||||
| `TestCount(pkg)` | `go test -list` | `int` |
|
||||
| `CheckPerms(dir)` | `filepath.Walk` | `[]PermIssue` |
|
||||
| `ModTidy()` | `go mod tidy` | `error` |
|
||||
|
||||
All methods use the `Run(name, args...)` helper which captures stdout, stderr, and exit code.
|
||||
|
||||
## PHP Quality Pipeline (pkg/php)
|
||||
|
||||
The `pkg/php` package provides structured wrappers around PHP ecosystem tools. Each tool has:
|
||||
|
||||
1. **Detection** -- checks for config files and vendor binaries (e.g., `DetectAnalyser`, `DetectPsalm`, `DetectRector`)
|
||||
2. **Options struct** -- configures the tool run
|
||||
3. **Execution function** -- builds the command, runs it, and returns structured results
|
||||
|
||||
### Supported Tools
|
||||
|
||||
| Function | Tool | Purpose |
|
||||
|----------|------|---------|
|
||||
| `Format()` | Laravel Pint | Code style formatting |
|
||||
| `Analyse()` | PHPStan / Larastan | Static analysis |
|
||||
| `RunPsalm()` | Psalm | Type-level static analysis |
|
||||
| `RunAudit()` | Composer audit + npm audit | Dependency vulnerability scanning |
|
||||
| `RunSecurityChecks()` | Built-in checks | .env exposure, debug mode, filesystem security |
|
||||
| `RunRector()` | Rector | Automated code refactoring |
|
||||
| `RunInfection()` | Infection | Mutation testing |
|
||||
| `RunTests()` | Pest / PHPUnit | Test execution |
|
||||
|
||||
### QA Pipeline
|
||||
|
||||
The pipeline system (`pipeline.go` + `runner.go`) organises checks into three stages:
|
||||
|
||||
- **Quick** -- audit, fmt, stan (fast, run on every push)
|
||||
- **Standard** -- psalm (if available), test
|
||||
- **Full** -- rector, infection (slow, run in full QA)
|
||||
|
||||
The `QARunner` builds `process.RunSpec` objects with dependency ordering (e.g., `stan` runs after `fmt`, `test` runs after `stan`). This allows future parallelisation while respecting ordering constraints.
|
||||
|
||||
### Project Detection (pkg/detect)
|
||||
|
||||
The `detect` package identifies project types by checking for marker files:
|
||||
|
||||
- `go.mod` present => Go project
|
||||
- `composer.json` present => PHP project
|
||||
|
||||
`DetectAll(dir)` returns all detected types, enabling polyglot project support.
|
||||
|
||||
## QA Command Layer (cmd/qa)
|
||||
|
||||
The `cmd/qa` package provides workflow-level commands that integrate with GitHub via the `gh` CLI:
|
||||
|
||||
- **watch** -- polls GitHub Actions for a specific commit, shows real-time status, drills into failure details (failed job, step, error line from logs)
|
||||
- **review** -- fetches open PRs, analyses CI status, review decisions, and merge readiness, suggests next actions
|
||||
- **health** -- scans all repos in a `repos.yaml` registry, reports aggregate CI health with pass rates
|
||||
- **issues** -- fetches issues across repos, categorises them (needs response, ready, blocked, triage), prioritises by labels and activity
|
||||
- **docblock** -- parses Go source with `go/ast`, counts exported symbols with and without doc comments, enforces a coverage threshold
|
||||
|
||||
Commands register themselves via `cli.RegisterCommands` in an `init()` function, making them available when the package is imported.
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Adding New Rules
|
||||
|
||||
Create a new YAML file in `catalog/` following the schema:
|
||||
|
||||
```yaml
|
||||
- id: go-xxx-001
|
||||
title: "Description of the issue"
|
||||
severity: medium # info, low, medium, high, critical
|
||||
languages: [go]
|
||||
tags: [security]
|
||||
pattern: 'regex-pattern'
|
||||
exclude_pattern: 'false-positive-filter'
|
||||
fix: "How to fix the issue"
|
||||
detection: regex
|
||||
auto_fixable: false
|
||||
example_bad: 'problematic code'
|
||||
example_good: 'corrected code'
|
||||
```
|
||||
|
||||
The file will be embedded automatically on the next build.
|
||||
|
||||
### Adding New Detection Types
|
||||
|
||||
The `Detection` field on `Rule` currently supports `"regex"`. The `Matcher` skips non-regex rules, so adding a new detection type (e.g., `"ast"` for Go AST patterns) requires:
|
||||
|
||||
1. Adding the new type to the `Validate()` method
|
||||
2. Creating a new matcher implementation
|
||||
3. Integrating it into `Scanner.ScanDir()`
|
||||
|
||||
### Loading External Catalogs
|
||||
|
||||
Use `LoadDir(path)` to load rules from a directory on disk rather than the embedded catalog:
|
||||
|
||||
```go
|
||||
cat, err := lintpkg.LoadDir("/path/to/custom/rules")
|
||||
```
|
||||
|
||||
This allows organisations to maintain private rule sets alongside the built-in catalog.
|
||||
271
docs/development.md
Normal file
271
docs/development.md
Normal file
|
|
@ -0,0 +1,271 @@
|
|||
---
|
||||
title: Development Guide
|
||||
description: How to build, test, and contribute to core/lint
|
||||
---
|
||||
|
||||
# Development Guide
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Go 1.26 or later
|
||||
- `core` CLI (for build and QA commands)
|
||||
- `gh` CLI (only needed for the `qa watch`, `qa review`, `qa health`, and `qa issues` commands)
|
||||
|
||||
## Building
|
||||
|
||||
The project uses the `core` build system. Configuration lives in `.core/build.yaml`.
|
||||
|
||||
```bash
|
||||
# Build the binary (outputs to ./bin/core-lint)
|
||||
core build
|
||||
|
||||
# Build targets: linux/amd64, linux/arm64, darwin/arm64, windows/amd64
|
||||
# CGO is disabled; the binary is fully static.
|
||||
```
|
||||
|
||||
To build manually with `go build`:
|
||||
|
||||
```bash
|
||||
go build -trimpath -ldflags="-s -w" -o bin/core-lint ./cmd/core-lint
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
core go test
|
||||
|
||||
# Run a single test by name
|
||||
core go test --run TestRule_Validate_Good
|
||||
|
||||
# Generate coverage report
|
||||
core go cov
|
||||
core go cov --open # Opens HTML report in browser
|
||||
```
|
||||
|
||||
The test suite covers all packages:
|
||||
|
||||
| Package | Test count | Focus |
|
||||
|---------|-----------|-------|
|
||||
| `pkg/lint` | ~89 | Rule validation, catalog loading, matcher, scanner, report, complexity, coverage, vulncheck, toolkit |
|
||||
| `pkg/detect` | 6 | Project type detection |
|
||||
| `pkg/php` | ~125 | All PHP tool wrappers (format, analyse, audit, security, refactor, mutation, test, pipeline, runner) |
|
||||
|
||||
Tests follow the `_Good`, `_Bad`, `_Ugly` suffix convention:
|
||||
- `_Good` -- happy path
|
||||
- `_Bad` -- expected error conditions
|
||||
- `_Ugly` -- edge cases and panics
|
||||
|
||||
### Test Examples
|
||||
|
||||
Testing rules against source content:
|
||||
|
||||
```go
|
||||
func TestMatcher_Match_Good(t *testing.T) {
|
||||
rules := []Rule{
|
||||
{
|
||||
ID: "test-001",
|
||||
Title: "TODO found",
|
||||
Severity: "low",
|
||||
Pattern: `TODO`,
|
||||
Detection: "regex",
|
||||
},
|
||||
}
|
||||
m, err := NewMatcher(rules)
|
||||
require.NoError(t, err)
|
||||
|
||||
findings := m.Match("example.go", []byte("// TODO: fix this"))
|
||||
assert.Len(t, findings, 1)
|
||||
assert.Equal(t, "test-001", findings[0].RuleID)
|
||||
assert.Equal(t, 1, findings[0].Line)
|
||||
}
|
||||
```
|
||||
|
||||
Testing complexity analysis without file I/O:
|
||||
|
||||
```go
|
||||
func TestAnalyseComplexitySource_Good(t *testing.T) {
|
||||
src := `package example
|
||||
func simple() { if true {} }
|
||||
func complex() {
|
||||
if a {} else if b {} else if c {}
|
||||
for i := range items {
|
||||
switch {
|
||||
case x: if y {}
|
||||
case z:
|
||||
}
|
||||
}
|
||||
}`
|
||||
results, err := AnalyseComplexitySource(src, "test.go", 3)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, results)
|
||||
}
|
||||
```
|
||||
|
||||
## Quality Assurance
|
||||
|
||||
```bash
|
||||
# Full QA pipeline: format, vet, lint, test
|
||||
core go qa
|
||||
|
||||
# Extended QA: includes race detection, vulnerability scan, security checks
|
||||
core go qa full
|
||||
|
||||
# Individual checks
|
||||
core go fmt # Format code
|
||||
core go vet # Run go vet
|
||||
core go lint # Run linter
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
lint/
|
||||
├── .core/
|
||||
│ └── build.yaml # Build configuration
|
||||
├── bin/ # Build output (gitignored)
|
||||
├── catalog/
|
||||
│ ├── go-correctness.yaml # Correctness rules (7 rules)
|
||||
│ ├── go-modernise.yaml # Modernisation rules (5 rules)
|
||||
│ └── go-security.yaml # Security rules (6 rules)
|
||||
├── cmd/
|
||||
│ ├── core-lint/
|
||||
│ │ └── main.go # CLI binary entry point
|
||||
│ └── qa/
|
||||
│ ├── cmd_qa.go # QA command group registration
|
||||
│ ├── cmd_watch.go # GitHub Actions monitoring
|
||||
│ ├── cmd_review.go # PR review status
|
||||
│ ├── cmd_health.go # Aggregate CI health
|
||||
│ ├── cmd_issues.go # Issue triage
|
||||
│ ├── cmd_docblock.go # Docblock coverage
|
||||
│ └── cmd_php.go # PHP QA subcommands
|
||||
├── pkg/
|
||||
│ ├── detect/
|
||||
│ │ ├── detect.go # Project type detection
|
||||
│ │ └── detect_test.go
|
||||
│ ├── lint/
|
||||
│ │ ├── catalog.go # Catalog loading and querying
|
||||
│ │ ├── complexity.go # Cyclomatic complexity (native AST)
|
||||
│ │ ├── coverage.go # Coverage tracking and comparison
|
||||
│ │ ├── matcher.go # Regex matching engine
|
||||
│ │ ├── report.go # Output formatters (text, JSON, JSONL)
|
||||
│ │ ├── rule.go # Rule type and validation
|
||||
│ │ ├── scanner.go # Directory walking and file scanning
|
||||
│ │ ├── tools.go # Toolkit (subprocess wrappers)
|
||||
│ │ ├── vulncheck.go # govulncheck JSON parser
|
||||
│ │ ├── testdata/
|
||||
│ │ │ └── catalog/
|
||||
│ │ │ └── test-rules.yaml
|
||||
│ │ └── *_test.go
|
||||
│ └── php/
|
||||
│ ├── analyse.go # PHPStan/Larastan/Psalm wrappers
|
||||
│ ├── audit.go # Composer audit + npm audit
|
||||
│ ├── format.go # Laravel Pint wrapper
|
||||
│ ├── mutation.go # Infection wrapper
|
||||
│ ├── pipeline.go # QA stage definitions
|
||||
│ ├── refactor.go # Rector wrapper
|
||||
│ ├── runner.go # Process spec builder
|
||||
│ ├── security.go # Security checks (.env, filesystem)
|
||||
│ ├── test.go # Pest/PHPUnit wrapper
|
||||
│ └── *_test.go
|
||||
├── lint.go # Root package: embedded catalog loader
|
||||
├── go.mod
|
||||
├── go.sum
|
||||
├── CLAUDE.md
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Writing New Rules
|
||||
|
||||
### Rule Schema
|
||||
|
||||
Each YAML file in `catalog/` contains an array of rule objects. Required fields:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `id` | string | Unique identifier (convention: `{lang}-{category}-{number}`, e.g., `go-sec-001`) |
|
||||
| `title` | string | Short human-readable description |
|
||||
| `severity` | string | One of: `info`, `low`, `medium`, `high`, `critical` |
|
||||
| `languages` | []string | Target languages (e.g., `[go]`, `[go, php]`) |
|
||||
| `pattern` | string | Detection pattern (regex for `detection: regex`) |
|
||||
| `fix` | string | Remediation guidance |
|
||||
| `detection` | string | Detection type (currently only `regex`) |
|
||||
|
||||
Optional fields:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `tags` | []string | Categorisation tags (e.g., `[security, injection]`) |
|
||||
| `exclude_pattern` | string | Regex to suppress false positives |
|
||||
| `found_in` | []string | Repos where the pattern was originally observed |
|
||||
| `example_bad` | string | Code example that triggers the rule |
|
||||
| `example_good` | string | Corrected code example |
|
||||
| `first_seen` | string | Date the pattern was first catalogued |
|
||||
| `auto_fixable` | bool | Whether automated fixing is feasible |
|
||||
|
||||
### Naming Convention
|
||||
|
||||
Rule IDs follow the pattern `{lang}-{category}-{number}`:
|
||||
|
||||
- `go-sec-*` -- Security rules
|
||||
- `go-cor-*` -- Correctness rules
|
||||
- `go-mod-*` -- Modernisation rules
|
||||
|
||||
### Testing a New Rule
|
||||
|
||||
Create a test that verifies the pattern matches expected code and does not match exclusions:
|
||||
|
||||
```go
|
||||
func TestNewRule_Matches(t *testing.T) {
|
||||
rules := []Rule{
|
||||
{
|
||||
ID: "go-xxx-001",
|
||||
Title: "My new rule",
|
||||
Severity: "medium",
|
||||
Languages: []string{"go"},
|
||||
Pattern: `my-pattern`,
|
||||
ExcludePattern: `safe-variant`,
|
||||
Detection: "regex",
|
||||
},
|
||||
}
|
||||
|
||||
m, err := NewMatcher(rules)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Should match
|
||||
findings := m.Match("example.go", []byte("code with my-pattern here"))
|
||||
assert.Len(t, findings, 1)
|
||||
|
||||
// Should not match (exclusion)
|
||||
findings = m.Match("example.go", []byte("code with safe-variant here"))
|
||||
assert.Empty(t, findings)
|
||||
}
|
||||
```
|
||||
|
||||
## Adding PHP Tool Support
|
||||
|
||||
To add support for a new PHP tool:
|
||||
|
||||
1. Create a new file in `pkg/php/` (e.g., `newtool.go`).
|
||||
2. Add a detection function that checks for config files or vendor binaries.
|
||||
3. Add an options struct and an execution function.
|
||||
4. Add a command in `cmd/qa/cmd_php.go` that wires the tool to the CLI.
|
||||
5. Add the tool to the pipeline stages in `pipeline.go` if appropriate.
|
||||
6. Write tests in a corresponding `*_test.go` file.
|
||||
|
||||
Follow the existing pattern -- each tool module exports:
|
||||
- `Detect*()` -- returns whether the tool is available
|
||||
- `Run*()` or the tool function -- executes the tool with options
|
||||
- A `*Options` struct -- configures behaviour
|
||||
|
||||
## Coding Standards
|
||||
|
||||
- **UK English** throughout: `colour`, `organisation`, `centre`, `modernise`, `analyse`, `serialise`
|
||||
- **Strict typing**: All function parameters and return values must have explicit types
|
||||
- **Testing**: Use `testify` assertions (`assert`, `require`)
|
||||
- **Error wrapping**: Use `fmt.Errorf("context: %w", err)` for error chains
|
||||
- **Formatting**: Standard Go formatting via `gofmt` / `core go fmt`
|
||||
|
||||
## Licence
|
||||
|
||||
This project is licenced under the EUPL-1.2.
|
||||
141
docs/index.md
Normal file
141
docs/index.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
---
|
||||
title: core/lint
|
||||
description: Pattern catalog, regex-based code checker, and quality assurance toolkit for Go and PHP projects
|
||||
---
|
||||
|
||||
# core/lint
|
||||
|
||||
`forge.lthn.ai/core/lint` is a standalone pattern catalog and code quality toolkit. It ships a YAML-based rule catalog for detecting security issues, correctness bugs, and modernisation opportunities in Go source code. It also provides a full PHP quality assurance pipeline and a suite of developer tooling wrappers.
|
||||
|
||||
The library is designed to be embedded into other tools. The YAML rule files are compiled into the binary at build time via `go:embed`, so there are no runtime file dependencies.
|
||||
|
||||
## Module Path
|
||||
|
||||
```
|
||||
forge.lthn.ai/core/lint
|
||||
```
|
||||
|
||||
Requires Go 1.26+.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### As a Library
|
||||
|
||||
```go
|
||||
import (
|
||||
lint "forge.lthn.ai/core/lint"
|
||||
lintpkg "forge.lthn.ai/core/lint/pkg/lint"
|
||||
)
|
||||
|
||||
// Load the embedded rule catalog.
|
||||
cat, err := lint.LoadEmbeddedCatalog()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Filter rules for Go, severity medium and above.
|
||||
rules := cat.ForLanguage("go")
|
||||
filtered := (&lintpkg.Catalog{Rules: rules}).AtSeverity("medium")
|
||||
|
||||
// Create a scanner and scan a directory.
|
||||
scanner, err := lintpkg.NewScanner(filtered)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
findings, err := scanner.ScanDir("./src")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Output results.
|
||||
lintpkg.WriteText(os.Stdout, findings)
|
||||
```
|
||||
|
||||
### As a CLI
|
||||
|
||||
```bash
|
||||
# Build the binary
|
||||
core build # produces ./bin/core-lint
|
||||
|
||||
# Scan the current directory with all rules
|
||||
core-lint lint check
|
||||
|
||||
# Scan with filters
|
||||
core-lint lint check --lang go --severity high ./pkg/...
|
||||
|
||||
# Output as JSON
|
||||
core-lint lint check --format json .
|
||||
|
||||
# Browse the catalog
|
||||
core-lint lint catalog list
|
||||
core-lint lint catalog list --lang go
|
||||
core-lint lint catalog show go-sec-001
|
||||
```
|
||||
|
||||
### QA Commands
|
||||
|
||||
The `qa` command group provides workflow-level quality assurance:
|
||||
|
||||
```bash
|
||||
# Go-focused
|
||||
core qa watch # Monitor GitHub Actions after a push
|
||||
core qa review # PR review status with actionable next steps
|
||||
core qa health # Aggregate CI health across all repos
|
||||
core qa issues # Intelligent issue triage
|
||||
core qa docblock # Check Go docblock coverage
|
||||
|
||||
# PHP-focused
|
||||
core qa fmt # Format PHP code with Laravel Pint
|
||||
core qa stan # Run PHPStan/Larastan static analysis
|
||||
core qa psalm # Run Psalm static analysis
|
||||
core qa audit # Audit composer and npm dependencies
|
||||
core qa security # Security checks (.env, filesystem, deps)
|
||||
core qa rector # Automated code refactoring
|
||||
core qa infection # Mutation testing
|
||||
core qa test # Run Pest or PHPUnit tests
|
||||
```
|
||||
|
||||
## Package Layout
|
||||
|
||||
| Package | Path | Description |
|
||||
|---------|------|-------------|
|
||||
| `lint` (root) | `lint.go` | Embeds YAML catalogs and exposes `LoadEmbeddedCatalog()` |
|
||||
| `pkg/lint` | `pkg/lint/` | Core library: Rule, Catalog, Matcher, Scanner, Report, Complexity, Coverage, VulnCheck, Toolkit |
|
||||
| `pkg/detect` | `pkg/detect/` | Project type detection (Go, PHP) by filesystem markers |
|
||||
| `pkg/php` | `pkg/php/` | PHP quality tools: format, analyse, audit, security, refactor, mutation, test, pipeline, runner |
|
||||
| `cmd/core-lint` | `cmd/core-lint/` | CLI binary (`core-lint lint check`, `core-lint lint catalog`) |
|
||||
| `cmd/qa` | `cmd/qa/` | QA workflow commands (watch, review, health, issues, docblock, PHP tools) |
|
||||
| `catalog/` | `catalog/` | YAML rule definitions (embedded at compile time) |
|
||||
|
||||
## Rule Catalogs
|
||||
|
||||
Three built-in YAML catalogs ship with the module:
|
||||
|
||||
| File | Rules | Focus |
|
||||
|------|-------|-------|
|
||||
| `go-security.yaml` | 6 | SQL injection, path traversal, XSS, timing attacks, log injection, secret leaks |
|
||||
| `go-correctness.yaml` | 7 | Unsynchronised goroutines, silent error swallowing, panics in library code, file deletion |
|
||||
| `go-modernise.yaml` | 5 | Replace legacy patterns with modern stdlib (`slices.Clone`, `slices.Sort`, `maps.Keys`, `errgroup`) |
|
||||
|
||||
Total: **18 rules** across 3 severity tiers (info, medium, high, critical). All rules target Go. The catalog is extensible -- add more YAML files to `catalog/` and they will be embedded automatically.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Direct dependencies:
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `forge.lthn.ai/core/cli` | CLI framework (`cli.Main()`, command registration, TUI styles) |
|
||||
| `forge.lthn.ai/core/go-i18n` | Internationalisation for CLI strings |
|
||||
| `forge.lthn.ai/core/go-io` | Filesystem abstraction for registry loading |
|
||||
| `forge.lthn.ai/core/go-log` | Structured logging and error wrapping |
|
||||
| `forge.lthn.ai/core/go-scm` | Repository registry (`repos.yaml`) for multi-repo commands |
|
||||
| `github.com/stretchr/testify` | Test assertions |
|
||||
| `gopkg.in/yaml.v3` | YAML parsing for rule catalogs |
|
||||
|
||||
The `pkg/lint` sub-package has minimal dependencies (only `gopkg.in/yaml.v3` and standard library). The heavier CLI and SCM dependencies live in `cmd/`.
|
||||
|
||||
## Licence
|
||||
|
||||
EUPL-1.2
|
||||
Loading…
Add table
Reference in a new issue