318 lines
13 KiB
Markdown
318 lines
13 KiB
Markdown
---
|
|
title: Architecture
|
|
description: Internal architecture of go-ansible -- key types, data flow, module dispatch, templating, and SSH transport.
|
|
---
|
|
|
|
# Architecture
|
|
|
|
This document explains how `go-ansible` works internally. The package is a single flat Go package (`package ansible`) with four distinct layers: **types**, **parser**, **executor**, and **SSH transport**.
|
|
|
|
## High-Level Data Flow
|
|
|
|
```
|
|
Playbook YAML ──► Parser ──► []Play ──► Executor ──► Module Handlers ──► SSH Client ──► Remote Host
|
|
│ │
|
|
Inventory YAML ──► Parser ──► Inventory Callbacks (OnTaskStart, OnTaskEnd, ...)
|
|
```
|
|
|
|
1. The **Parser** reads YAML files and produces typed Go structs (`Play`, `Task`, `Inventory`).
|
|
2. The **Executor** iterates over plays and tasks, resolving hosts from inventory, evaluating conditions, expanding templates, and dispatching each task to the appropriate module handler.
|
|
3. **Module handlers** translate Ansible module semantics (e.g. `apt`, `copy`, `systemd`) into shell commands.
|
|
4. The **SSH client** executes those commands on remote hosts and returns stdout, stderr, and exit codes.
|
|
|
|
## Key Types
|
|
|
|
All types live in `types.go`.
|
|
|
|
### Playbook and Play
|
|
|
|
A `Playbook` is simply a wrapper around a slice of `Play` values. Each `Play` targets a set of hosts and contains ordered task lists:
|
|
|
|
```go
|
|
type Play struct {
|
|
Name string // Human-readable play name
|
|
Hosts string // Host pattern ("all", "webservers", "web1")
|
|
Become bool // Enable privilege escalation
|
|
BecomeUser string // User to escalate to (default: root)
|
|
GatherFacts *bool // Whether to collect host facts (default: true)
|
|
Vars map[string]any // Play-scoped variables
|
|
PreTasks []Task // Run before roles
|
|
Tasks []Task // Main task list
|
|
PostTasks []Task // Run after tasks
|
|
Roles []RoleRef // Roles to apply
|
|
Handlers []Task // Triggered by notify
|
|
Tags []string // Play-level tags
|
|
Environment map[string]string // Environment variables
|
|
Serial any // Batch size (int or string)
|
|
}
|
|
```
|
|
|
|
### Task
|
|
|
|
A `Task` is the fundamental unit of work. The `Module` and `Args` fields are not part of the YAML schema directly -- they are extracted at parse time by custom `UnmarshalYAML` logic that scans for known module keys:
|
|
|
|
```go
|
|
type Task struct {
|
|
Name string // Human-readable task name
|
|
Module string // Derived: module name (e.g. "apt", "shell")
|
|
Args map[string]any // Derived: module arguments
|
|
Register string // Variable name to store result
|
|
When any // Condition (string or []string)
|
|
Loop any // Items to iterate ([]any or var reference)
|
|
LoopControl *LoopControl // Loop variable naming
|
|
Notify any // Handler(s) to trigger on change
|
|
IgnoreErrors bool // Continue on failure
|
|
Become *bool // Task-level privilege escalation
|
|
Block []Task // Block/rescue/always error handling
|
|
Rescue []Task
|
|
Always []Task
|
|
// ... and more (tags, retries, delegate_to, etc.)
|
|
}
|
|
```
|
|
|
|
Free-form module syntax (e.g. `shell: echo hello`) is stored as `Args["_raw_params"]`.
|
|
|
|
### TaskResult
|
|
|
|
Every module handler returns a `TaskResult`:
|
|
|
|
```go
|
|
type TaskResult struct {
|
|
Changed bool // Whether the task made a change
|
|
Failed bool // Whether the task failed
|
|
Skipped bool // Whether the task was skipped
|
|
Msg string // Human-readable message
|
|
Stdout string // Command stdout
|
|
Stderr string // Command stderr
|
|
RC int // Exit code
|
|
Results []TaskResult // Per-item results (for loops)
|
|
Data map[string]any // Module-specific structured data
|
|
Duration time.Duration // Execution time
|
|
}
|
|
```
|
|
|
|
Results can be captured via `register:` and referenced in later `when:` conditions or `{{ var.stdout }}` templates.
|
|
|
|
### Inventory
|
|
|
|
```go
|
|
type Inventory struct {
|
|
All *InventoryGroup
|
|
}
|
|
|
|
type InventoryGroup struct {
|
|
Hosts map[string]*Host
|
|
Children map[string]*InventoryGroup
|
|
Vars map[string]any
|
|
}
|
|
|
|
type Host struct {
|
|
AnsibleHost string
|
|
AnsiblePort int
|
|
AnsibleUser string
|
|
AnsiblePassword string
|
|
AnsibleSSHPrivateKeyFile string
|
|
AnsibleConnection string
|
|
AnsibleBecomePassword string
|
|
Vars map[string]any // Custom vars via inline YAML
|
|
}
|
|
```
|
|
|
|
Host resolution supports the `all` pattern, group names, and individual host names. The `GetHosts()` function performs pattern matching, and `GetHostVars()` collects variables by walking the group hierarchy (group vars are inherited, host vars take precedence).
|
|
|
|
### Facts
|
|
|
|
The `Facts` struct holds basic system information gathered from remote hosts via shell commands:
|
|
|
|
```go
|
|
type Facts struct {
|
|
Hostname string
|
|
FQDN string
|
|
Distribution string // e.g. "ubuntu"
|
|
Version string // e.g. "24.04"
|
|
Architecture string // e.g. "x86_64"
|
|
Kernel string // e.g. "6.8.0"
|
|
// ...
|
|
}
|
|
```
|
|
|
|
Facts are gathered automatically at the start of each play (unless `gather_facts: false`) and are available for templating via `{{ ansible_hostname }}`, `{{ ansible_distribution }}`, etc.
|
|
|
|
## Parser
|
|
|
|
The parser (`parser.go`) handles four types of YAML file:
|
|
|
|
| Method | Input | Output |
|
|
|--------|-------|--------|
|
|
| `ParsePlaybook(path)` | Playbook YAML | `[]Play` |
|
|
| `ParseInventory(path)` | Inventory YAML | `*Inventory` |
|
|
| `ParseTasks(path)` | Task list YAML | `[]Task` |
|
|
| `ParseRole(name, tasksFrom)` | Role directory | `[]Task` |
|
|
|
|
Iterator variants (`ParsePlaybookIter`, `ParseTasksIter`, `GetHostsIter`, `AllHostsIter`) return `iter.Seq` values for lazy, range-based consumption.
|
|
|
|
### Module Extraction
|
|
|
|
Ansible tasks embed the module name as a YAML key rather than a fixed field. The parser handles this via a custom `UnmarshalYAML` on `Task`:
|
|
|
|
1. Decode the YAML node into both a raw `map[string]any` and the typed struct.
|
|
2. Iterate over the map keys, skipping known structural keys (`name`, `register`, `when`, etc.).
|
|
3. Match remaining keys against `KnownModules` (a list of 68 entries covering both FQCN and short forms).
|
|
4. Any key containing a dot that is not in the known list is also accepted (to support collection modules).
|
|
5. Store the module name in `Task.Module` and its value in `Task.Args`.
|
|
|
|
The `with_items` legacy syntax is automatically normalised to `Loop`.
|
|
|
|
### Role Resolution
|
|
|
|
`ParseRole` searches multiple directory patterns for the role's `tasks/main.yml` (or a custom `tasks_from` file):
|
|
|
|
- `{basePath}/roles/{name}/tasks/`
|
|
- `{basePath}/../roles/{name}/tasks/`
|
|
- `{basePath}/playbooks/roles/{name}/tasks/`
|
|
|
|
It also loads `defaults/main.yml` and `vars/main.yml` from the role directory, merging them into the parser's variable context.
|
|
|
|
## Executor
|
|
|
|
The `Executor` (`executor.go`) is the orchestration engine. It holds all runtime state:
|
|
|
|
```go
|
|
type Executor struct {
|
|
parser *Parser
|
|
inventory *Inventory
|
|
vars map[string]any // Global variables
|
|
facts map[string]*Facts // Per-host facts
|
|
results map[string]map[string]*TaskResult // host -> register_name -> result
|
|
handlers map[string][]Task // Handler registry
|
|
notified map[string]bool // Which handlers have been triggered
|
|
clients map[string]*SSHClient // Cached SSH connections
|
|
|
|
// Callbacks
|
|
OnPlayStart func(play *Play)
|
|
OnTaskStart func(host string, task *Task)
|
|
OnTaskEnd func(host string, task *Task, result *TaskResult)
|
|
OnPlayEnd func(play *Play)
|
|
|
|
// Options
|
|
Limit string
|
|
Tags []string
|
|
SkipTags []string
|
|
CheckMode bool
|
|
Verbose int
|
|
}
|
|
```
|
|
|
|
### Play Execution Order
|
|
|
|
For each play, the executor follows this sequence:
|
|
|
|
1. Resolve target hosts from inventory (applying `Limit` if set).
|
|
2. Merge play-level `vars` into the global variable context.
|
|
3. **Gather facts** on each host (unless `gather_facts: false`).
|
|
4. Execute **pre_tasks** in order.
|
|
5. Execute **roles** in order.
|
|
6. Execute **tasks** in order.
|
|
7. Execute **post_tasks** in order.
|
|
8. Run any **notified handlers**.
|
|
|
|
### Condition Evaluation
|
|
|
|
The `evaluateWhen` method processes `when:` clauses. It supports:
|
|
|
|
- Boolean literals: `true`, `false`, `True`, `False`
|
|
- Negation: `not <condition>`
|
|
- Inline boolean expressions with `and`, `or`, and parentheses
|
|
- Registered variable checks: `result is defined`, `result is success`, `result is failed`, `result is changed`, `result is skipped`
|
|
- Variable truthiness: checks `vars` map for the condition as a key, evaluating booleans, non-empty strings, and non-zero integers
|
|
- Default filter handling: `var | default(value)` always evaluates to true (permissive)
|
|
- Multiple conditions (AND semantics): all must pass
|
|
|
|
### Templating
|
|
|
|
Jinja2-style `{{ expression }}` placeholders are resolved by `templateString`. The resolution order is:
|
|
|
|
1. Jinja2 filters (`| default(...)`, `| bool`, `| trim`)
|
|
2. `lookup()` expressions (`env`, `file`)
|
|
3. Registered variable dot-access (`result.stdout`, `result.rc`)
|
|
4. Global variables (`vars` map)
|
|
5. Task-local variables
|
|
6. Host variables from inventory
|
|
7. Host facts (`ansible_hostname`, `ansible_distribution`, etc.)
|
|
|
|
Unresolved expressions are returned verbatim (e.g. `{{ undefined_var }}` remains as-is).
|
|
|
|
Template file processing (`TemplateFile`) performs a basic Jinja2-to-Go-template conversion for the `template` module, with a fallback to simple `{{ }}` substitution.
|
|
|
|
### Loops
|
|
|
|
The executor supports `loop:` (and the legacy `with_items:`) with configurable loop variable names via `loop_control`. Loop results are aggregated into a single `TaskResult` with a `Results` slice.
|
|
|
|
### Block / Rescue / Always
|
|
|
|
Error handling blocks follow Ansible semantics:
|
|
|
|
1. Execute all tasks in `block`.
|
|
2. If any `block` task fails and `rescue` is defined, execute the rescue tasks.
|
|
3. Always execute `always` tasks regardless of success or failure.
|
|
|
|
### Handler Notification
|
|
|
|
When a task produces `Changed: true` and has a `notify` field, the named handler(s) are marked. After all tasks in the play complete, notified handlers are executed in the order they are defined.
|
|
|
|
## Module Dispatch
|
|
|
|
The `executeModule` method in `modules.go` normalises the module name (adding the `ansible.builtin.` prefix if absent) and dispatches to the appropriate handler via a `switch` statement.
|
|
|
|
Each module handler:
|
|
|
|
1. Extracts arguments using helper functions (`getStringArg`, `getBoolArg`).
|
|
2. Constructs one or more shell commands.
|
|
3. Executes them via the SSH client.
|
|
4. Parses the output into a `TaskResult`.
|
|
|
|
Some modules (e.g. `debug`, `set_fact`, `fail`, `assert`) are purely local and do not require SSH.
|
|
|
|
### Argument Helpers
|
|
|
|
```go
|
|
func getStringArg(args map[string]any, key, def string) string
|
|
func getBoolArg(args map[string]any, key string, def bool) bool
|
|
```
|
|
|
|
These handle type coercion (e.g. `"yes"`, `"true"`, `"1"` all evaluate to `true` for `getBoolArg`).
|
|
|
|
## SSH Transport
|
|
|
|
The `SSHClient` (`ssh.go`) manages connections to remote hosts.
|
|
|
|
### Authentication
|
|
|
|
Authentication methods are tried in order:
|
|
|
|
1. **Explicit key file** -- from `ansible_ssh_private_key_file` or the `KeyFile` config field.
|
|
2. **Default keys** -- `~/.ssh/id_ed25519`, then `~/.ssh/id_rsa`.
|
|
3. **Password** -- both `ssh.Password` and `ssh.KeyboardInteractive` are registered.
|
|
|
|
### Host Key Verification
|
|
|
|
The client uses `~/.ssh/known_hosts` for host key verification via `golang.org/x/crypto/ssh/knownhosts`. If the file does not exist, it is created automatically.
|
|
|
|
### Privilege Escalation (become)
|
|
|
|
When `become` is enabled (at play or task level), commands are wrapped with `sudo`:
|
|
|
|
- **With password**: `sudo -S -u {user} bash -c '{command}'` (password piped via stdin)
|
|
- **Without password**: `sudo -n -u {user} bash -c '{command}'` (passwordless sudo)
|
|
|
|
### File Transfer
|
|
|
|
File uploads use `cat >` piped via an SSH session's stdin rather than SCP. This approach is simpler and works well with `become`, as the `sudo` wrapper can be applied to the `cat` command. Downloads use `cat` with quoted paths.
|
|
|
|
### Connection Lifecycle
|
|
|
|
SSH connections are created lazily (on first use per host) and cached in the executor's `clients` map. The `Executor.Close()` method terminates all open connections.
|
|
|
|
## Concurrency
|
|
|
|
The executor uses a `sync.RWMutex` to protect shared state (variables, results, SSH client cache). Tasks within a play execute sequentially per host. The `clients` map is locked during connection creation to prevent duplicate connections to the same host.
|