Running flows and suites

Invocation

-flow is the flow name (not a file path) as declared in YAML under -templates. -templates is required — the loader walks it recursively, picks up every kind: flow and kind: suite YAML, and keeps the catalog in memory for this process only. There is no templates database and no post-run sync.

Choosing a config

-c <file> (or -config <file>) loads the env. The shipped config/lab.yaml is the canonical single-gNB lab. Other shipped configs:

Workload knobs

Each UE runs the FSM independently. The engine spawns according to the rate, then waits for in-flight UEs to terminate. Total wall-clock time = max(-duration, -repetitions / -rate) + drain time.

A run with -repetitions 1 and no -duration/-rate is single-UE mode — the report includes a per-message step log. Multi-UE runs aggregate metrics but skip the step log (it would be unhelpful at scale).

Output

By default run-flow prints a human-readable table to stdout; logs go to stderr.

For machine-readable output, pass -output json or -output junit. With -output-file <path>, the structured report goes to that file and the human summary still prints to stdout. The JSON report carries the full execution result (flow name, duration, metrics, steps, event log, post-checks, final states, all_passed, and related fields).

Historical catalogs and report browsing live in the control plane when agents are attached — the CLI itself is stateless and has no report list command. See Agent mode.

Tracing

-trace adds three things:

1. TX/RX hex dump of every wire frame (printed inline with the table).
2. JSON trace of every FSM step (printed at the end).
3. Trace-level zerolog output to stderr.

Tracing is expensive — it serialises every frame. Strip it for load tests.

Subscribers

Two paths on the CLI:

• -s <file>: load subscribers from YAML into an in-memory set for this process. UEs take the next row in order; there is no local DB write.
• -gen-subscriber: synthesize a fresh subscriber per UE in memory. No file, no set contention. Use this for runs against lab AMFs / dev proxies that don't validate auth.

If neither flag is set, the run fails fast with no subscribers configured. When the in-memory set is exhausted (more UEs than rows), further UEs fail rather than blocking on a wait queue.

See subscribers for generation, provisioning into a core, and how agent-mode jobs receive sets from the control plane.

Exit codes

run-flow exits on the flow verdict — the same answer the report prints first. all_passed in the JSON still means “no check failed”; that is a narrower fact than the verdict. A run whose UEs never reached a terminal state can fail no check and still verdict-fail.

• 0 — verdict pass
• 1 — verdict fail, or the engine returned an error
• 2 — usage error (bad flags / output format)
• 3 — verdict unsupported: every completed UE landed on a terminal the flow declares in unsupported_states. The core answered coherently that it does not implement the scenario. Not a failure — a third outcome so a gate can treat missing features as skip rather than red.

For CI gating, rely on the exit code and capture -output json|junit for artifacts. See CI integration.

Reports

The CLI does not write a local SQLite catalog. Capture the run you just executed:

Troubleshooting

flow %q not found — the flow name didn't load from -templates. Confirm the YAML is under that directory, declares the expected name:, and parses cleanly (d3x-run lint -templates <dir> or d3x-run describe -flow <file>).

error: load templates: ... — a YAML in the templates dir failed to parse or validate. The error message names the file and line.

no subscribers configured — neither -s nor -gen-subscriber. Pass a subscribers file or enable generation.

Flow times out at wait_* — the AMF/peer didn't send the expected response in time. Check -trace to see what was sent and whether the response matched the FSM's event: clause. Common cause: PLMN mismatch.

-rate ignored — single-UE runs (-repetitions 1, no -duration) burst, ignoring rate. Multi-UE runs respect it.

type: cmd refused — cmd actions execute a local program named by the flow YAML and are off unless you pass -allow-cmd-actions. Leave the flag unset unless you trust every template in -templates.

Running suites

d3x-run run-suite executes a suite — an ordered list of flow steps — as one or more cycles. Each cycle produces a single suite result with one child result per step. The sections below cover suite invocation, the cycle model, suite reports, and exit-code behaviour.

Suite invocation

-suite is the name (not the path) as declared in suite YAML under -templates. The templates tree ships dozens of suites — compliance and security passes for gNB, AUSF, SMF, UDM, UPF, and NRF, plus demo suites. See the flow and suite catalog for the full list, and browse the source YAML in d3x-templates.

Cycle vs step

A cycle is one full traversal of the suite's steps: list. A step is one entry in that list, executing one flow with its own workload. The runner repeats the cycle -repetitions times (or until -duration elapses).

Within a cycle, steps are strictly serial — no parallelism between steps. Each step independently draws its own subscribers from the in-memory set (or synthesizes them), runs its flow, and finishes before the next step starts.

Workload at the CLI vs in the suite

The CLI flags drive the outer loop (number of cycles). Workload knobs inside a cycle live in the suite YAML on each step:

Per-step workload (repetitions, rate, duration, timeout) lives in the YAML — see writing flows and suites.

run-suite has no -rate flag. Steps run strictly serially, so there is no suite-level spawn rate; set rate per step in the YAML when a step needs to throttle its own UEs.

CLI overrides

-trace and -gen-subscriber OR with each step's setting — one of them being true enables the feature for that step. Useful when a long suite YAML has trace off but you want to debug one cycle.

Suite reports

Each cycle produces a suite-level result with one child result per step. Capture it with -output json / -output-file on the CLI, or inspect history in the control plane when the suite was dispatched to an agent.

Exit code

Exit 0 only when every cycle's AllPassed is true and no cycle was aborted. A single failing step (with stop_on_failure: true, the default) makes the cycle abort and produces a non-zero exit.

always_run cleanup steps execute even after an abort, but their pass/fail still counts toward the cycle's AllPassed.

Per-step success vs cycle abort

A step can fail without aborting the cycle if it sets stop_on_failure: false. In that case the cycle continues to the next step but cycle.AllPassed is still false. Use this for negative-test steps that are expected to fail intermittently and shouldn't gate the cleanup steps that follow.

Suite troubleshooting

Suite stops after one step — the step's AllPassed was false and stop_on_failure: true (default). Set stop_on_failure: false on the step if it's an expected partial failure.

always_run step didn't execute — check the trace; the cycle might have been cancelled before reaching the cleanup. always_run runs after an abort but not after a context cancellation (Ctrl+C, deadline).

Subscriber set exhausted mid-cycle — every step draws from the same process-local set (or synthesizes). Make sure the set size ≥ the largest step's repetitions, or use gen_subscriber: true per step / -gen-subscriber on the CLI.

Other subcommands

The same binary also covers discovery, fuzz, subscribers, and the user-plane receiver. Full flag tables are on the CLI reference (generated from d3x-run --help).

Where to go next

• CI integration — gate on exit codes and capture json/junit.
• Agent mode — attach a long-lived worker to the control plane.
• CLI reference — every flag, generated from d3x-run --help.