fidelity-kit compare
fidelity-kit compare
Compare matching workloads using run-level bootstrap confidence intervals
Usage
fidelity-kit compare --a <a> --b <b> [--out <out>] [--seed <seed>] [--iterations <iterations>] [--json] [--regression-threshold <regression-threshold>]
| Flag | Type | Required | Description |
|---|---|---|---|
--a | string | Yes | |
--b | string | Yes | Aliases: baseline |
--out | string | No | Default: performance-results |
--seed | number | No | Default: 1 |
--iterations | number | No | Default: 2000 |
--json | boolean | No | Default: false |
--regression-threshold | number | No | Fail any workload whose A/B CI exceeds 1 + threshold (fraction) |