Skip to main content

fidelity-kit compare

fidelity-kit compare​

Compare matching workloads using run-level bootstrap confidence intervals

Usage​

fidelity-kit compare --a <a> --b <b> [--out <out>] [--seed <seed>] [--iterations <iterations>] [--json] [--regression-threshold <regression-threshold>]
FlagTypeRequiredDescription
--astringYes
--bstringYesAliases: baseline
--outstringNoDefault: performance-results
--seednumberNoDefault: 1
--iterationsnumberNoDefault: 2000
--jsonbooleanNoDefault: false
--regression-thresholdnumberNoFail any workload whose A/B CI exceeds 1 + threshold (fraction)