Benchmark tooling

torch_lattice_bench.run.main()[source]
Return type:

None

class torch_lattice_bench.harness.BenchmarkCase(name, group, params, setup, prepare, run, backward=None, metrics=None, units=(), modes=None)[source]

Bases: object

Parameters:
name: str
group: str
params: tuple[Params, ...]
setup: Callable[[Params], Any]
prepare: Callable[[Any], Any]
run: Callable[[Any], Any]
backward: Callable[[Any], Any] | None
metrics: MetricFactory | None
units: tuple[str, ...]
modes: tuple[Mode, ...] | None
supports(mode)[source]
Return type:

bool

Parameters:

mode (Mode)

class torch_lattice_bench.harness.BenchmarkResult(case, group, mode, device, params, warmup, repeats, median_ms, min_ms, p90_ms, p95_ms, samples_ms, workload, units, skipped=False, notes='')[source]

Bases: object

Parameters:
case: str
group: str
mode: Mode
device: str
params: dict[str, Any]
warmup: int
repeats: int
median_ms: float | None
min_ms: float | None
p90_ms: float | None
p95_ms: float | None
samples_ms: tuple[float, ...]
workload: WorkloadMetrics
units: dict[str, float]
skipped: bool
notes: str
to_json()[source]
Return type:

dict[str, Any]

type torch_lattice_bench.harness.Mode = Literal['cold_op', 'hot_op', 'backward']
exception torch_lattice_bench.harness.SkipCase[source]

Bases: RuntimeError

torch_lattice_bench.harness.run_case(case, params, *, mode, device, warmup, repeats)[source]
Return type:

BenchmarkResult | None

Parameters:
torch_lattice_bench.harness.run_cases(cases, *, modes, device, warmup, repeats, include=None, keep_going=True, on_start=None, on_result=None, on_skip=None, on_error=None)[source]
Return type:

list[BenchmarkResult]

Parameters:

  • cases (Iterable[BenchmarkCase])

  • modes (Sequence[Mode])

  • device (str)

  • warmup (int)

  • repeats (int)

  • include (str | None)

  • keep_going (bool)

  • on_start (ProgressStart | None)

  • on_result (ProgressResult | None)

  • on_skip (ProgressSkip | None)

  • on_error (ProgressError | None)