test(benchmark): print a readable summary table
CI / policy (push) Successful in 7s
CI / validate (push) Successful in 1m5s
CI / build (push) Successful in 1m38s
CI / policy (pull_request) Successful in 10s
CI / security (push) Successful in 1m19s
CI / test (push) Successful in 3m29s
CI / validate (pull_request) Successful in 1m42s
CI / benchmark (push) Successful in 4m13s
CI / security (pull_request) Successful in 56s
CI / build (pull_request) Successful in 1m42s
CI / test (pull_request) Successful in 3m8s
CI / images (push) Successful in 1m15s
CI / required (push) Successful in 1s
CI / benchmark (pull_request) Successful in 3m37s
CI / images (pull_request) Successful in 1m11s
CI / required (pull_request) Successful in 1s

Print the calibration and a Markdown table with one aligned row per
scenario as soon as it finishes: both planning times, mean, budget, ratio,
multiplier, outcome, placements, determinism and budget status. Write
directly to stdout so Vitest adds no per-line headers. The complete TSV
report still follows the table.

Refs #281, #285

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
nessarandClaude Opus 5.5 committed 2026-10-17 02:09:54 +02:00
1 parent cfdb28c90a
commit 3bbe989eca
4 files changed
+127 -28

No files matched your search

+11 -5
View File
@@ -20,11 +20,17 @@ speed, so a fixed wall-clock limit was flaky.
Before the protocol, in the same process, the spec times the frozen engine
(`src/app/services/frozen-engine/leave-planner.core.frozen.mjs`, the engine at
a871f8f) on the `two years` scenario, best of three, and prints
`calibration\tfrozen two years best of 3\t<ms>`. Each scenario's budget is its
multiplier times that calibration; the mean `planningMs` of its two measured
repetitions must not exceed it. A scenario without a multiplier fails too.
`#286` tightens the multipliers once the speed target is met.
a871f8f) on the `two years` scenario, best of three, and prints the result.
Each scenario's budget is its multiplier times that calibration; the mean
`planningMs` of its two measured repetitions must not exceed it. A scenario
without a multiplier fails too. `#286` tightens the multipliers once the speed
target is met.
The job log shows a Markdown table with one row per scenario, printed as soon
as that scenario finishes. Each row gives both measured planning times, their
mean, the budget, the ratio to the calibration, the multiplier, the outcome,
placements, determinism and `ok`, `OVER` or `no budget`. The complete
tab-separated report follows the table.
| Scenario | Mean planning (s) | Ratio to calibration | Multiplier |
| --------------------------- | ----------------- | -------------------- | ---------- |