ci(benchmark): size the hang guard for the slowest runner
CI / policy (push) Successful in 6s
CI / validate (push) Successful in 53s
CI / build (push) Successful in 1m18s
CI / policy (pull_request) Successful in 9s
CI / security (push) Successful in 1m12s
CI / validate (pull_request) Successful in 1m18s
CI / benchmark (push) Successful in 3m56s
CI / test (push) Successful in 4m6s
CI / build (pull_request) Successful in 36s
CI / security (pull_request) Successful in 1m5s
CI / test (pull_request) Successful in 3m2s
CI / images (push) Successful in 1m15s
CI / required (push) Successful in 1s
CI / benchmark (pull_request) Successful in 7m18s
CI / images (pull_request) Successful in 1m50s
CI / required (pull_request) Successful in 1s
CI / policy (push) Successful in 6s
CI / validate (push) Successful in 53s
CI / build (push) Successful in 1m18s
CI / policy (pull_request) Successful in 9s
CI / security (push) Successful in 1m12s
CI / validate (pull_request) Successful in 1m18s
CI / benchmark (push) Successful in 3m56s
CI / test (push) Successful in 4m6s
CI / build (pull_request) Successful in 36s
CI / security (pull_request) Successful in 1m5s
CI / test (pull_request) Successful in 3m2s
CI / images (push) Successful in 1m15s
CI / required (push) Successful in 1s
CI / benchmark (pull_request) Successful in 7m18s
CI / images (pull_request) Successful in 1m50s
CI / required (pull_request) Successful in 1s
The first PR run landed on srvhome, about 1.9 times slower than nessar-desktop. The fixed 300-second timeout killed its protocol during the loaded year, although every completed scenario was within its relative budget. Raise the CI timeout to 600 seconds and the test timeout to 570 seconds so they only guard against hangs, and record the first CI-runner ratios. Refs #285 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
63b50c4311
commit
cfdb28c90a
5 files changed
+27
-10
No files matched your search
@@ -6,7 +6,7 @@
|
||||
unchanged Node acceptance protocol on the documented development laptop:
|
||||
**298.6 seconds**, including one warm-up and two measured repetitions of all
|
||||
seven scenarios, without coverage, with one worker and a 2 GiB Node heap.
|
||||
The full protocol keeps a 300-second timeout only as a hang guard (see
|
||||
The full protocol keeps a 600-second timeout only as a hang guard (see
|
||||
[Relative CI gate](#relative-ci-gate)). All expected outcomes,
|
||||
placement counts, final stages and determinism checks pass. The fixture inputs
|
||||
are unchanged; the test now prints its existing report.
|
||||
@@ -39,11 +39,28 @@ repetitions must not exceed it. A scenario without a multiplier fails too.
|
||||
Ratios come from the development machine (calibration about 2.74 s); the
|
||||
multipliers add roughly 1.5x headroom, rounded up, with a floor of 0.05 for the
|
||||
two tiny scenarios. A failure lists, per scenario, the measured mean, the
|
||||
budget, the multiplier and the calibration in milliseconds. The 300-second
|
||||
`timeout` in CI is only a hang guard.
|
||||
budget, the multiplier and the calibration in milliseconds.
|
||||
|
||||
TODO: measure the ratios on at least one CI runner; no runner numbers are
|
||||
recorded yet.
|
||||
The first CI runs of PR #287 (commit `63b50c4`) give consistent ratios on two
|
||||
runners:
|
||||
|
||||
| Scenario | nessar-desktop | srvhome |
|
||||
| --------------------------- | -------------- | ------- |
|
||||
| calibration | 3.03 s | 5.71 s |
|
||||
| representative year | 0.17 | 0.14 |
|
||||
| typed obligations | 1.83 | 1.75 |
|
||||
| chained earned entitlements | 0.011 | 0.005 |
|
||||
| deliberate conflict | 0.006 | 0.003 |
|
||||
| loaded year | 16.6 | killed |
|
||||
| two years | 0.65 | killed |
|
||||
| three years | 1.99 | killed |
|
||||
|
||||
srvhome is about 1.9 times slower than nessar-desktop, so its whole protocol
|
||||
needs roughly 400 seconds. The original 300-second `timeout` killed it during
|
||||
the loaded year, while every completed scenario was within budget. The CI
|
||||
`timeout` is therefore 600 seconds and the test timeout 570 seconds. They are
|
||||
only hang guards and do not judge speed; #286 lowers them once the speed
|
||||
target is met.
|
||||
|
||||
The post-removal functional integration checks, browser QA and unchanged-input
|
||||
handoff for #78 are recorded in [the #77 verification report](../trace-free-planning-verification.md).
|
||||
|
||||
@@ -159,7 +159,7 @@ seconds (−33%), three years from 10.2 to 5.9 seconds and two years from 2.9 to
|
||||
from 262.0 to 180.3 seconds.
|
||||
|
||||
Because the target is not met, the #285 multipliers stay at current speed and
|
||||
the 300-second hang-guard timeout stays as it is. #286 remains open for a
|
||||
the hang-guard timeout is not lowered. #286 remains open for a
|
||||
separate decision on search changes.
|
||||
|
||||
### Remaining loaded-year profile
|
||||
|
||||
Reference in new issue
Block a user