perf(planning): restore the integrated benchmark gate
CI / policy (push) Successful in 7s
CI / validate (push) Successful in 1m23s
CI / test (push) Successful in 3m17s
CI / benchmark (push) Failing after 5m52s
CI / build (push) Successful in 1m31s
CI / security (push) Successful in 1m39s
CI / images (push) Skipped
CI / policy (pull_request) Successful in 7s
CI / validate (pull_request) Successful in 1m11s
CI / test (pull_request) Successful in 3m17s
CI / benchmark (pull_request) Failing after 5m57s
CI / build (pull_request) Successful in 1m30s
CI / security (pull_request) Successful in 1m40s
CI / images (pull_request) Skipped
CI / required (push) Failing after 2s
CI / required (pull_request) Failing after 1s

Reduce objective evaluation and calendar allocation work while preserving planner outcomes and deterministic explanations. Restore required benchmark gating and record the unchanged protocol, differential checks, and browser evidence for #78.
This commit is contained in:
nessar authored and Nessar committed 2026-10-16 14:01:32 +02:00
1 parent 0852cd4444
commit cbde23f572
9 files changed
+594 -118

No files matched your search

+28 -31
View File
@@ -2,44 +2,41 @@
## Composable optimization rules: current verification status
The #70–#75 implementation uses explicit active optimization configurations in
the benchmark fixtures. Before trace removal, the full functional coverage run passed 652 tests
across 34 files under the 2 GB Node heap limit, excluding the separate benchmark.
That run does not establish a passing performance protocol.
[Issue #78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78) passes the
unchanged Node acceptance protocol on the documented development laptop:
**298.6 seconds**, including one warm-up and two measured repetitions of all
seven scenarios, without coverage, with one worker and a 2 GiB Node heap.
The full protocol retains its 300-second timeout. All expected outcomes,
placement counts, final stages and determinism checks pass. The fixture inputs
are unchanged; the test now prints its existing report.
The post-removal functional integration checks, browser QA and unchanged-input
handoff for #78 are recorded in [the #77 verification report](../trace-free-planning-verification.md).
On 2026-10-15, the #76 worktree based on `ea42561` passed 630 functional tests
across 33 files with coverage under the 2 GB Node heap limit. Complete outcome
comparisons against that baseline matched for all seven benchmark scenarios.
The separate, unchanged benchmark did not complete within its 300-second limit
(external timeout exit 124). Performance acceptance and any further optimization
remain with #78.
The benchmark is required again in Gitea Actions, including the image and
aggregate gates. [The #78 report](issue-78.md) records the baseline timeout,
profiling, before/after diagnostics, full protocol output, correctness checks
and browser limitations. The local pass has only 1.4 seconds of external-timeout
headroom; CI-runner performance and live workflow validation remain unverified.
The existing migration protection stays in place under
[issue #268](https://gitea.nessar.fr/nessar/vac-optimizer/issues/268).
Before the Gitea migration, GitLab pipeline 320 passed all 652 functional tests
in 109.43 seconds, then the seven-scenario protocol timed out at its unchanged
300-second limit without an out-of-memory failure. In an earlier diagnostic run, the loaded-year warm-up took
184.36 seconds and its first measured repetition began about 251.78 seconds into
the protocol. Those diagnostic timings precede the final period-projection cache
and the original-decision-trace removal in #76, and are not a result for either.
Before these optimizations, the trace-free integrated baseline at `2f4af14`
timed out after 300 seconds without a complete report. The #76 verification
previously passed 630 functional tests across 33 files with coverage under the
same heap setting, while its unchanged benchmark also timed out. Before the
Gitea migration, GitLab pipeline 320 passed 652 functional tests, but its
separate benchmark timed out as well. An earlier loaded-year warm-up took
184.36 seconds; that historical sample predates the final period-projection
cache and trace removal and is not a measurement of the current engine.
At the user's request, #75 finishes on saved-rule migration and functional
verification. #78 owns benchmark verification and implementation of further
optimizations when needed, after trace removal in #76 and integrated functional verification in #77. All seven scenarios, one warm-up and two measured repetitions per
scenario, the 2 GB heap limit and the 300-second timeout remain required for #78.
At the user's request, the Gitea CI configuration defines the benchmark in a
separate job with `continue-on-error: true` while #78 is open. Functional coverage
tests and all other required checks remain blocking. #78 must restore the required benchmark gate
after the unchanged protocol passes. A green pipeline during this exception
does not establish a passing benchmark. No automatic elapsed-time planning
cutoff is introduced. Gitea Actions is enabled; live workflow validation is tracked in
[issue #268](https://gitea.nessar.fr/nessar/vac-optimizer/issues/268). The converted
workflow preserves the separate advisory benchmark job. See the
[CI migration status](../../README.md#continuous-integration).
#75 owns functional saved-rule migration, #76 removes tracing, #77 verifies
retained behavior and #78 owns performance acceptance. No automatic elapsed-time
engine cutoff is introduced. No reference-phone run is claimed.
The results below document earlier implementations and protocols. They are
historical measurements rather than the performance result for composable rules or trace removal. Their trace counts, timings and controls describe the former implementation only.
The measurements below describe earlier implementations and protocols. Their
trace counters and timings are historical; current evidence is in
[the #78 report](issue-78.md).
## Reference-phone protocol