ci: make the deferred planning benchmark non-blocking

Run the unchanged benchmark as a separate advisory job while #78 is open.
Keep functional coverage and other checks required, and retain the 2 GB heap
and 300-second protocol limit. Record #78's obligation to restore the gate.

Refs: #75, #78
This commit is contained in:
nessar committed 2026-10-10 16:36:11 +02:00
1 parent 6811142299
commit 98e87fecee
5 files changed
+31 -14

No files matched your search

@@ -18,6 +18,7 @@ If the measured baseline already passes, keep the engine unchanged and record th
- [ ] Keep the existing seven benchmark scenarios and inputs, including the loaded year, typed joint obligations, chained earned entitlements and two- and three-year horizons. Keep explicit optimization configurations, quotas, expected outcomes, placement counts, completed-stage and determinism assertions.
- [ ] Run the real existing benchmark protocol: one warm-up and two measured repetitions per scenario, outside coverage instrumentation, with a 2 GB Node heap limit and the 300-second timeout for the full protocol. All scenarios and repetitions must finish and pass; a timeout, skipped scenario or partial report does not satisfy this ticket.
- [ ] Restore the benchmark as a required CI check after the unchanged protocol passes. While this ticket is open, the user approved a separate automatic benchmark job with `allow_failure: true` so its failure does not block functional delivery. Passing CI during this exception does not satisfy benchmark acceptance or justify closing this ticket.
- [ ] After #76 removes tracing and #77 verifies the retained behavior, benchmark outcome and explanation delivery without obsolete trace counters, trace transfer or trace-acknowledgement waits. Do not reintroduce tracing as profiling infrastructure.
- [ ] When the baseline fails, profile CPU, allocation and repeated planning work; apply optimizations and rerun the unchanged protocol until it passes. Preserve planning semantics, rule ordering, joint obligations, earned-entitlement chronology, final-plan quality and deterministic tie-breaking. Use behavior-level or differential tests to demonstrate that caching, deduplication or pruning is valid across the relevant configurations.
- [ ] Preserve responsive browser interaction, explicit cancellation, heartbeat monitoring and best-valid-partial-plan recovery during representative and long-horizon calculations. Keep the existing duration-estimation contract and recalibrate estimates only when measured results require it.
@@ -30,7 +31,7 @@ If the measured baseline already passes, keep the engine unchanged and record th
The previous real protocol timed out at 300 seconds without an out-of-memory failure. The loaded-year warm-up took about 184 seconds and its first measured repetition started about 252 seconds into the protocol. Original trace recording and repeated allocation previews were observed costs; the benefit of trace removal has not yet been measured. These are starting points for profiling, not permission to change optimization quality.
This ticket authorizes verification and implementation of performance improvements when needed. Functional migration remains in #75, trace removal in #76 and trace-removal functional verification in #77. Existing CI benchmark limits remain unchanged.
This ticket authorizes verification and implementation of performance improvements when needed. Functional migration remains in #75, trace removal in #76 and trace-removal functional verification in #77. Existing benchmark limits remain unchanged; only the benchmark job's blocking status is temporarily waived at the user's request until this ticket restores it.
## Blocked by