From 63b50c4311530f83faced121c7f49a0cae898f89 Mon Sep 17 00:00:00 2001 From: Nessar Date: Sat, 17 Oct 2026 00:40:56 +0200 Subject: [PATCH] docs(benchmark): record the remaining gap to the speed target The exact optimizations bring the loaded year from 66.0 s to 44.1 s, but the target of 10 s for the loaded year and 3 s for the other scenarios is not reached for the loaded year, typed obligations and three years. Record the gap and the remaining profile, keep the relative-gate multipliers and the 300-second hang guard unchanged, and note that ADR 0003's five-second target should be revisited together with a decision on search changes. Refs #286 Co-Authored-By: Claude Opus 5.5 --- docs/benchmark/issue-279.md | 63 +++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/benchmark/issue-279.md b/docs/benchmark/issue-279.md index 365977b..f9dba98 100644 --- a/docs/benchmark/issue-279.md +++ b/docs/benchmark/issue-279.md @@ -135,3 +135,66 @@ above. The check now takes about 60 seconds with coverage. All seven scenarios keep byte-identical outcomes, explanations and final states. The functional suite passes 684 tests across 35 files. + +## #286: speed target not reached + +The target is a loaded year under 10 seconds and every other scenario under 3 +seconds, using the existing protocol in Node on the development laptop. The +exact optimizations of #282 to #284 do not reach it. Means from the latest +protocol run, with the #285 calibration of 2.854 seconds: + +| Scenario | Target | Latest | Gap factor | +| --------------------------- | -----: | -------: | ---------: | +| Loaded year | < 10 s | 44.094 s | 4.4x | +| Typed obligations | < 3 s | 5.071 s | 1.7x | +| Three years | < 3 s | 5.887 s | 2.0x | +| Two years | < 3 s | 1.865 s | met | +| Representative year | < 3 s | 0.402 s | met | +| Chained earned entitlements | < 3 s | 0.025 s | met | +| Deliberate conflict | < 3 s | 0.013 s | met | + +From develop after PR #278 to now, the loaded year went from 66.0 to 44.1 +seconds (−33%), three years from 10.2 to 5.9 seconds and two years from 2.9 to +1.9 seconds. Typed obligations is unchanged at about 5 seconds. The protocol went +from 262.0 to 180.3 seconds. + +Because the target is not met, the #285 multipliers stay at current speed and +the 300-second hang-guard timeout stays as it is. #286 remains open for a +separate decision on search changes. + +### Remaining loaded-year profile + +A fresh CPU profile of the bundled core at the #285 commit took 43.5 seconds, +with outcomes still byte-identical: + +| Inclusive cost | Seconds | +| ----------------------------------------------------------------------- | ------: | +| Candidate evaluation (`evaluateRuleCandidate`) | 21.4 | +| — active objectives, including the pruning walk | 16.7 | +| — of which objective evaluation (`evaluateOptimization`) | 7.7 | +| — of which plan observations | 4.2 | +| — candidate finalization | 2.7 | +| Date assignment and release (`assignDateToBalance`, `releasePlacement`) | 8.1 | +| Garbage collection | 5.1 | +| Idle time between asynchronous slices | 4.6 | + +The remaining cost is spread over about 1.2 million candidate evaluations +rather than concentrated in one computation. Further exact work could: + +- yield with something cheaper than a 1 ms timer between 8 ms slices, saving up + to the 4.6 seconds of idle time; +- validate optimization configurations once instead of on every evaluation, + about 1 second; +- copy fewer balances and earned activations when assigning a date. + +Together these are worth perhaps 8 to 10 seconds, still well above 10 seconds +for the loaded year. Reaching the target needs fewer candidate evaluations, +which is a search change outside #279's exactness constraint. + +### ADR 0003 + +ADR 0003 targets five seconds for the dependency-free planner. Under composable +optimization rules, the loaded year takes about 44 seconds in Node on the +development laptop even after these exact optimizations. That target should be +revisited for the loaded year, together with the decision on search changes. +This record does not revise the ADR.