docs(benchmark): record the remaining gap to the speed target
CI / policy (push) Successful in 6s
CI / validate (push) Successful in 1m1s
CI / build (push) Successful in 1m27s
CI / policy (pull_request) Successful in 9s
CI / security (push) Successful in 1m16s
CI / validate (pull_request) Successful in 1m34s
CI / benchmark (push) Successful in 4m31s
CI / test (push) Successful in 4m36s
CI / build (pull_request) Successful in 39s
CI / test (pull_request) Successful in 3m24s
CI / security (pull_request) Successful in 1m11s
CI / images (push) Successful in 1m16s
CI / required (push) Successful in 1s
CI / benchmark (pull_request) Failing after 6m0s
CI / images (pull_request) Skipped
CI / required (pull_request) Failing after 2s
CI / policy (push) Successful in 6s
CI / validate (push) Successful in 1m1s
CI / build (push) Successful in 1m27s
CI / policy (pull_request) Successful in 9s
CI / security (push) Successful in 1m16s
CI / validate (pull_request) Successful in 1m34s
CI / benchmark (push) Successful in 4m31s
CI / test (push) Successful in 4m36s
CI / build (pull_request) Successful in 39s
CI / test (pull_request) Successful in 3m24s
CI / security (pull_request) Successful in 1m11s
CI / images (push) Successful in 1m16s
CI / required (push) Successful in 1s
CI / benchmark (pull_request) Failing after 6m0s
CI / images (pull_request) Skipped
CI / required (pull_request) Failing after 2s
The exact optimizations bring the loaded year from 66.0 s to 44.1 s, but the target of 10 s for the loaded year and 3 s for the other scenarios is not reached for the loaded year, typed obligations and three years. Record the gap and the remaining profile, keep the relative-gate multipliers and the 300-second hang guard unchanged, and note that ADR 0003's five-second target should be revisited together with a decision on search changes. Refs #286 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
acdb19352a
commit
63b50c4311
1 file changed
+63
@@ -135,3 +135,66 @@ above. The check now takes about 60 seconds with coverage.
|
||||
|
||||
All seven scenarios keep byte-identical outcomes, explanations and final states.
|
||||
The functional suite passes 684 tests across 35 files.
|
||||
|
||||
## #286: speed target not reached
|
||||
|
||||
The target is a loaded year under 10 seconds and every other scenario under 3
|
||||
seconds, using the existing protocol in Node on the development laptop. The
|
||||
exact optimizations of #282 to #284 do not reach it. Means from the latest
|
||||
protocol run, with the #285 calibration of 2.854 seconds:
|
||||
|
||||
| Scenario | Target | Latest | Gap factor |
|
||||
| --------------------------- | -----: | -------: | ---------: |
|
||||
| Loaded year | < 10 s | 44.094 s | 4.4x |
|
||||
| Typed obligations | < 3 s | 5.071 s | 1.7x |
|
||||
| Three years | < 3 s | 5.887 s | 2.0x |
|
||||
| Two years | < 3 s | 1.865 s | met |
|
||||
| Representative year | < 3 s | 0.402 s | met |
|
||||
| Chained earned entitlements | < 3 s | 0.025 s | met |
|
||||
| Deliberate conflict | < 3 s | 0.013 s | met |
|
||||
|
||||
From develop after PR #278 to now, the loaded year went from 66.0 to 44.1
|
||||
seconds (−33%), three years from 10.2 to 5.9 seconds and two years from 2.9 to
|
||||
1.9 seconds. Typed obligations is unchanged at about 5 seconds. The protocol went
|
||||
from 262.0 to 180.3 seconds.
|
||||
|
||||
Because the target is not met, the #285 multipliers stay at current speed and
|
||||
the 300-second hang-guard timeout stays as it is. #286 remains open for a
|
||||
separate decision on search changes.
|
||||
|
||||
### Remaining loaded-year profile
|
||||
|
||||
A fresh CPU profile of the bundled core at the #285 commit took 43.5 seconds,
|
||||
with outcomes still byte-identical:
|
||||
|
||||
| Inclusive cost | Seconds |
|
||||
| ----------------------------------------------------------------------- | ------: |
|
||||
| Candidate evaluation (`evaluateRuleCandidate`) | 21.4 |
|
||||
| — active objectives, including the pruning walk | 16.7 |
|
||||
| — of which objective evaluation (`evaluateOptimization`) | 7.7 |
|
||||
| — of which plan observations | 4.2 |
|
||||
| — candidate finalization | 2.7 |
|
||||
| Date assignment and release (`assignDateToBalance`, `releasePlacement`) | 8.1 |
|
||||
| Garbage collection | 5.1 |
|
||||
| Idle time between asynchronous slices | 4.6 |
|
||||
|
||||
The remaining cost is spread over about 1.2 million candidate evaluations
|
||||
rather than concentrated in one computation. Further exact work could:
|
||||
|
||||
- yield with something cheaper than a 1 ms timer between 8 ms slices, saving up
|
||||
to the 4.6 seconds of idle time;
|
||||
- validate optimization configurations once instead of on every evaluation,
|
||||
about 1 second;
|
||||
- copy fewer balances and earned activations when assigning a date.
|
||||
|
||||
Together these are worth perhaps 8 to 10 seconds, still well above 10 seconds
|
||||
for the loaded year. Reaching the target needs fewer candidate evaluations,
|
||||
which is a search change outside #279's exactness constraint.
|
||||
|
||||
### ADR 0003
|
||||
|
||||
ADR 0003 targets five seconds for the dependency-free planner. Under composable
|
||||
optimization rules, the loaded year takes about 44 seconds in Node on the
|
||||
development laptop even after these exact optimizations. That target should be
|
||||
revisited for the loaded year, together with the decision on search changes.
|
||||
This record does not revise the ADR.
|
||||
Reference in new issue
Block a user