perf(planning): plan from the best-ranked obligation solution and remove repeated work #302

New Issue

2026-10-18T01:52:04+02:00

nessar commented

2026-10-18 01:52:04 +02:00

Part of #78
Resolves the decision in #288
Unblocks #286

Problem Statement

As the planner's only developer, I want calculations to be fast. Under the current Bun benchmark protocol on the development laptop:

the loaded year takes about 66 seconds, against a 10-second target;
typed obligations takes about 5.4 seconds, against 3 seconds;
three years takes about 8.3 seconds, against 3 seconds.

These figures are the recorded ratios multiplied by the 3.79-second calibration. The exact work of #279 cannot close these gaps.

The #288 analysis measured where the work goes.

Obligation previews. Obligation rules can usually be satisfied jointly in several ways; call each way an obligation solution. To choose one, the planner runs the whole preference chain on each candidate solution as a trial, a preview. It keeps the solution whose preview scores best on the full objective vector, discards that preview, and runs the preference stages again on the winner.

When the enumeration of obligation solutions completes, every solution is previewed. There is no cap.
When the enumeration stops at its limit and every obligation is a date window, the solutions are ranked by their pre-preference score and the top eight are previewed.

In the loaded year, the eight previews and the real run amount to nine complete preference chains, which is 88% of about 1.07 million candidate evaluations. In typed obligations, 30 previews account for 96% of the evaluations. In both scenarios, every preview reaches the same objective values and the best-ranked solution wins.

Overhead outside scoring. Three years has no obligations, so previews do not affect it. Its time goes to repeated work outside candidate scoring:

The holiday-week stage takes about 3.6 seconds while scoring only about 32,000 candidates. For every public-holiday week, it re-ranks every placement as a potential donor, although that ranking depends only on the plan, not on the week.
Retaining the best valid plan for cancellation takes about 1.3 seconds: 816 retentions at about 1.6 ms each.
769 timer yields of about 1.1 ms each take about 0.85 seconds.

Solution

One deliberate change. Choose the obligation solution with the planner's existing pre-preference ranking and run the preference stages once on it. No previews are run.

Exact optimizations, done first. They leave every planning outcome, stable plan explanation and final-state snapshot unchanged:

reuse the donor ranking across public-holiday weeks;
retain the best valid plan more cheaply;
yield between asynchronous slices without a timer;
validate each optimization configuration once;
copy less state when assigning dates.

The deliberate change affects a small share of requests. The frozen engine is rebuilt in its own reviewed commit, and the before/after comparison is recorded.

Targets.

Scenario	Target
Loaded year	under 10 seconds
Three years	under 5 seconds
Every other scenario	under 3 seconds

All are measured with the current Bun protocol on the development laptop. The three-year target is relaxed from 3 to 5 seconds, which ADR 0006 allows: it sets no validated limit for multi-year horizons.

Evidence from the #288 analysis. The figures come from a scratch, instrumented copy of develop at 880efe4, Bun 1.4.3 and the development laptop, in one session.

"Asynchronous" means the core's asynchronous adapter run in the same thread. That is the benchmark's path without the worker, so the protocol adds some worker overhead on top.
The first 7.5-second loaded-year figure in the #288 discussion was synchronous. Synchronous timings leave out yields and incumbent retention.

Scenario	Change	Synchronous	Asynchronous	Result
Loaded year	without previews	49–53 → 6.7 s	→ 9.2 s	byte-identical plan
Typed obligations	without previews	4.4 → 0.2 s	→ 0.2 s	byte-identical plan
Three years	donor-ranking reuse	5.1 → 3.6 s	7.5 → 5.4 s	5,000 of 5,000 random requests identical
Other four scenarios	without previews	unchanged	unchanged	byte-identical

The remaining exact steps are estimated, not prototyped. For three years: up to 0.85 seconds from yields and up to 1.3 seconds from retention, which should bring it to about 3.5 to 4.6 seconds. For the loaded year: about 0.5 seconds from retention plus its share of yield time, which gives headroom under 10 seconds.

Browsers usually clamp nested timers to at least 4 ms. The yield change may therefore save several seconds on real devices: 769 yields at 4 ms is about 3 seconds in three years.

Effect of dropping previews over the 5,000 seeded requests of the frozen-engine check (seeds 280000–284999):

Result	Requests
Identical	4,970
Same objective values, different plan	6
Better	1
Worse	23
Changed outcome kind	0

Of the 23 worse requests, 14 differ only in the final tie-breaks: 8 on placement date and 6 on balance expiry. The other 9 are worse on a real objective:

Seed	Worse objective	Today → without previews
280112	longest work sequence	2 → 3
280204	composed optimization	6 → 3
280686	longest work sequence	2 → 3
283162	supporting placements	0 → 1
283484	longest work sequence	4 → 5
283930	`tw0` preference	2 → 1
284628	longest work sequence	8 → 16
284696	non-working-day gain	3 → 0
284824	short work-sequence deficit	0 → 2

Keeping two or three previews would cut those 9 to 3 or 1, but it would leave the loaded year above 25 seconds.

User stories

As a planner user whose obligation rules can be met in many ways, I want my plan in a few seconds instead of about a minute, so that I can iterate on balances and rules.
As a planner user with a multi-year horizon, I want my plan within about five seconds on a desktop, without any change to the plan I get.
As a planner user, I want my obligation rules still satisfied jointly and exactly, and a request that yields a plan today never to become a planning conflict.
As a planner user, I want Cancel to keep returning the best valid plan found so far, labelled as an incomplete calculation.
As the developer, I want each exact optimization proven byte-identical against the current frozen engine before the baseline moves, so that only one commit carries a deliberate behavior change.
As the developer, I want that deliberate change isolated in one reviewed commit that rebuilds the frozen engine. I want its before/after comparison recorded with reproducible seeds, so that the accepted regressions are auditable.
As the developer, I want the relative benchmark gate tightened to the new speed, so that a regression fails CI.

Acceptance criteria

Exact steps. Each step is its own commit and keeps outcomes, stable explanations and final-state snapshots byte-identical against the current frozen engine, including the cancelled asynchronous comparisons and their step counts:
- reuse the donor ranking of one plan across all public-holiday weeks;
- make incumbent retention cheaper without changing which plan is retained or what Cancel returns;
- a yield between asynchronous slices that browsers do not clamp, which still processes cancellation and heartbeat messages between slices;
- validate each optimization configuration once per calculation;
- copy fewer balances and earned activations when assigning a date.
Each step records same-session before/after measurements in the benchmark documentation.
Choosing the obligation solution.
- With more than one obligation solution, the planner orders them by the existing pre-preference ranking: active objectives, then tie-breaks, evaluated on the obligation-only state. Equal ranks keep enumeration order.
- The planner runs the preference stages once, on the first solution in that order. No previews run.
- The eight-preview limit and the uncapped preview of every solution are removed.
- With a single obligation solution, the planner behaves as it does today.
No new conflicts. A request that produces a plan today does not become a PlanningConflict when the chosen solution fails earned-entitlement chronology. See the proposed fallback below.
Benchmark.
- All seven scenarios keep their expected outcomes, placement counts, completed stages and determinism.
- With the current Bun protocol on the development laptop, the loaded year finishes under 10 seconds, three years under 5 seconds and every other scenario under 3 seconds.
- Before/after figures are recorded.
Frozen engine.
- The frozen engine is rebuilt in its own reviewed commit, from the commit that introduces the solution-selection change.
- The differential check passes against the rebuilt engine.
- The generator-coverage assertions still pass.
Comparison record. The benchmark documentation records the comparison over the 5,000 requests: the counts, the 9 objective-level regressions with their seeds and values, and the command to reproduce them.
Gate (#286).
- The relative gate's multipliers are tightened to the measured speed.
- The hang guard is lowered to about three times the expected run on the slowest runner. srvhome is about 1.9 times slower than nessar-desktop.
- The CI benchmark passes on at least two runners.
Decision record. An ADR records:
- that outcome-changing search changes are accepted under the frozen-engine rebuild discipline;
- that the obligation solution is chosen by pre-preference ranking;
- the benchmark targets: 10 seconds for the loaded year, 5 seconds for multi-year horizons, 3 seconds otherwise;
- that ADR 0003's five-second target is revisited.
Checks. The required correctness, coverage, static and build checks pass.

Implementation Decisions

Agreed in the #288 discussion:

Option B is adopted: plan from the best-ranked obligation solution, without previews, plus the exact steps above.
The three-year target becomes 5 seconds, and it is reached with exact steps only. Three years has no obligations, so its plans do not change.
Option E, previewing in parallel workers, is rejected. With previews removed, there is nothing left to parallelize. On its own, E keeps nine times the CPU work, which is the wrong trade on a phone. It also needs a worker pool, per-worker context rebuilds and multi-worker heartbeats and cancellation, and it changes the benchmark protocol's single worker.
Option C, polishing tie-breaks only in the final stage, is rejected. It only speeds things up by making previews cheaper, so it adds nothing after B. Across the 5,000 requests it also produced 53 worse and 17 better results, against B's 23 and 1.
Option D, adding several days per loss-prevention iteration, is not needed for the 5-second target. It would change outcomes and has not been measured.
Boundary.
- All work stays inside the planning core, including its asynchronous adapter.
- The core's public calculation, counterfactual and checkpoint-explanation entry points do not change. Neither do its outcome and snapshot types.
- Stage names and stage reporting are unchanged.
- The worker, the calculation service and the components are untouched.
Ranking. The ranking is the one already used to select the eight previews in truncated window searches. It now applies to every case with more than one obligation solution.
Donor-ranking reuse. It was prototyped as a per-plan cache of each placement's release objectives, shared by every public-holiday week.
- Each week still filters out its own dates and walks the placements in the same order with the same yields, so the ranking and the step sequence are unchanged.
- It was identical on all 5,000 random requests and on three years. It cut three years from 7.5 to 5.4 seconds asynchronously.
Commit order:
1. The exact steps, gated by the current frozen engine.
2. The selection change, with the frozen-engine rebuild in its own commit and the comparison record.
3. The gate multipliers and the lower hang guard.
4. The ADR and documentation, which may land with step 2.

Proposals to settle during implementation:

Earned-entitlement fallback.
- If the run on the chosen solution ends in an earned-entitlement-order conflict, the planner retries the next solution in ranked order. It reports a conflict only when every solution fails.
- This matches today's behavior, where a conflict needs every preview to fail. When earned obligations exist, the enumeration is always complete, so every solution is a candidate.
- No such case appeared in the 5,000 requests.
Cheaper retention.
- Today each retention finalizes and clones the candidate and derives its preference results, metrics and provenance immediately.
- Deferring that derivation until a checkpoint or cancellation needs it should save most of the 1.3 seconds. This is valid only if retained plans are never mutated afterwards; prove it with the cancelled asynchronous frozen-engine comparisons.
Yield. A message-channel task is the likely mechanism. Measure it in a browser as well, because nested timers are usually clamped to at least 4 ms there.
Calibration engine.
- Keep the current frozen bundle, at a871f8f, as the relative gate's fixed calibration engine. Its two-year ratios then stay comparable with the recorded history, and the rebuilt bundle serves only as the differential reference.
- The alternative is to recalibrate every multiplier in the same commit.
Evidence on large plans.
- The 5,000-request generator only produces small requests, of 7 to 70 days. For large plans, the only evidence is the loaded year and typed obligations, where every preview tied.
- Add a one-off sample of about 50 loaded-year-sized random requests to the recorded comparison. It takes about 40 minutes and is not a CI gate.
Comparison runner. Keep the before/after runner as an opt-in script, outside the functional suite and CI.

Unresolved:

Standing quality check. Should a "no worse than the previous engine on N random requests" report become a standing check, beyond this one-off record?
Duration estimate. Estimates will overstate requests with many obligation solutions and multi-year horizons. #78 allows recalibrating only when measured results require it, so decide after measuring.

Testing Decisions

What makes a good test here. It checks behavior at the core's public calculation entry point: the same request in, and the expected planning outcome, plan explanation and final-state snapshot out. Tests do not assert preview counts, cache hits or internal helper calls.
Frozen-engine differential check, unchanged bundle. It gates each exact step, including the cancelled asynchronous comparisons. Those compare step counts, checkpoints and retained plans, so they also show that the donor reuse, the cheaper retention and the new yield keep the step sequence and what Cancel returns.
Frozen-engine differential check, rebuilt bundle. It gates everything after the selection change.
Benchmark spec. The seven scenarios, their inputs and their assertions are unchanged. Per-scenario reporting and the relative gate carry the new speed.
New behavior tests at the public entry point:
- With two obligation solutions that rank differently before preferences, the plan is built from the better-ranked one.
- If the fallback is adopted: when the best-ranked solution fails earned-entitlement chronology and another succeeds, the outcome is a plan, not a conflict.
Cancellation and responsiveness. The existing cancellation, worker-lifetime and heartbeat specs cover cancellation during the obligations and preference stages, retention of the incumbent plan, and responsiveness with the new yield.
Validating configurations once. The existing planning-objectives and optimization-integration specs keep invalid configurations reported as invalid planning input.
Prior art:
- the frozen-engine spec and its async cancelled cases;
- the benchmark and benchmark-budget specs;
- the optimization-integration and leave-planner-cancellation specs;
- the fast-check property tests for plan observations;
- the planning-duration service spec.

Out of Scope

Previews run in parallel workers (option E), tie-break polishing only in the final stage (option C), adding several days per loss-prevention iteration (option D) and any solver dependency (rejected in ADR 0003).
Changes to rule priority semantics, optimization measures, tie-break order, or the shape of the stable plan explanation.
Changes to the seven benchmark scenarios, their inputs or their assertions.
Worker, calculation service and UI changes, including progress feedback (#300).
Browser and reference-phone measurement, beyond checking the yield in a browser.

Further Notes

Why previews existed. The code states the intent as "compare complete previews so obligation DFS does not commit to its first feasible branch." (DFS is the depth-first search that enumerates obligation solutions.) The pre-preference ranking still avoids taking the first branch found. What is lost is the case where a lower-ranked solution ends up better once the preferences run.
Removing an unbounded cost. Today, a completed enumeration previews every solution, so cost grows with the number of obligation solutions. B removes that.
#300, progress feedback. The obligations and holiday-week stages will take a much smaller share of the time, so coordinate its stage weights after this lands.
Evidence. The analysis prototype was an instrumented scratch copy and was not committed. Its figures come from one session; only same-session pairs are comparable.

Part of #78 Resolves the decision in #288 Unblocks #286 ## Problem Statement As the planner's only developer, I want calculations to be fast. Under the current Bun benchmark protocol on the development laptop: - the loaded year takes about 66 seconds, against a 10-second target; - typed obligations takes about 5.4 seconds, against 3 seconds; - three years takes about 8.3 seconds, against 3 seconds. These figures are the recorded ratios multiplied by the 3.79-second calibration. The exact work of #279 cannot close these gaps. The #288 analysis measured where the work goes. **Obligation previews.** Obligation rules can usually be satisfied jointly in several ways; call each way an **obligation solution**. To choose one, the planner runs the whole preference chain on each candidate solution as a trial, a **preview**. It keeps the solution whose preview scores best on the full objective vector, discards that preview, and runs the preference stages again on the winner. - When the enumeration of obligation solutions completes, every solution is previewed. There is no cap. - When the enumeration stops at its limit and every obligation is a date window, the solutions are ranked by their pre-preference score and the top eight are previewed. In the loaded year, the eight previews and the real run amount to nine complete preference chains, which is 88% of about 1.07 million candidate evaluations. In typed obligations, 30 previews account for 96% of the evaluations. In both scenarios, **every preview reaches the same objective values and the best-ranked solution wins**. **Overhead outside scoring.** Three years has no obligations, so previews do not affect it. Its time goes to repeated work outside candidate scoring: - The holiday-week stage takes about 3.6 seconds while scoring only about 32,000 candidates. For every public-holiday week, it re-ranks every placement as a potential donor, although that ranking depends only on the plan, not on the week. - Retaining the best valid plan for cancellation takes about 1.3 seconds: 816 retentions at about 1.6 ms each. - 769 timer yields of about 1.1 ms each take about 0.85 seconds. ## Solution **One deliberate change.** Choose the obligation solution with the planner's existing pre-preference ranking and run the preference stages once on it. No previews are run. **Exact optimizations, done first.** They leave every planning outcome, stable plan explanation and final-state snapshot unchanged: - reuse the donor ranking across public-holiday weeks; - retain the best valid plan more cheaply; - yield between asynchronous slices without a timer; - validate each optimization configuration once; - copy less state when assigning dates. The deliberate change affects a small share of requests. The frozen engine is rebuilt in its own reviewed commit, and the before/after comparison is recorded. **Targets.** | Scenario | Target | | --- | --- | | Loaded year | under 10 seconds | | Three years | under 5 seconds | | Every other scenario | under 3 seconds | All are measured with the current Bun protocol on the development laptop. The three-year target is relaxed from 3 to 5 seconds, which ADR 0006 allows: it sets no validated limit for multi-year horizons. **Evidence from the #288 analysis.** The figures come from a scratch, instrumented copy of develop at `880efe4`, Bun 1.4.3 and the development laptop, in one session. - "Asynchronous" means the core's asynchronous adapter run in the same thread. That is the benchmark's path without the worker, so the protocol adds some worker overhead on top. - The first 7.5-second loaded-year figure in the #288 discussion was synchronous. Synchronous timings leave out yields and incumbent retention. | Scenario | Change | Synchronous | Asynchronous | Result | | --- | --- | ---: | ---: | --- | | Loaded year | without previews | 49–53 → 6.7 s | → 9.2 s | byte-identical plan | | Typed obligations | without previews | 4.4 → 0.2 s | → 0.2 s | byte-identical plan | | Three years | donor-ranking reuse | 5.1 → 3.6 s | 7.5 → 5.4 s | 5,000 of 5,000 random requests identical | | Other four scenarios | without previews | unchanged | unchanged | byte-identical | The remaining exact steps are estimated, not prototyped. For three years: up to 0.85 seconds from yields and up to 1.3 seconds from retention, which should bring it to about 3.5 to 4.6 seconds. For the loaded year: about 0.5 seconds from retention plus its share of yield time, which gives headroom under 10 seconds. Browsers usually clamp nested timers to at least 4 ms. The yield change may therefore save several seconds on real devices: 769 yields at 4 ms is about 3 seconds in three years. **Effect of dropping previews** over the 5,000 seeded requests of the frozen-engine check (seeds 280000–284999): | Result | Requests | | --- | ---: | | Identical | 4,970 | | Same objective values, different plan | 6 | | Better | 1 | | Worse | 23 | | Changed outcome kind | 0 | Of the 23 worse requests, 14 differ only in the final tie-breaks: 8 on placement date and 6 on balance expiry. The other 9 are worse on a real objective: | Seed | Worse objective | Today → without previews | | --- | --- | --- | | 280112 | longest work sequence | 2 → 3 | | 280204 | composed optimization | 6 → 3 | | 280686 | longest work sequence | 2 → 3 | | 283162 | supporting placements | 0 → 1 | | 283484 | longest work sequence | 4 → 5 | | 283930 | `tw0` preference | 2 → 1 | | 284628 | longest work sequence | 8 → 16 | | 284696 | non-working-day gain | 3 → 0 | | 284824 | short work-sequence deficit | 0 → 2 | Keeping two or three previews would cut those 9 to 3 or 1, but it would leave the loaded year above 25 seconds. ## User stories 1. As a planner user whose obligation rules can be met in many ways, I want my plan in a few seconds instead of about a minute, so that I can iterate on balances and rules. 2. As a planner user with a multi-year horizon, I want my plan within about five seconds on a desktop, without any change to the plan I get. 3. As a planner user, I want my obligation rules still satisfied jointly and exactly, and a request that yields a plan today never to become a planning conflict. 4. As a planner user, I want Cancel to keep returning the best valid plan found so far, labelled as an incomplete calculation. 5. As the developer, I want each exact optimization proven byte-identical against the current frozen engine before the baseline moves, so that only one commit carries a deliberate behavior change. 6. As the developer, I want that deliberate change isolated in one reviewed commit that rebuilds the frozen engine. I want its before/after comparison recorded with reproducible seeds, so that the accepted regressions are auditable. 7. As the developer, I want the relative benchmark gate tightened to the new speed, so that a regression fails CI. ## Acceptance criteria - [ ] **Exact steps.** Each step is its own commit and keeps outcomes, stable explanations and final-state snapshots byte-identical against the current frozen engine, including the cancelled asynchronous comparisons and their step counts: - reuse the donor ranking of one plan across all public-holiday weeks; - make incumbent retention cheaper without changing which plan is retained or what Cancel returns; - a yield between asynchronous slices that browsers do not clamp, which still processes cancellation and heartbeat messages between slices; - validate each optimization configuration once per calculation; - copy fewer balances and earned activations when assigning a date. Each step records same-session before/after measurements in the benchmark documentation. - [ ] **Choosing the obligation solution.** - With more than one obligation solution, the planner orders them by the existing pre-preference ranking: active objectives, then tie-breaks, evaluated on the obligation-only state. Equal ranks keep enumeration order. - The planner runs the preference stages once, on the first solution in that order. No previews run. - The eight-preview limit and the uncapped preview of every solution are removed. - With a single obligation solution, the planner behaves as it does today. - [ ] **No new conflicts.** A request that produces a plan today does not become a `PlanningConflict` when the chosen solution fails earned-entitlement chronology. See the proposed fallback below. - [ ] **Benchmark.** - All seven scenarios keep their expected outcomes, placement counts, completed stages and determinism. - With the current Bun protocol on the development laptop, the loaded year finishes under 10 seconds, three years under 5 seconds and every other scenario under 3 seconds. - Before/after figures are recorded. - [ ] **Frozen engine.** - The frozen engine is rebuilt in its own reviewed commit, from the commit that introduces the solution-selection change. - The differential check passes against the rebuilt engine. - The generator-coverage assertions still pass. - [ ] **Comparison record.** The benchmark documentation records the comparison over the 5,000 requests: the counts, the 9 objective-level regressions with their seeds and values, and the command to reproduce them. - [ ] **Gate (#286).** - The relative gate's multipliers are tightened to the measured speed. - The hang guard is lowered to about three times the expected run on the slowest runner. srvhome is about 1.9 times slower than nessar-desktop. - The CI benchmark passes on at least two runners. - [ ] **Decision record.** An ADR records: - that outcome-changing search changes are accepted under the frozen-engine rebuild discipline; - that the obligation solution is chosen by pre-preference ranking; - the benchmark targets: 10 seconds for the loaded year, 5 seconds for multi-year horizons, 3 seconds otherwise; - that ADR 0003's five-second target is revisited. - [ ] **Checks.** The required correctness, coverage, static and build checks pass. ## Implementation Decisions **Agreed in the #288 discussion:** - **Option B is adopted:** plan from the best-ranked obligation solution, without previews, plus the exact steps above. - **The three-year target becomes 5 seconds,** and it is reached with exact steps only. Three years has no obligations, so its plans do not change. - **Option E, previewing in parallel workers, is rejected.** With previews removed, there is nothing left to parallelize. On its own, E keeps nine times the CPU work, which is the wrong trade on a phone. It also needs a worker pool, per-worker context rebuilds and multi-worker heartbeats and cancellation, and it changes the benchmark protocol's single worker. - **Option C, polishing tie-breaks only in the final stage, is rejected.** It only speeds things up by making previews cheaper, so it adds nothing after B. Across the 5,000 requests it also produced 53 worse and 17 better results, against B's 23 and 1. - **Option D, adding several days per loss-prevention iteration, is not needed** for the 5-second target. It would change outcomes and has not been measured. - **Boundary.** - All work stays inside the planning core, including its asynchronous adapter. - The core's public calculation, counterfactual and checkpoint-explanation entry points do not change. Neither do its outcome and snapshot types. - Stage names and stage reporting are unchanged. - The worker, the calculation service and the components are untouched. - **Ranking.** The ranking is the one already used to select the eight previews in truncated window searches. It now applies to every case with more than one obligation solution. - **Donor-ranking reuse.** It was prototyped as a per-plan cache of each placement's release objectives, shared by every public-holiday week. - Each week still filters out its own dates and walks the placements in the same order with the same yields, so the ranking and the step sequence are unchanged. - It was identical on all 5,000 random requests and on three years. It cut three years from 7.5 to 5.4 seconds asynchronously. - **Commit order:** 1. The exact steps, gated by the current frozen engine. 2. The selection change, with the frozen-engine rebuild in its own commit and the comparison record. 3. The gate multipliers and the lower hang guard. 4. The ADR and documentation, which may land with step 2. **Proposals to settle during implementation:** - **Earned-entitlement fallback.** - If the run on the chosen solution ends in an earned-entitlement-order conflict, the planner retries the next solution in ranked order. It reports a conflict only when every solution fails. - This matches today's behavior, where a conflict needs every preview to fail. When earned obligations exist, the enumeration is always complete, so every solution is a candidate. - No such case appeared in the 5,000 requests. - **Cheaper retention.** - Today each retention finalizes and clones the candidate and derives its preference results, metrics and provenance immediately. - Deferring that derivation until a checkpoint or cancellation needs it should save most of the 1.3 seconds. This is valid only if retained plans are never mutated afterwards; prove it with the cancelled asynchronous frozen-engine comparisons. - **Yield.** A message-channel task is the likely mechanism. Measure it in a browser as well, because nested timers are usually clamped to at least 4 ms there. - **Calibration engine.** - Keep the current frozen bundle, at `a871f8f`, as the relative gate's fixed calibration engine. Its two-year ratios then stay comparable with the recorded history, and the rebuilt bundle serves only as the differential reference. - The alternative is to recalibrate every multiplier in the same commit. - **Evidence on large plans.** - The 5,000-request generator only produces small requests, of 7 to 70 days. For large plans, the only evidence is the loaded year and typed obligations, where every preview tied. - Add a one-off sample of about 50 loaded-year-sized random requests to the recorded comparison. It takes about 40 minutes and is not a CI gate. - **Comparison runner.** Keep the before/after runner as an opt-in script, outside the functional suite and CI. **Unresolved:** - **Standing quality check.** Should a "no worse than the previous engine on N random requests" report become a standing check, beyond this one-off record? - **Duration estimate.** Estimates will overstate requests with many obligation solutions and multi-year horizons. #78 allows recalibrating only when measured results require it, so decide after measuring. ## Testing Decisions - **What makes a good test here.** It checks behavior at the core's public calculation entry point: the same request in, and the expected planning outcome, plan explanation and final-state snapshot out. Tests do not assert preview counts, cache hits or internal helper calls. - **Frozen-engine differential check, unchanged bundle.** It gates each exact step, including the cancelled asynchronous comparisons. Those compare step counts, checkpoints and retained plans, so they also show that the donor reuse, the cheaper retention and the new yield keep the step sequence and what Cancel returns. - **Frozen-engine differential check, rebuilt bundle.** It gates everything after the selection change. - **Benchmark spec.** The seven scenarios, their inputs and their assertions are unchanged. Per-scenario reporting and the relative gate carry the new speed. - **New behavior tests at the public entry point:** - With two obligation solutions that rank differently before preferences, the plan is built from the better-ranked one. - If the fallback is adopted: when the best-ranked solution fails earned-entitlement chronology and another succeeds, the outcome is a plan, not a conflict. - **Cancellation and responsiveness.** The existing cancellation, worker-lifetime and heartbeat specs cover cancellation during the obligations and preference stages, retention of the incumbent plan, and responsiveness with the new yield. - **Validating configurations once.** The existing planning-objectives and optimization-integration specs keep invalid configurations reported as invalid planning input. - **Prior art:** - the frozen-engine spec and its async cancelled cases; - the benchmark and benchmark-budget specs; - the optimization-integration and leave-planner-cancellation specs; - the fast-check property tests for plan observations; - the planning-duration service spec. ## Out of Scope - Previews run in parallel workers (option E), tie-break polishing only in the final stage (option C), adding several days per loss-prevention iteration (option D) and any solver dependency (rejected in ADR 0003). - Changes to rule priority semantics, optimization measures, tie-break order, or the shape of the stable plan explanation. - Changes to the seven benchmark scenarios, their inputs or their assertions. - Worker, calculation service and UI changes, including progress feedback (#300). - Browser and reference-phone measurement, beyond checking the yield in a browser. ## Further Notes - **Why previews existed.** The code states the intent as "compare complete previews so obligation DFS does not commit to its first feasible branch." (DFS is the depth-first search that enumerates obligation solutions.) The pre-preference ranking still avoids taking the first branch found. What is lost is the case where a lower-ranked solution ends up better once the preferences run. - **Removing an unbounded cost.** Today, a completed enumeration previews every solution, so cost grows with the number of obligation solutions. B removes that. - **#300, progress feedback.** The obligations and holiday-week stages will take a much smaller share of the time, so coordinate its stage weights after this lands. - **Evidence.** The analysis prototype was an instrumented scratch copy and was not committed. Its figures come from one session; only same-session pairs are comparable.

nessar added the ready-for-agent label 2026-10-18 01:52:04 +02:00

nessar referenced this issue

2026-10-18 01:52:10 +02:00

perf(planning): decide on search changes to reach the loaded-year speed target #288

nessar referenced this issue from a commit

2026-10-18 22:07:14 +02:00

test(benchmark): keep the a871f8f engine as a fixed calibration engine

nessar referenced this issue from a commit

2026-10-18 22:07:14 +02:00

perf(planning): reuse the donor ranking across public-holiday weeks

nessar referenced this issue from a commit

2026-10-18 22:07:14 +02:00