nessar/vac-optimizer
1
0 Fork 0
Code Issues 3 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

perf(planning): plan from the best-ranked obligation solution

Browse Source 
With several obligation solutions, the planner ran the whole preference
chain on candidate solutions as previews: every solution after a
complete enumeration, or the eight best-ranked after a truncated window
search. It now orders all solutions by the existing pre-preference
ranking (active objectives, then tie-breaks, of each obligation-only
plan; equal ranks keep enumeration order) and runs the preference
stages once, on the first. If that run ends with earned leave available
only after its final merged time-off period, the stages restart on the
next solution, so a request that planned before never becomes a
conflict.

The loaded year drops from 43 to 5.5 seconds and typed obligations from
3.3 to 0.1 seconds, with byte-identical benchmark plans. Over the 5,000
seeded requests, 23 plans become worse (9 on a real objective) and 1
better; over 50 loaded-year-sized requests, 4 become worse on a real
objective. ADR 0010 records the decision; docs/benchmark/issue-302.md
records the comparison and the opt-in runner that reproduces it.

The frozen-engine check fails until the frozen engine is rebuilt from
this commit in the next one.

Part of #302.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
nessarandClaude Opus 5.5 committed 2026-10-18 21:18:06 +02:00
1 parent aa6a0c6f1b
commit 8168137cd5
13 files changed
+579 -278
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/benchmark/README.md
+6
View File
@@ -138,6 +138,12 @@ if an outcome kind, rule type, year-crossing horizon or public-holiday horizon
stops appearing. Requests stay small because candidate search grows quickly
with horizon length and balances; the slowest case takes about one second.
The generator is `src/app/services/frozen-engine/random-planning-requests.ts`.
`compare-engines.ts` beside it is an opt-in runner, outside the functional suite
and CI, that classifies how the current engine's plans differ from the
calibration engine's over the same seeds; [the #302 record](issue-302.md#comparison-record)
shows its use.
The fixture is `src/app/services/frozen-engine/leave-planner.core.frozen.mjs`,
a generated bundle with `date-holidays` left external, and its hand-written
`leave-planner.core.frozen.d.mts`. Rebuild it only for an intended behavior
docs/benchmark/issue-302.md
+109
View File
@@ -131,3 +131,112 @@ Across the five exact steps, the same harness measured three years at
7.45 seconds before the first step and 2.97 seconds after the last, two years at
2.27 and 0.92 seconds, typed obligations at 5.11 and 3.27 seconds, and the loaded
year at 58.26 and 44.05 seconds. These figures come from different sessions.
## Choosing the obligation solution
This is the one deliberate behavior change
([ADR 0010](../adr/0010-rank-obligation-solutions-before-preferences.md)).
When obligation rules can be satisfied jointly in several ways, the planner used
to run the whole preference chain on candidate solutions as previews: every
solution when their enumeration completed, or the eight best-ranked when a
window search stopped at its limit. It kept the solution whose preview scored
best and ran the preference stages again on it. The planner now orders the
solutions by the pre-preference ranking that chose those eight, the active
objectives then the tie-breaks of each obligation-only plan, with enumeration
order breaking equal ranks. It runs the preference stages once, on the first
solution. If that run ends with earned leave available only after its final
merged time-off period, the stages restart on the next solution, and a
conflict is reported only when every solution fails.
### Timing
Measured like the exact steps, against the last exact step, with two measured
calculations per engine. All seven scenarios keep byte-identical outcomes,
explanations and final states.
| Scenario | Before (s) | After (s) |
| --------------------------- | ---------: | --------: |
| representative year | 0.26 | 0.26 |
| typed obligations | 3.26 | 0.11 |
| chained earned entitlements | 0.00 | 0.01 |
| deliberate conflict | 0.00 | 0.00 |
| loaded year | 43.09 | 5.54 |
| two years | 0.94 | 0.97 |
| three years | 3.10 | 3.06 |
### Comparison record
The opt-in runner compares the current engine with the calibration engine
(`a871f8f`, which still previews) on the frozen-engine check's 5,000 seeded
requests. It runs outside the functional suite and CI:
```sh
bun src/app/services/frozen-engine/compare-engines.ts small 280000 5000
```
| Result | Requests |
| --------------------------------- | -------: |
| Identical | 4,970 |
| Same objective values, other plan | 6 |
| Better | 1 |
| Worse | 23 |
| Changed outcome kind | 0 |
Fourteen of the worse requests differ only in a final tie-break: eight on
placement dates and six on balance expiry. The other nine are worse on a real
objective:
| Seed | Worse objective | Previews → ranking |
| ------ | --------------------------- | ------------------ |
| 280112 | longest work sequence | 2 → 3 |
| 280204 | composed optimization | 6 → 3 |
| 280686 | longest work sequence | 2 → 3 |
| 283162 | supporting placements | 0 → 1 |
| 283484 | longest work sequence | 4 → 5 |
| 283930 | `tw0` preference | 2 → 1 |
| 284628 | longest work sequence | 8 → 16 |
| 284696 | non-working-day gain | 3 → 0 |
| 284824 | short work-sequence deficit | 0 → 2 |
Seed 282254 is better: its `tw1` preference goes from 1 to 2. The runner prints
every non-identical seed with its first differing objective; a first seed and
a count of 1, such as `small 284696 1`, reproduce a single request.
None of these 5,000 requests reaches the earned-entitlement fallback. A
separate search of seeds 200000 to 259999 with the same generator found 28
requests whose best-ranked solution earns its leave too late; all 28 plan from
a later solution, and 26 of them are identical to the calibration engine's
plan. Without the fallback, each would have become a conflict.
### Loaded-year-sized sample
The 5,000 requests last 7 to 70 days. A second generator,
`generateLoadedYearRequest`, builds one-year requests shaped like the `loaded
year` scenario: four to eight CP or RTT balances of 6 to 14 days, two to four
obligation windows of two or three days in distinct months, up to two
preference windows, and the six presets. Their window search always stops at its
solution limit, so the calibration engine previews eight solutions. Fifty of
them were compared, in five parallel runs of ten seeds:
```sh
bun src/app/services/frozen-engine/compare-engines.ts loaded-year 302000 50
```
| Result | Requests |
| ------------------------------- | -------: |
| Identical | 45 |
| Worse, placement-date tie-break | 1 |
| Worse on a real objective | 4 |
| Better or changed outcome kind | 0 |
| Seed | Worse objective | Previews → ranking |
| ------ | --------------------- | ------------------ |
| 302011 | longest work sequence | 14 → 19 |
| 302033 | public-holiday weeks | 14 → 13 |
| 302040 | `window-4` preference | 2 → 1 |
| 302049 | longest work sequence | 25 → 30 |
On these larger plans, ranking loses a real objective more often than on the
small requests: 4 in 50, against 9 in 5,000. The synchronous calibration engine
took 3,924 seconds for the 50 requests, about 78 seconds each, and the current
engine 289 seconds, about 5.8 seconds each.