# Issue #279: incremental exact scoring

Step-by-step record for [issue #279](https://gitea.nessar.fr/nessar/vac-optimizer/issues/279).
Each step keeps planning outcomes, stable explanations and final states identical
to develop after PR #278 (`a871f8f`). The
[frozen-engine differential check](README.md#frozen-engine-differential-check)
guards every step.

All measurements use Node 24.18.0, Bun 1.3.14 and Vitest 5.0.3 on the development
laptop, with no other test, build or profiling workload running alongside the
protocol. Only same-session before/after pairs are comparable.

```sh
NODE_OPTIONS=--max-old-space-size=2048 timeout 300s bunx vitest run src/app/benchmark/leave-planner-benchmark.spec.ts --maxWorkers=1
```

The per-scenario figures below are the means of the two measured calculations.

## Baseline profile

A CPU profile of the bundled core over all seven scenarios, with
`calculationDate: '2024-01-01'`, put 50.5 of 67.0 loaded-year seconds in
candidate evaluation (`evaluateRuleCandidate`). Of those, 41.3 seconds went to
active-objective evaluation: 15.6 seconds in time-off-period observations,
11.5 seconds in building observations and 7.6 seconds in sorting placement dates
for the placement tie key and the observation cache key. The asynchronous
adapter's 1 ms timer yields cost about 7% on typed obligations.

## #282: consolidate plan observations

Time-off periods, work sequences, public-holiday weeks, balance observations,
the placement list and both tie keys now live in
`src/app/services/plan-observations.ts`. The code moved without behavior
changes. All seven scenarios produce byte-identical outcomes, explanations and
final states compared with `a871f8f`.

| Measurement         |    Before |     After |
| ------------------- | --------: | --------: |
| Vitest duration     | 262.000 s | 262.030 s |
| Loaded year         |  66.010 s |  66.497 s |
| Typed obligations   |   4.887 s |   4.964 s |
| Two years           |   2.914 s |   2.977 s |
| Three years         |  10.203 s |  10.049 s |
| Representative year |   0.557 s |   0.569 s |

The differences are within run-to-run noise. The functional suite passes
666 tests across 34 files, including the frozen-engine check.

## #283: incremental plan observations

Each placements map now carries an immutable occupancy: its sorted placement
dates, time-off period groups, public-holiday-week completeness and lazily
computed calendar observations. A candidate derived from its parent by one added
or removed date, or by a copy with the same dates, updates only the affected
groups and holiday weeks. Unchanged groups keep their cached periods and
observation objects. Maps built from snapshots, maps observed against another
calendar and dates that are not in-horizon working days fall back to the
from-scratch computation.

The from-scratch computation remains the oracle. A fast-check property test
applies random add, remove and retype sequences across year boundaries, horizon
edges, declared days, weekends and public holidays, and compares every step with
it. An opt-in shadow mode (`VAC_OBSERVATION_SHADOW=1` or
`setObservationShadowMode(true)`) compares both on every observation and throws
on any mismatch. It passed across the frozen-engine check and all seven
benchmark scenarios, whose outcomes, explanations and final states stay
byte-identical.

| Measurement         |    Before |     After |
| ------------------- | --------: | --------: |
| Vitest duration     | 262.030 s | 193.680 s |
| Loaded year         |  66.497 s |  48.489 s |
| Typed obligations   |   4.964 s |   4.947 s |
| Two years           |   2.977 s |   1.906 s |
| Three years         |  10.049 s |   6.152 s |
| Representative year |   0.569 s |   0.441 s |

A fresh loaded-year CPU profile of the bundled core took 46.8 seconds. Of the
1.68 million observed placement maps, one was built from scratch; the rest were
derived. Observations now take about 4 seconds. Candidate evaluation still
dominates: 24.3 seconds in `evaluateRuleCandidate`, including 15.2 seconds of
active objectives and 6.9 seconds of candidate finalization, plus 6.2 seconds
assigning dates and about 5 seconds each of garbage collection and timer idle
time.

The functional suite passes 669 tests across 35 files.

## #284: prune candidates that cannot beat the protected prefix

Before #284, `evaluateRuleCandidate` finalized every candidate before comparing
its objectives. In the loaded year, 650,369 of 1,233,535 evaluations (53%) then
ended at a worse protected prefix. For typed obligations it was 74,336 of
273,470, and for three years 38,048 of 193,621.

Finalizing never moves or retypes a placement. It edits only rule
provenance, supporting status, earned activations and earned balances. Before
finalizing, the evaluation therefore walks the candidate's protected prefix in
objective order, using only objectives whose pre-finalize value is already their
exact final value, so the bound is tight:

- time-window and if-then preferences read only placement dates and types,
  plus declared days;
- public-holiday-week, time-off-period and work-sequence optimizations read
  only the placement dates;
- leave-balance optimizations are exact only when no enabled if-then rule earns
  days and the candidate has no earned activation;
- leave-placement optimizations have no safe bound, because finalizing rewrites
  supporting status.

The walk prunes at the first differing objective when it is worse than the
baseline. It stops without pruning at a better value or at an objective without
a safe bound. A pruned candidate would have failed finalization or returned at
the same comparison, and incumbents are retained only after the full objective
vector is built. Pruning therefore never changes which candidate wins or which
incumbent is retained.

The frozen-engine check now also runs every tenth seed through both engines'
asynchronous adapter, cancelled after a seed-derived number of steps. It
compares outcomes, artifacts, every checkpoint, the explanation and outcome
callbacks, and the step count. Of those 500 cases, 422 end incomplete and 217
of those keep an incumbent. Pruning on any differing objective breaks both
variants. Wrongly treating leave-placement or earning leave-balance objectives
as exact went undetected, because the generated requests rarely put those
objectives in a protected prefix, so their exclusion rests on the argument
above. The check now takes about 60 seconds with coverage.

| Measurement         |    Before |     After |
| ------------------- | --------: | --------: |
| Vitest duration     | 193.680 s | 180.250 s |
| Loaded year         |  48.489 s |  44.287 s |
| Typed obligations   |   4.947 s |   5.023 s |
| Two years           |   1.906 s |   1.837 s |
| Three years         |   6.152 s |   5.935 s |
| Representative year |   0.441 s |   0.400 s |

All seven scenarios keep byte-identical outcomes, explanations and final states.
The functional suite passes 684 tests across 35 files.

## #286: speed target not reached

The target is a loaded year under 10 seconds and every other scenario under 3
seconds, using the existing protocol in Node on the development laptop. The
exact optimizations of #282 to #284 do not reach it. Means from the latest
protocol run, with the #285 calibration of 2.854 seconds:

| Scenario                    | Target |   Latest | Gap factor |
| --------------------------- | -----: | -------: | ---------: |
| Loaded year                 | < 10 s | 44.094 s |       4.4x |
| Typed obligations           |  < 3 s |  5.071 s |       1.7x |
| Three years                 |  < 3 s |  5.887 s |       2.0x |
| Two years                   |  < 3 s |  1.865 s |        met |
| Representative year         |  < 3 s |  0.402 s |        met |
| Chained earned entitlements |  < 3 s |  0.025 s |        met |
| Deliberate conflict         |  < 3 s |  0.013 s |        met |

From develop after PR #278 to now, the loaded year went from 66.0 to 44.1
seconds (−33%), three years from 10.2 to 5.9 seconds and two years from 2.9 to
1.9 seconds. Typed obligations is unchanged at about 5 seconds. The protocol went
from 262.0 to 180.3 seconds.

Because the target is not met, the #285 multipliers stay at current speed and
the hang-guard timeout is not lowered. #286 remains open for a
separate decision on search changes.

### Remaining loaded-year profile

A fresh CPU profile of the bundled core at the #285 commit took 43.5 seconds,
with outcomes still byte-identical:

| Inclusive cost                                                          | Seconds |
| ----------------------------------------------------------------------- | ------: |
| Candidate evaluation (`evaluateRuleCandidate`)                          |    21.4 |
| — active objectives, including the pruning walk                         |    16.7 |
| — of which objective evaluation (`evaluateOptimization`)                |     7.7 |
| — of which plan observations                                            |     4.2 |
| — candidate finalization                                                |     2.7 |
| Date assignment and release (`assignDateToBalance`, `releasePlacement`) |     8.1 |
| Garbage collection                                                      |     5.1 |
| Idle time between asynchronous slices                                   |     4.6 |

The remaining cost is spread over about 1.2 million candidate evaluations
rather than concentrated in one computation. Further exact work could:

- yield with something cheaper than a 1 ms timer between 8 ms slices, saving up
  to the 4.6 seconds of idle time;
- validate optimization configurations once instead of on every evaluation,
  about 1 second;
- copy fewer balances and earned activations when assigning a date.

Together these are worth perhaps 8 to 10 seconds, still well above 10 seconds
for the loaded year. Reaching the target needs fewer candidate evaluations,
which is a search change outside #279's exactness constraint.

### ADR 0003

ADR 0003 targets five seconds for the dependency-free planner. Under composable
optimization rules, the loaded year takes about 44 seconds in Node on the
development laptop even after these exact optimizations. That target should be
revisited for the loaded year, together with the decision on search changes.
This record does not revise the ADR.