Compare commits

..
1 Commits
Author SHA1 Message Date
nessar 859a5f1105 perf(planning): restore the integrated benchmark gate
CI / policy (push) Successful in 32s
CI / validate (push) Successful in 55s
CI / build (push) Successful in 37s
CI / policy (pull_request) Successful in 9s
CI / security (push) Successful in 55s
CI / validate (pull_request) Successful in 43s
CI / test (push) Successful in 2m26s
CI / build (pull_request) Successful in 35s
CI / security (pull_request) Successful in 45s
CI / test (pull_request) Successful in 2m10s
CI / benchmark (push) Failing after 5m52s
CI / images (push) Skipped
CI / required (push) Failing after 2s
CI / benchmark (pull_request) Failing after 15m10s
CI / images (pull_request) Skipped
CI / required (pull_request) Failing after 1s
Reduce objective evaluation and calendar allocation work while preserving planner outcomes and deterministic explanations. Restore required benchmark gating and record the unchanged protocol, differential checks, and browser evidence for #78.
2026-10-16 08:42:27 +02:00
9 changed files with 594 additions and 118 deletions

No files matched your search

+4 -6
View File
@@ -76,8 +76,6 @@ jobs:
name: benchmark
runs-on: ubuntu-latest
container: node:24.21.0
# shortcut: advisory while #78 is open; restore the gate after the protocol passes.
continue-on-error: true
env:
NODE_OPTIONS: --max-old-space-size=2048
steps:
@@ -148,7 +146,7 @@ jobs:
images:
name: images
needs: [policy, validate, test, build, security]
needs: [policy, validate, test, benchmark, build, security]
if: >-
success() &&
(github.event_name == 'push' ||
@@ -192,17 +190,17 @@ jobs:
required:
name: required
if: ${{ always() }}
needs: [policy, validate, test, build, security, images]
needs: [policy, validate, test, benchmark, build, security, images]
runs-on: ubuntu-latest
container: node:24.21.0
env:
CHECK_RESULTS: ${{ needs.policy.result }} ${{ needs.validate.result }} ${{ needs.test.result }} ${{ needs.build.result }} ${{ needs.security.result }}
CHECK_RESULTS: ${{ needs.policy.result }} ${{ needs.validate.result }} ${{ needs.test.result }} ${{ needs.benchmark.result }} ${{ needs.build.result }} ${{ needs.security.result }}
IMAGE_RESULT: ${{ needs.images.result }}
IMAGE_EXPECTED: ${{ github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) }}
steps:
- name: Require successful validation and expected publication
run: |
test "$CHECK_RESULTS" = 'success success success success success'
test "$CHECK_RESULTS" = 'success success success success success success' || exit 1
if [ "$IMAGE_EXPECTED" = 'true' ]; then
test "$IMAGE_RESULT" = 'success'
else
+2 -2
View File
@@ -224,7 +224,7 @@ Open <http://localhost:4200>. The development server reloads when source files c
Build or start the application, then open `/benchmark`. The isolated harness covers a representative year, typed obligations, chained earned entitlements, a deliberate conflict, a loaded year, and two- and three-year horizons with dated annual balances. It reports preparation time, planning latency, planning-work time, total wall time through explanation delivery, UI timer delay, available main-thread heap measurements, outcome, last completed stage, placement count, and determinism.
The release protocol, recorded desktop measurements, and hardware limitations are documented in [docs/benchmark/README.md](docs/benchmark/README.md). No reference-phone run or multi-year duration ceiling is established; timing measurements are release evidence, not CI assertions.
The release protocol, recorded desktop measurements, and hardware limitations are documented in [docs/benchmark/README.md](docs/benchmark/README.md). No reference-phone run or individual multi-year duration ceiling is established. CI requires the complete seven-scenario Node protocol to finish within 300 seconds; individual scenario timings remain diagnostic evidence.
## Docker
@@ -241,7 +241,7 @@ The image builds the Angular application with Bun and Node, then runs the genera
The workflow runs on branch/tag pushes, PR opening/reopening/synchronization/edits, and manual dispatch. Manual runs validate without publishing images. Required jobs cover policy regression tests, GitFlow and Conventional Commits, ESLint, Prettier, application/test type checks, Vitest coverage, the production build, Semgrep SAST, and Trivy dependency/secret scans. Coverage, build output, and SARIF reports are retained for seven days.
Functional tests use one worker and a 2 GiB Node heap. The separate benchmark job runs without coverage and keeps its five-minute protocol timeout. At the user's request, it has `continue-on-error: true` while [#78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78) is open. #78 must restore the benchmark gate after the protocol passes. Passing functional checks during this exception does not establish benchmark acceptance.
Functional tests use one worker and a 2 GiB Node heap. The separate required benchmark job runs without coverage and keeps its five-minute protocol timeout, one warm-up and two measured repetitions of all seven scenarios. [#78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78) restores this gate after a complete local pass in 298.6 seconds; see [the measurements and hardware limits](docs/benchmark/issue-78.md). Benchmark success is required for image publication and the aggregate `required` job. The narrow local timing margin still needs confirmation on the CI runner.
The `required` job aggregates successful validation and expected image publication, accepting the image job's intentional skip for fork PRs and manual runs. Image jobs require successful validation and are limited to pushes and same-repository PRs. Registry credentials are confined to the publishing step.
+28 -31
View File
@@ -2,41 +2,38 @@
## Composable optimization rules: current verification status
The #70–#75 implementation uses explicit active optimization configurations in
the benchmark fixtures. Before trace removal, the full functional coverage run passed 652 tests
across 34 files under the 2 GB Node heap limit, excluding the separate benchmark.
That run does not establish a passing performance protocol.
[Issue #78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78) passes the
unchanged Node acceptance protocol on the documented development laptop:
**298.6 seconds**, including one warm-up and two measured repetitions of all
seven scenarios, without coverage, with one worker and a 2 GiB Node heap.
The full protocol retains its 300-second timeout. All expected outcomes,
placement counts, final stages and determinism checks pass. The fixture inputs
are unchanged; the test now prints its existing report.
On 2026-10-15, the #76 worktree based on `ea42561` passed 630 functional tests
across 33 files with coverage under the 2 GB Node heap limit. Complete outcome
comparisons against that baseline matched for all seven benchmark scenarios.
The separate, unchanged benchmark did not complete within its 300-second limit
(external timeout exit 124). Performance acceptance and any further optimization
remain with #78.
The benchmark is required again in Gitea Actions, including the image and
aggregate gates. [The #78 report](issue-78.md) records the baseline timeout,
profiling, before/after diagnostics, full protocol output, correctness checks
and browser limitations. The local pass has only 1.4 seconds of external-timeout
headroom; CI-runner performance and live workflow validation remain unverified.
The existing migration protection stays in place under
[issue #268](https://gitea.nessar.fr/nessar/vac-optimizer/issues/268).
Before the Gitea migration, GitLab pipeline 320 passed all 652 functional tests
in 109.43 seconds, then the seven-scenario protocol timed out at its unchanged
300-second limit without an out-of-memory failure. In an earlier diagnostic run, the loaded-year warm-up took
184.36 seconds and its first measured repetition began about 251.78 seconds into
the protocol. Those diagnostic timings precede the final period-projection cache
and the original-decision-trace removal in #76, and are not a result for either.
Before these optimizations, the trace-free integrated baseline at `2f4af14`
timed out after 300 seconds without a complete report. The #76 verification
previously passed 630 functional tests across 33 files with coverage under the
same heap setting, while its unchanged benchmark also timed out. Before the
Gitea migration, GitLab pipeline 320 passed 652 functional tests, but its
separate benchmark timed out as well. An earlier loaded-year warm-up took
184.36 seconds; that historical sample predates the final period-projection
cache and trace removal and is not a measurement of the current engine.
At the user's request, #75 finishes on saved-rule migration and functional
verification. #78 owns benchmark verification and implementation of further
optimizations when needed, after trace removal in #76 and integrated functional verification in #77. All seven scenarios, one warm-up and two measured repetitions per
scenario, the 2 GB heap limit and the 300-second timeout remain required for #78.
At the user's request, the Gitea CI configuration defines the benchmark in a
separate job with `continue-on-error: true` while #78 is open. Functional coverage
tests and all other required checks remain blocking. #78 must restore the required benchmark gate
after the unchanged protocol passes. A green pipeline during this exception
does not establish a passing benchmark. No automatic elapsed-time planning
cutoff is introduced. Gitea Actions is enabled; live workflow validation is tracked in
[issue #268](https://gitea.nessar.fr/nessar/vac-optimizer/issues/268). The converted
workflow preserves the separate advisory benchmark job. See the
[CI migration status](../../README.md#continuous-integration).
#75 owns functional saved-rule migration, #76 removes tracing, #77 verifies
retained behavior and #78 owns performance acceptance. No automatic elapsed-time
engine cutoff is introduced. No reference-phone run is claimed.
The results below document earlier implementations and protocols. They are
historical measurements rather than the performance result for composable rules or trace removal. Their trace counts, timings and controls describe the former implementation only.
The measurements below describe earlier implementations and protocols. Their
trace counters and timings are historical; current evidence is in
[the #78 report](issue-78.md).
## Reference-phone protocol
+15
View File
@@ -0,0 +1,15 @@
scenario repetition preparation_ms planning_ms wall_ms outcome last_stage placements deterministic max_ui_delay_ms planning_work_ms peak_main_thread_heap_bytes
representative year 1 0.1 564.4 564.6 Plan dates 30 baseline 8.0 552.6 ""
representative year 2 0.0 555.0 555.1 Plan dates 30 yes 8.1 544.4 ""
typed obligations 1 0.1 5040.7 5040.8 Plan dates 10 baseline 11.3 5030.5 ""
typed obligations 2 0.1 5141.2 5141.3 Plan dates 10 yes 12.9 5131.1 ""
chained earned entitlements 1 0.1 29.0 29.1 Plan dates 3 baseline 0.0 21.6 ""
chained earned entitlements 2 0.1 24.7 24.8 Plan dates 3 yes 0.0 17.4 ""
deliberate conflict 1 0.1 13.0 13.2 PlanningConflict 0 baseline 0.0 3.1 ""
deliberate conflict 2 0.0 12.6 12.7 PlanningConflict 0 yes 0.0 2.8 ""
loaded year 1 0.1 76341.7 76341.8 Plan dates 72 baseline 16.8 76326.9 ""
loaded year 2 0.1 79093.8 79094.0 Plan dates 72 yes 16.3 79074.9 ""
two years 1 0.1 3391.7 3391.9 Plan dates 60 baseline 12.9 3375.3 ""
two years 2 0.1 3282.6 3282.8 Plan dates 60 yes 10.9 3267.1 ""
three years 1 0.1 11333.3 11333.5 Plan dates 90 baseline 13.9 11313.4 ""
three years 2 0.1 11394.2 11394.3 Plan dates 90 yes 15.7 11368.4 ""
1 scenario repetition preparation_ms planning_ms wall_ms outcome last_stage placements deterministic max_ui_delay_ms planning_work_ms peak_main_thread_heap_bytes
2 representative year 1 0.1 564.4 564.6 Plan dates 30 baseline 8.0 552.6
3 representative year 2 0.0 555.0 555.1 Plan dates 30 yes 8.1 544.4
4 typed obligations 1 0.1 5040.7 5040.8 Plan dates 10 baseline 11.3 5030.5
5 typed obligations 2 0.1 5141.2 5141.3 Plan dates 10 yes 12.9 5131.1
6 chained earned entitlements 1 0.1 29.0 29.1 Plan dates 3 baseline 0.0 21.6
7 chained earned entitlements 2 0.1 24.7 24.8 Plan dates 3 yes 0.0 17.4
8 deliberate conflict 1 0.1 13.0 13.2 PlanningConflict 0 baseline 0.0 3.1
9 deliberate conflict 2 0.0 12.6 12.7 PlanningConflict 0 yes 0.0 2.8
10 loaded year 1 0.1 76341.7 76341.8 Plan dates 72 baseline 16.8 76326.9
11 loaded year 2 0.1 79093.8 79094.0 Plan dates 72 yes 16.3 79074.9
12 two years 1 0.1 3391.7 3391.9 Plan dates 60 baseline 12.9 3375.3
13 two years 2 0.1 3282.6 3282.8 Plan dates 60 yes 10.9 3267.1
14 three years 1 0.1 11333.3 11333.5 Plan dates 90 baseline 13.9 11313.4
15 three years 2 0.1 11394.2 11394.3 Plan dates 90 yes 15.7 11368.4
+345
View File
@@ -0,0 +1,345 @@
# Issue #78: integrated planner performance
Measurements on 2026-10-15/16 (Europe/Paris), starting from the trace-free
integrated planner at `2f4af14869c47836d2c1008157136f75c9d3e98b`.
[Issue #78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78) supplies the
acceptance criteria. The original spec, baseline source, profiles, diagnostic
runners and complete local logs are retained in `/tmp/vac-issue-78`.
## Required protocol
```sh
NODE_OPTIONS=--max-old-space-size=2048 timeout 300s bunx vitest run src/app/benchmark/leave-planner-benchmark.spec.ts --maxWorkers=1
```
| Run | Result |
| --------------------------------- | ----------------------------------------------------------------------------------------------------- |
| Unmodified baseline | External timeout, exit 124, 300.008 seconds; no complete report. |
| Optimized planner | Exit 0, 298.551 seconds externally; Vitest reports 294.16 seconds, with 292.958 seconds in the test. |
| Restart-interrupted optimized run | Stopped by the agent-server restart before a result; excluded from acceptance and timing comparisons. |
The passing run includes the existing typed-obligation precheck, all seven
warm-ups, and two measured repetitions of every scenario. Coverage is disabled,
the Node heap is limited to 2048 MiB and the full command retains its 300-second
timeout. No other build, test or profiling workload ran alongside it.
[The complete 14-row report](issue-78-protocol.tsv) preserves every measured row.
Every repetition has its expected outcome, placement count and final stage;
each second repetition matches its first complete outcome. The deliberate
conflict remains a conflict with zero placements. The other scenarios complete
through `dates`, with 30, 10, 3, 72, 60 and 90 placements respectively.
The fixture module is byte-for-byte unchanged, SHA-256
`16e057fa042c898d203f71e018f183c9fbecb5cd0edbf0ce0f3f25277cb00d3a`.
The test change only prints its existing report; all correctness assertions,
configurations, quotas, horizons, warm-ups and repetition counts remain intact.
There are no trace counters, transfers or acknowledgement waits.
The passing command has only **1.449 seconds of external-timeout headroom**.
This is a local pass, not a runner-speed guarantee. Gitea Actions now requires
benchmark success before image publication and the aggregate `required` check.
A successful live PR run, publication setup and migration-gate activation remain
with [#268](https://gitea.nessar.fr/nessar/vac-optimizer/issues/268). No issue is
closed by this local verification.
## Environment and memory
AMD Ryzen 5 7640U, 12 logical CPUs, approximately 16 GiB RAM; Linux
`6.18.33.2-microsoft-standard-WSL2`. Node 24.18.0, Bun 1.3.14, Vitest 5.0.3,
TypeScript 6.0.3, Angular core 22.2.1 and date-holidays 3.30.2. The calculation
calendar date is fixed to `2024-01-01`, matching the benchmark test. Node runs
are unthrottled. These versions differ from the configured CI container's
Node 24.21.0 and Bun 1.4.2; live CI performance has not been measured here.
The passing protocol's sampled peak process-tree RSS is **1,528,056 KiB**,
sampled every 500 ms across the timeout/Bun/Vitest process tree. RSS includes
multiple processes and runtime overhead; it is not a JS-heap measurement.
Chromium's `performance.memory` is unavailable in the Node protocol, so its
`peak_main_thread_heap_bytes` cells remain blank.
## Profile and changes
A separate, explicitly interrupted 20-second loaded-year diagnostic used Node's
inspector CPU and allocation sampling. Candidate finalization, state copying,
calendar observation construction, period bounds and work-sequence scanning
were substantial costs. Allocation sampling included collected objects; its
cumulative allocation estimates are not peak live memory or protocol timings.
The diagnostic retained a valid 72-placement partial plan on cancellation.
A separate 20-second repeated-work probe observed 29 preference-pass executions
and no identical input-state revisit among cache misses. Complete obligation
previews remain in place; there is no new state-equivalence or search-budget
shortcut.
The change addresses those measured paths:
- Evaluate objectives in their actual priority order. Once a protected prefix
loses, or an admissible candidate loses to the current best candidate, lower
objective values cannot reverse that lexicographic comparison. Full vectors
remain available for retained improvements, displayed explanations and date
analysis; tied-prefix additions keep their original retention behavior.
- Build cached calendar observations only when an evaluated objective needs
them. The existing request-local, 4096-entry geometry cache retains its
boundary; balance quantities, provenance and supporting status remain outside
it. Work-sequence lengths use the prepared working-day prefixes and period
anchors instead of rebuilding every sequence's date array for scoring.
- Share existing placements for reading during a single-date assignment, while
continuing to copy balances and earned activations. Bundle and relocation
paths detach placements before changing supporting status. Snapshot,
provenance and other mutable-copy paths retain their deep copies.
- Reuse calendar-request validation during finalization instead of looking up
and scanning the same provenance again for each placement.
No candidate destinations, allocation choices, obligation previews or rule
priorities are removed. Search limits, earned-entitlement chronology, final
quality criteria and deterministic tie-breaks are unchanged. No elapsed-time
engine cutoff is introduced.
## Differential and diagnostic measurements
Complete outcomes, stable explanations and final-state snapshots are compared
with the saved baseline. The configuration matrix contains 182 cases: supported
conditions, aggregations and directions; personal objectives before and after
loss prevention; typed obligations and preferences; declarations; at-risk and
deferred balances; custom deficit thresholds; and empty/disabled rule lists.
Three permanent regression cases also pin reordered and custom objective
vectors, placement dates, provenance and first decisive objectives.
All seven completed diagnostic comparisons also match in full, including the
loaded year and both longer horizons. Table cells show **before → after**.
| Scenario | Preparation (ms) | Planning work (s) | Wall through explanation (s) | Maximum timer delay (ms) | Sampled Node heap (MiB) |
| --------------------------- | ---------------- | ----------------- | ---------------------------- | ------------------------ | ----------------------- |
| representative year | 39.7 → 37.5 | 1.261 → 0.787 | 1.302 → 0.826 | 7.7 → 8.5 | 76.5 → 90.8 |
| typed obligations | 10.9 → 9.4 | 8.563 → 5.642 | 8.574 → 5.652 | 7.8 → 11.2 | 127.7 → 115.1 |
| chained earned entitlements | 5.4 → 6.7 | 0.025 → 0.026 | 0.030 → 0.033 | 0.0 → 0.0 | — → — |
| deliberate conflict | 10.1 → 9.5 | 0.004 → 0.004 | 0.013 → 0.013 | 0.0 → 0.0 | — → — |
| loaded year | 8.8 → 9.1 | 143.648 → 72.029 | 143.664 → 72.043 | 14.6 → 16.2 | 206.1 → 177.3 |
| two years | 17.6 → 12.3 | 5.470 → 2.962 | 5.491 → 2.977 | 13.3 → 11.0 | 186.9 → 164.8 |
| three years | 16.4 → 14.4 | 19.674 → 10.287 | 19.696 → 10.310 | 15.8 → 16.1 | 289.2 → 253.5 |
The cold diagnostics run each unchanged scenario once, without warm-up or
coverage, through `calculatePlanningAsyncInternal`, with the same 2048 MiB heap
and calculation date. They are separate from the acceptance protocol. Occasional
focused/static checks ran during the baseline investigation; those diagnostic
timings are informational, not an isolated speedup estimate. The final diagnostic
run has no competing build/test workload.
Diagnostic preparation ends at the first non-preparation progress callback;
planning work comes from the core's timing artifact; wall time ends after
explanation delivery. In the protocol TSV, `preparation_ms` instead measures
request cloning, while calendar preparation is included in `planning_ms`.
Diagnostic timer delay and `process.memoryUsage().heapUsed` are sampled every
50 ms. Heap is the whole Node process, including earlier scenarios' live data
and caches. A dash means no interval sample was taken before a fast scenario
finished. These are neither allocation totals nor worker-isolated heap claims.
## Native browser checks
Production build, native Chrome 150.0.7871.186 Workers, 375×915 viewport,
device scale 1, reduced motion and 4× CPU throttling on the laptop above.
Each completion uses a new page and worker in one isolated Chrome process,
without a planner warm-up; browser/resource caches are shared between pages.
The cancellation checks use a separate isolated Chrome process. No other
build, test or profiler runs alongside these measurements.
| Scenario | Outcome usable (s) | Calendar ready (s) | Full artifact delivery (s) | Maximum page timer delay (ms) | Peak page heap (MiB) | Chrome RSS before / after outcome (MiB) |
| ------------------- | ------------------ | ------------------ | -------------------------- | ----------------------------- | -------------------- | --------------------------------------- |
| representative year | 2.070 | 2.574 | 2.158 | 886.5 | 22.8 | 1482.5 / 1488.4 |
| two years | 5.864 | 6.879 | 6.878 | 1287.9 | 22.8 | 1538.1 / 1540.3 |
| three years | 15.865 | 17.864 | 17.861 | 1987.2 | 21.1 | 1503.2 / 1506.8 |
Every completion returns `Plan` through `dates`, with 30, 60 or 90 placements,
its stable explanation and one planning request. Worker heartbeats continue
through the long calculations. There are no unexpected page exceptions or
page-level horizontal overflow at the measured viewport.
During the three-year worker calculation, keyboard editing changed the first
visible balance from 20 to 99. The captured request remained at 20, the plan
still contained 90 placements, and no second calculation was sent. Its displayed
estimate therefore followed the pending edited configuration (22–136 seconds).
The unchanged 90-day inputs in the cancellation runs retain their 11–68 second
range; cancelled outcomes add no completion calibration sample. The existing
session calibration and request-to-rendered-calendar contract are unchanged;
these samples do not justify changing the rough initial estimate.
Page heap is Chromium's main-thread `performance.memory` sample every 50 ms;
it excludes the worker heap. Chrome RSS sums the isolated browser process tree
at 500 ms intervals and outcome/artifact boundaries, including workers,
renderers and shared pages counted in each process's RSS. It is not a unique
physical-memory or worker-heap measurement. The post-outcome samples extend
through calendar readiness and artifact delivery. Calendar readiness is
observed by a 50 ms page timer after the loading overlay disappears, rather
than a physical paint measurement.
The observed page timer delays include calendar rendering and reach about
**2.0 seconds**. Inputs accept edits during worker planning; final calendar
rendering remains a visible main-thread pause.
Both cancellation flows activate Cancel with Enter and send one cancel message,
with no additional planning request:
| Cancellation point | Cancel to outcome (ms) | Cancel to artifacts (ms) | Retained placements |
| ---------------------------------------------------------- | ---------------------- | ------------------------ | ------------------- |
| Before a valid plan, controlled startup | 61.3 | 155.0 | 0 |
| Active optimization, after a valid 90-placement checkpoint | 20.6 | 1979.4 | 90 |
The preparation probe queues outgoing worker messages until the keyboard action,
then releases the plan and cancel in order. It verifies cancellation before any
valid plan, rather than ordinary startup-latency performance. It reports
`PlanningIncomplete` without an incumbent or a false conflict. The active
cancellation displays **Calculation cancelled — partial plan**, retains all 90
placements and delivers its partial-plan explanation. The first keyboard probe
omitted the Enter character event and issued no cancel; it was corrected and
excluded from cancellation evidence.
No physical reference-phone run is available. Desktop CPU throttling does not
establish phone performance or a duration ceiling. The browser reference
protocol still uses three measured repetitions; the required Node protocol uses
two. These counts are unchanged.
## Correctness, static checks and build
```sh
NODE_OPTIONS=--max-old-space-size=2048 bunx vitest run --coverage --maxWorkers=1 --exclude='src/app/benchmark/**'
bunx tsc --noEmit -p tsconfig.app.json
bunx tsc --noEmit -p tsconfig.spec.json
bun run lint
bun run format:check
bunx prettier --check 'scripts/ci-*.mjs' '.gitea/workflows/*.yml'
node scripts/ci-policy.test.mjs
node scripts/ci-image.test.mjs
bun run build
git diff --check
```
All pass. Functional coverage: **633 tests across 33 files**, 93.61% statements,
85.99% branches, 95.70% functions and 95.44% lines, in 92.05 seconds with the
existing 2 GiB Node heap. The separate benchmark also passes, as recorded above.
Coverage settings and thresholds are unchanged. The source changes were also
checked with the 266-test focused planner suite during implementation.
A local probe runs the actual aggregate shell body for 12 combinations of
benchmark and image outcomes. Failed, cancelled and skipped benchmarks block
it; successful benchmarks still preserve the intentional fork/manual image
skip. Both `images` and `required` depend on `benchmark`. The aggregate's first
failed check exits explicitly, without depending on the runner's shell flags.
The production build retains the existing warning budgets: initial bundle
2.57 MB against 2.50 MB, and calendar-year styles 12.98 kB against 8 kB. Both
remain below their unchanged error limits. Sandbox restrictions prevented the
first build/policy attempt; reruns with local socket/child-process permissions
passed. Semgrep and Trivy are not installed locally, so their security jobs,
actual image publication and a live Gitea PR run remain unverified.
## Reproduction
Use the recorded Node/Bun versions and frozen lockfile. Run the exact acceptance
command above from the integrated baseline checkout for the before measurement,
and from this working result for the after measurement. The test prints the
entire TSV report, including failures of its unchanged assertions.
For the separate cold diagnostics, `measure-core.mjs` runs the bundled baseline
or optimized core over the unchanged `benchmarkScenarios()` JSON in fixture
order. Generate that JSON in a separate process so the measured Node process
does not include Angular fixture-export overhead:
```sh
mkdir -p tmp /tmp/vac-issue-78
cat > tmp/issue78-fixtures.ts <<'TS'
import '@angular/compiler';
import { benchmarkScenarios } from '../src/app/benchmark/leave-planner-benchmark';
console.log(JSON.stringify(benchmarkScenarios()));
TS
bun build tmp/issue78-fixtures.ts --target=node --outfile /tmp/vac-issue-78/fixtures.mjs
node /tmp/vac-issue-78/fixtures.mjs > /tmp/vac-issue-78/scenarios.json
```
The diagnostic runner supplies `calculationDate: '2024-01-01'`, captures
outcome/explanation/final-state callbacks, observes preparation progress and
samples timer delay and process heap every 50 ms:
```sh
bun build src/app/services/leave-planner.core.ts --target=node --outfile /tmp/vac-issue-78/optimized-core.mjs
node --max-old-space-size=2048 /tmp/vac-issue-78/measure-core.mjs optimized
node --max-old-space-size=2048 /tmp/vac-issue-78/compare-configurations.mjs
```
The baseline bundle and complete requests, results and profiling sources are
kept beside those runners. CPU/allocation profiling and the cancelled repeated-
work probe are diagnostics only; neither counts as a completed benchmark run.
Save the following as `/tmp/vac-issue-78/measure-core.mjs` to recreate the cold
measurements and complete comparison snapshots. Build `baseline-core.mjs` from
the recorded baseline with the same `bun build` command, then run with
`baseline`; use `optimized` for this implementation.
```js
import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
const engine = process.argv[2] || 'baseline';
const { calculatePlanningAsyncInternal } = await import(`./${engine}-core.mjs`);
const directory = new URL(`./${engine}-measurements/`, import.meta.url);
mkdirSync(directory, { recursive: true });
const scenarios = JSON.parse(readFileSync(new URL('./scenarios.json', import.meta.url)));
for (const scenario of scenarios.filter(
(item) => !process.argv[3] || item.name === process.argv[3],
)) {
let artifacts;
let outcomeAt;
let preparationMs;
let peakHeapUsedBytes = 0;
let maxTimerDelayMs = 0;
const start = performance.now();
let lastTick = start;
const sample = setInterval(() => {
const tick = performance.now();
maxTimerDelayMs = Math.max(maxTimerDelayMs, tick - lastTick - 50);
lastTick = tick;
peakHeapUsedBytes = Math.max(peakHeapUsedBytes, process.memoryUsage().heapUsed);
}, 50);
const outcome = await calculatePlanningAsyncInternal(
{
...scenario.request,
calculationDate: '2024-01-01',
progress: (stage) => {
if (stage !== 'preparation') preparationMs ??= performance.now() - start;
},
},
undefined,
(value) => {
artifacts = value;
},
() => {
outcomeAt = performance.now() - start;
},
);
clearInterval(sample);
const measurement = {
scenario: scenario.name,
diagnostic: true,
warmups: 0,
repetitions: 1,
preparationMs,
outcomeMs: outcomeAt,
planningWorkMs: artifacts.timings?.planningWorkMs,
wallMs: performance.now() - start,
maxTimerDelayMs,
peakHeapUsedBytes,
peakRssKiB: process.resourceUsage().maxRSS,
outcome: outcome.kind,
placements: outcome.placements?.length,
lastStage: outcome.lastCompletedStage,
};
writeFileSync(
new URL(scenario.name.replaceAll(' ', '-') + '.json', directory),
JSON.stringify({
measurement,
outcome,
explanation: artifacts.explanation,
finalState: artifacts.finalState,
}),
);
console.log(JSON.stringify(measurement));
}
```
+3 -3
View File
@@ -21,11 +21,11 @@ configuration; the Compose files remain manual deployment options.
| `policy` | Tests the policy and publisher, then validates the checked-out commit, GitFlow, PR title, commits, and stable release version. |
| `validate` | Runs lint, formatting, and application/test type checks. |
| `test` | Runs functional coverage with one worker and a 2 GiB Node heap; uploads coverage for seven days even on failure. |
| `benchmark` | Runs outside coverage with the same worker/heap limits and 300-second protocol timeout. Advisory while #78 is open. |
| `benchmark` | Required seven-scenario protocol, with one warm-up and two measured repetitions, outside coverage with one worker, a 2 GiB Node heap and the unchanged 300-second timeout. |
| `build` | Builds browser/server production bundles and retains `dist` for seven days. |
| `security` | Runs Semgrep JavaScript/TypeScript SAST and Trivy dependency/secret scans; retains SARIF reports for seven days. Scanner execution must succeed; findings remain reports rather than a new severity gate. |
| `images` | Publishes branch/SHA images after required validation, or promotes an existing SHA image for a stable tag. Runs only for pushes and same-repository PRs. |
| `required` | Fails on failed, cancelled, or skipped required validation. Requires image publication when expected; accepts its intentional skip for fork PRs and manual dispatch. Excludes the advisory benchmark. |
| `images` | Publishes branch/SHA images after required validation, including the benchmark, or promotes an existing SHA image for a stable tag. Runs only for pushes and same-repository PRs. |
| `required` | Fails on failed, cancelled, or skipped required validation, including the benchmark. Requires image publication when expected; accepts its intentional skip for fork PRs and manual dispatch. |
Push and PR runs are separate so PR title and rebase checks run on the PR event.
New runs cancel older runs for the same event/ref. Image jobs serialize per
@@ -96,6 +96,7 @@ describe('leave planner benchmark protocol', () => {
loaded[0].preparationMs + loaded[0].planningMs,
);
const report = benchmarkReport(measurements);
console.info(report);
expect(report.split('\n')).toHaveLength(15);
expect(report).toContain('planning_ms\twall_ms');
expect(report).toContain('planning_work_ms\tpeak_main_thread_heap_bytes');
+138 -76
View File
@@ -501,9 +501,8 @@ const occurrenceCache = new WeakMap<SearchContext, Map<string, string[]>>();
const calendarPatternCache = new WeakMap<SearchContext, Map<string, string[][]>>();
const preferencePatternCache = new WeakMap<SearchContext, Map<string, string[][][]>>();
const declaredWindowCache = new WeakMap<SearchContext, Map<string, DeclaredNonWorkingDay[]>>();
type CalendarOptimizationObservations = Pick<
OptimizationObservations,
'publicHolidayWeeks' | 'timeOffPeriods' | 'workSequences'
type CalendarOptimizationObservations = Partial<
Pick<OptimizationObservations, 'publicHolidayWeeks' | 'timeOffPeriods' | 'workSequences'>
>;
const calendarObservationCache = new WeakMap<
SearchContext,
@@ -720,18 +719,20 @@ function reason(code: string, message: string, ruleIds?: string[]): PlanningReas
return ruleIds?.length ? { code, message, ruleIds } : { code, message };
}
function cloneState(state: State): State {
function cloneState(state: State, placements?: State['placements']): State {
return {
balances: state.balances.map((balance) => ({ ...balance })),
placements: new Map(
[...state.placements.entries()].map(([date, placement]) => [
date,
{
...placement,
ruleIds: new Set(placement.ruleIds),
},
]),
),
placements:
placements ??
new Map(
[...state.placements.entries()].map(([date, placement]) => [
date,
{
...placement,
ruleIds: new Set(placement.ruleIds),
},
]),
),
earnedActivations: state.earnedActivations.map((activation) => ({
...activation,
dates: [...activation.dates],
@@ -1513,7 +1514,8 @@ function assignDateToBalance(
) {
return null;
}
const result = cloneState(state);
// Existing placements stay read-only; this assignment owns only its new placement.
const result = cloneState(state, new Map(state.placements));
const target = result.balances.find((item) => item.id === balanceId)!;
target.remaining--;
result.placements.set(date, {
@@ -2105,8 +2107,10 @@ function finalizeRuleCandidate(state: State, context: SearchContext, before: Sta
const patternCredits = new Map<string, Set<string>>();
for (const placement of state.placements.values()) {
let lostCalendarRequest = false;
let hasCalendarRequest = false;
for (const id of placement.ruleIds) {
const rule = context.rules.find((item) => item.id === id);
if (rule?.type === 'optimization') continue;
const valid = counterfactualRuleValid(
placement.date,
placement.type,
@@ -2135,18 +2139,16 @@ function finalizeRuleCandidate(state: State, context: SearchContext, before: Sta
withinRequest = patternCredits.get(id)!.has(placement.date);
}
if (valid && withinRequest) {
hasCalendarRequest = true;
if (rule?.type === 'time_window') windowCredits.set(id, credited + 1);
continue;
}
if (!valid && rule?.force === 'obligation') return null;
if (result === state) result = cloneState(state);
result.placements.get(placement.date)!.ruleIds.delete(id);
lostCalendarRequest ||= rule?.type !== 'optimization';
lostCalendarRequest = true;
}
const current = result.placements.get(placement.date)!;
const hasCalendarRequest = [...current.ruleIds].some((id) =>
context.rules.some((rule) => rule.id === id && rule.type !== 'optimization'),
);
const previous = before.placements.get(placement.date);
let supporting = current.supporting;
if (hasCalendarRequest) supporting = false;
@@ -2272,7 +2274,8 @@ function* calendarPlacementCandidates(
if (index === missing.length) {
const benefit = supportingCalendarBenefit(before, current, context);
for (const date of additions) {
const placement = current.placements.get(date)!;
const placement = { ...current.placements.get(date)! };
current.placements.set(date, placement);
const balance = current.balances.find((item) => item.id === placement.balanceId)!;
placement.supporting = benefit && !isAtRisk(balance, context);
}
@@ -2367,18 +2370,40 @@ function evaluateRuleCandidate(
addition = false,
baseline = activeObjectiveVector(before, context, throughIndex),
incumbentVector = objectiveVector(before, context),
): { state: State; vector: PlanningObjectiveVector } | undefined {
): { state: State; vector?: PlanningObjectiveVector } | undefined {
if (!candidate) return undefined;
const state = finalizeRuleCandidate(candidate, context, before);
if (!state) return undefined;
const active = activeObjectiveVector(state, context);
const comparison = compareObjectiveLists(active.slice(0, throughIndex + 1), baseline);
const active: PlanningObjective[] = [];
let comparison = 0;
let incumbentComparison = 0;
for (const objective of activeObjectives(state, context)) {
const index = active.length;
active.push(objective);
if (index <= throughIndex && comparison === 0) {
comparison = compareObjectiveValues(
objective.direction,
objective.value,
baseline[index].value,
);
// A worse protected prefix cannot improve the jointly valid incumbent.
if (comparison > 0) return undefined;
}
if (incumbentComparison === 0)
incumbentComparison = compareObjectiveValues(
objective.direction,
objective.value,
incumbentVector.objectives[index + 1].value,
);
if (incumbentComparison > 0 && (comparison < 0 || index >= throughIndex))
return addition && comparison === 0 ? undefined : { state };
}
const vector = objectiveVector(state, context, active);
// Preserve the best valid candidate even if the active prefix rejects it.
if (compareObjectiveLists(vector.objectives, incumbentVector.objectives) < 0)
context.retain?.(state, undefined, vector);
// shortcut: single additions cannot cross equal-value plateaus; add coordinated balance bundles when a measured configuration needs them.
if (comparison > 0 || (addition && comparison === 0)) return undefined;
if (addition && comparison === 0) return undefined;
return { state, vector };
}
@@ -2622,6 +2647,7 @@ function* optimizeRulePlacements(
bestVector,
);
if (!evaluated) return false;
if (!evaluated.vector) return true;
const compared = compareObjectiveLists(
evaluated.vector.objectives,
bestVector.objectives,
@@ -2842,8 +2868,14 @@ function* optimizeRulePlacements(
(candidate) => {
if (rule?.type === 'optimization' && !isAtRisk(balance, context)) {
beforeCalendar ??= optimizationObservations(result, context);
candidate.placements.get(date)!.supporting =
supportingCalendarBenefit(beforeCalendar, candidate, context);
candidate.placements.set(date, {
...candidate.placements.get(date)!,
supporting: supportingCalendarBenefit(
beforeCalendar,
candidate,
context,
),
});
}
return consider(candidate, true);
},
@@ -3306,46 +3338,49 @@ function activeObjectiveVector(
context: SearchContext,
throughIndex = Number.POSITIVE_INFINITY,
): PlanningObjective[] {
const preferences = preferenceResultsForState(state, context);
let observations: OptimizationObservations | undefined;
return context.rules
.filter((rule) => rule.enabled && rule.force === 'preference')
.slice(0, throughIndex + 1)
.map((rule) => {
if (rule.type === 'optimization') {
observations ??= context.rules.some(
(item) =>
item.type === 'optimization' &&
item.configuration.elements !== 'leave-balances',
)
? optimizationObservations(state, context)
: {
return [...activeObjectives(state, context, throughIndex)];
}
function* activeObjectives(
state: State,
context: SearchContext,
throughIndex = Number.POSITIVE_INFINITY,
): Generator<PlanningObjective> {
let preferences: PreferenceResult[] | undefined;
let balances: OptimizationObservations | undefined;
let calendar: OptimizationObservations | undefined;
let index = 0;
for (const rule of context.rules) {
if (!rule.enabled || rule.force !== 'preference') continue;
if (index++ > throughIndex) return;
if (rule.type === 'optimization') {
const observations =
rule.configuration.elements === 'leave-balances'
? (balances ??= {
leaveBalances: optimizationBalanceObservations(state, context),
publicHolidayWeeks: [],
timeOffPeriods: [],
workSequences: [],
leavePlacements: [],
};
return {
id: `optimization:${rule.id}` as const,
direction: rule.configuration!.direction,
value: evaluateStateOptimization(
rule.configuration!,
state,
context,
observations,
),
label: rule.name,
};
}
return {
id: `preference:${rule.id}` as const,
direction: 'maximize' as const,
})
: (calendar ??= optimizationObservations(state, context));
yield {
id: `optimization:${rule.id}`,
direction: rule.configuration.direction,
value: evaluateStateOptimization(rule.configuration, state, context, observations),
label: rule.name,
};
} else {
preferences ??= preferenceResultsForState(state, context);
yield {
id: `preference:${rule.id}`,
direction: 'maximize',
value:
preferences.find((preference) => preference.ruleId === rule.id)?.satisfied ?? 0,
label: rule.name,
};
});
}
}
}
function tieBreakObjectives(state: State): PlanningObjective[] {
@@ -3513,11 +3548,15 @@ function optimizationObservations(
byDates = new Map();
calendarObservationCache.set(context, byDates);
}
let calendar = byDates.get(key);
if (calendar) byDates.delete(key);
else {
calendar = Object.freeze({
publicHolidayWeeks: Object.freeze(
const calendar = byDates.get(key) ?? {};
if (byDates.has(key)) byDates.delete(key);
else if (byDates.size >= 4096) byDates.delete(byDates.keys().next().value!);
byDates.set(key, calendar);
let bounds: ReturnType<typeof periodBounds> | undefined;
return {
leaveBalances: optimizationBalanceObservations(state, context),
get publicHolidayWeeks() {
return (calendar.publicHolidayWeeks ??= Object.freeze(
context.holidayWeeks.map((week) =>
Object.freeze({
value: week.value,
@@ -3526,37 +3565,60 @@ function optimizationObservations(
),
}),
),
),
timeOffPeriods: Object.freeze(
));
},
get timeOffPeriods() {
return (calendar.timeOffPeriods ??= Object.freeze(
periods
? periods.map((period) =>
Object.freeze({
nonWorkingDates: Object.freeze([...period.nonWorkingDates]),
}),
)
: periodBounds(state, context).map(({ startIndex, endIndex }) =>
: (bounds ??= periodBounds(state, context)).map(({ startIndex, endIndex }) =>
Object.freeze({
nonWorkingDates: calendarPeriod(startIndex, endIndex, context)
.nonWorkingDates,
}),
),
),
workSequences: Object.freeze(
workSequenceDates(state, context).map(({ dates, internal }) =>
Object.freeze({ length: dates.length, internal }),
));
},
get workSequences() {
return (calendar.workSequences ??= Object.freeze(
workSequenceLengths((bounds ??= periodBounds(state, context)), context).map(
(sequence) => Object.freeze(sequence),
),
),
});
if (byDates.size >= 4096) byDates.delete(byDates.keys().next().value!);
}
byDates.set(key, calendar);
return {
leaveBalances: optimizationBalanceObservations(state, context),
...calendar,
));
},
leavePlacements: [...state.placements.values()],
};
}
function workSequenceLengths(
bounds: ReturnType<typeof periodBounds>,
context: SearchContext,
): OptimizationObservations['workSequences'] {
if (!context.workingDays.length) return [];
let nextIndex = context.calendarIndex.get(context.horizon.start)!;
const sequences = bounds.map(({ anchors }, index) => {
const previousIndex = nextIndex;
nextIndex = context.calendarIndex.get(anchors[anchors.length - 1])! + 1;
return {
length:
context.workingPrefix[context.calendarIndex.get(anchors[0])!] -
context.workingPrefix[previousIndex],
internal: index > 0,
};
});
sequences.push({
length:
context.workingPrefix[context.calendarIndex.get(context.horizon.end)!] -
context.workingPrefix[nextIndex],
internal: false,
});
return sequences;
}
function workSequenceDates(state: State, context: SearchContext) {
const sequences: string[][] = [];
let current: string[] = [];
@@ -71,6 +71,64 @@ describe('integrated optimization and earned leave', () => {
]);
});
it.each([
{
name: 'holiday value first',
rules: [presets[1], presets[0], ...presets.slice(2)],
dates: ['2024-05-06', '2024-05-07', '2024-05-10'],
values: [0, 5, 0, 6, 5, 0, 0],
decisive: 'optimization:default-holiday-week-value',
},
{
name: 'non-working-day gain first',
rules: [presets[2], presets[0], presets[1], ...presets.slice(3)],
dates: ['2024-05-03', '2024-05-10', '2024-05-17'],
values: [0, 9, 0, 0, 4, 4, 0],
decisive: 'optimization:default-non-working-day-gain',
},
{
name: 'maximum holiday value after loss prevention',
rules: [
presets[0],
{
...presets[1],
configuration: {
...presets[1].configuration,
aggregation: 'maximum' as const,
},
},
...presets.slice(2),
],
dates: ['2024-05-06', '2024-05-07', '2024-05-10'],
values: [0, 0, 5, 6, 5, 0, 0],
decisive: 'optimization:default-unplaced-at-risk',
},
])('preserves complete vectors and placement explanations with $name', (fixture) => {
let artifacts: PlanningArtifacts = {};
const outcome = calculatePlanningInternal(
{
...composedRequest,
horizon: { start: '2024-04-29', end: '2024-05-18' },
balances: [{ id: 'cp', type: 'cp', quantity: 3, expiryDate: '2024-05-17' }],
rules: fixture.rules,
},
undefined,
(value) => (artifacts = value),
);
if (outcome.kind !== 'Plan' || artifacts.explanation?.kind !== 'PlanExplanation')
throw new Error('Expected a complete plan and explanation.');
expect(outcome.placements.map(({ date }) => date)).toEqual(fixture.dates);
for (const placement of artifacts.explanation.placements) {
expect(placement.objectiveVector.objectives.map(({ value }) => value)).toEqual([
...fixture.values,
'2024-05-17|2024-05-17|2024-05-17',
fixture.dates.join('|'),
]);
expect(placement.firstDecisiveObjective).toBe(fixture.decisive);
expect(placement.ruleIds).toEqual([fixture.decisive.slice('optimization:'.length)]);
}
});
it('preserves the first equivalent funder while evaluating typed window alternatives', () => {
const outcome = calculatePlanningInternal({
...composedRequest,