vac-optimizer/docs/benchmark/issue-78.md

Issue #78: integrated planner performance

Measurements on 2026-10-15/16 (Europe/Paris), starting from the trace-free integrated planner at 2f4af14869c47836d2c1008157136f75c9d3e98b. Issue #78 supplies the acceptance criteria. The original spec, baseline source, profiles, diagnostic runners and complete local logs are retained in /tmp/vac-issue-78.

Required protocol

NODE_OPTIONS=--max-old-space-size=2048 timeout 300s bunx vitest run src/app/benchmark/leave-planner-benchmark.spec.ts --maxWorkers=1

Run	Result
Unmodified baseline	External timeout, exit 124, 300.008 seconds; no complete report.
Optimized planner	Exit 0, 298.551 seconds externally; Vitest reports 294.16 seconds, with 292.958 seconds in the test.
Restart-interrupted optimized run	Stopped by the agent-server restart before a result; excluded from acceptance and timing comparisons.

The passing run includes the existing typed-obligation precheck, all seven warm-ups, and two measured repetitions of every scenario. Coverage is disabled, the Node heap is limited to 2048 MiB and the full command retains its 300-second timeout. No other build, test or profiling workload ran alongside it.

The complete 14-row report preserves every measured row. Every repetition has its expected outcome, placement count and final stage; each second repetition matches its first complete outcome. The deliberate conflict remains a conflict with zero placements. The other scenarios complete through dates, with 30, 10, 3, 72, 60 and 90 placements respectively.

The fixture module is byte-for-byte unchanged, SHA-256 16e057fa042c898d203f71e018f183c9fbecb5cd0edbf0ce0f3f25277cb00d3a. The test change only prints its existing report; all correctness assertions, configurations, quotas, horizons, warm-ups and repetition counts remain intact. There are no trace counters, transfers or acknowledgement waits.

The passing command has only 1.449 seconds of external-timeout headroom. This is a local pass, not a runner-speed guarantee. Gitea Actions now requires benchmark success before image publication and the aggregate required check. A successful live PR run, publication setup and migration-gate activation remain with #268. No issue is closed by this local verification.

Environment and memory

AMD Ryzen 5 7640U, 12 logical CPUs, approximately 16 GiB RAM; Linux 6.18.33.2-microsoft-standard-WSL2. Node 24.18.0, Bun 1.3.14, Vitest 5.0.3, TypeScript 6.0.3, Angular core 22.2.1 and date-holidays 3.30.2. The calculation calendar date is fixed to 2024-01-01, matching the benchmark test. Node runs are unthrottled. These versions differ from the configured CI container's Node 24.21.0 and Bun 1.4.2; live CI performance has not been measured here.

The passing protocol's sampled peak process-tree RSS is 1,528,056 KiB, sampled every 500 ms across the timeout/Bun/Vitest process tree. RSS includes multiple processes and runtime overhead; it is not a JS-heap measurement. Chromium's performance.memory is unavailable in the Node protocol, so its peak_main_thread_heap_bytes cells remain blank.

Profile and changes

A separate, explicitly interrupted 20-second loaded-year diagnostic used Node's inspector CPU and allocation sampling. Candidate finalization, state copying, calendar observation construction, period bounds and work-sequence scanning were substantial costs. Allocation sampling included collected objects; its cumulative allocation estimates are not peak live memory or protocol timings. The diagnostic retained a valid 72-placement partial plan on cancellation.

A separate 20-second repeated-work probe observed 29 preference-pass executions and no identical input-state revisit among cache misses. Complete obligation previews remain in place; there is no new state-equivalence or search-budget shortcut.

The change addresses those measured paths:

• Evaluate objectives in their actual priority order. Once a protected prefix loses, or an admissible candidate loses to the current best candidate, lower objective values cannot reverse that lexicographic comparison. Full vectors remain available for retained improvements, displayed explanations and date analysis; tied-prefix additions keep their original retention behavior.
• Build cached calendar observations only when an evaluated objective needs them. The existing request-local, 4096-entry geometry cache retains its boundary; balance quantities, provenance and supporting status remain outside it. Work-sequence lengths use the prepared working-day prefixes and period anchors instead of rebuilding every sequence's date array for scoring.
• Share existing placements for reading during a single-date assignment, while continuing to copy balances and earned activations. Bundle and relocation paths detach placements before changing supporting status. Snapshot, provenance and other mutable-copy paths retain their deep copies.
• Reuse calendar-request validation during finalization instead of looking up and scanning the same provenance again for each placement.

No candidate destinations, allocation choices, obligation previews or rule priorities are removed. Search limits, earned-entitlement chronology, final quality criteria and deterministic tie-breaks are unchanged. No elapsed-time engine cutoff is introduced.

Differential and diagnostic measurements

Complete outcomes, stable explanations and final-state snapshots are compared with the saved baseline. The configuration matrix contains 182 cases: supported conditions, aggregations and directions; personal objectives before and after loss prevention; typed obligations and preferences; declarations; at-risk and deferred balances; custom deficit thresholds; and empty/disabled rule lists. Three permanent regression cases also pin reordered and custom objective vectors, placement dates, provenance and first decisive objectives.

All seven completed diagnostic comparisons also match in full, including the loaded year and both longer horizons. Table cells show before → after.

Scenario	Preparation (ms)	Planning work (s)	Wall through explanation (s)	Maximum timer delay (ms)	Sampled Node heap (MiB)
representative year	39.7 → 37.5	1.261 → 0.787	1.302 → 0.826	7.7 → 8.5	76.5 → 90.8
typed obligations	10.9 → 9.4	8.563 → 5.642	8.574 → 5.652	7.8 → 11.2	127.7 → 115.1
chained earned entitlements	5.4 → 6.7	0.025 → 0.026	0.030 → 0.033	0.0 → 0.0	— → —
deliberate conflict	10.1 → 9.5	0.004 → 0.004	0.013 → 0.013	0.0 → 0.0	— → —
loaded year	8.8 → 9.1	143.648 → 72.029	143.664 → 72.043	14.6 → 16.2	206.1 → 177.3
two years	17.6 → 12.3	5.470 → 2.962	5.491 → 2.977	13.3 → 11.0	186.9 → 164.8
three years	16.4 → 14.4	19.674 → 10.287	19.696 → 10.310	15.8 → 16.1	289.2 → 253.5

The cold diagnostics run each unchanged scenario once, without warm-up or coverage, through calculatePlanningAsyncInternal, with the same 2048 MiB heap and calculation date. They are separate from the acceptance protocol. Occasional focused/static checks ran during the baseline investigation; those diagnostic timings are informational, not an isolated speedup estimate. The final diagnostic run has no competing build/test workload.

Diagnostic preparation ends at the first non-preparation progress callback; planning work comes from the core's timing artifact; wall time ends after explanation delivery. In the protocol TSV, preparation_ms instead measures request cloning, while calendar preparation is included in planning_ms.

Diagnostic timer delay and process.memoryUsage().heapUsed are sampled every 50 ms. Heap is the whole Node process, including earlier scenarios' live data and caches. A dash means no interval sample was taken before a fast scenario finished. These are neither allocation totals nor worker-isolated heap claims.

Native browser checks

Production build, native Chrome 150.0.7871.186 Workers, 375×915 viewport, device scale 1, reduced motion and 4× CPU throttling on the laptop above. Each completion uses a new page and worker in one isolated Chrome process, without a planner warm-up; browser/resource caches are shared between pages. The cancellation checks use a separate isolated Chrome process. No other build, test or profiler runs alongside these measurements.

Scenario	Outcome usable (s)	Calendar ready (s)	Full artifact delivery (s)	Maximum page timer delay (ms)	Peak page heap (MiB)	Chrome RSS before / after outcome (MiB)
representative year	2.070	2.574	2.158	886.5	22.8	1482.5 / 1488.4
two years	5.864	6.879	6.878	1287.9	22.8	1538.1 / 1540.3
three years	15.865	17.864	17.861	1987.2	21.1	1503.2 / 1506.8

Every completion returns Plan through dates, with 30, 60 or 90 placements, its stable explanation and one planning request. Worker heartbeats continue through the long calculations. There are no unexpected page exceptions or page-level horizontal overflow at the measured viewport.

During the three-year worker calculation, keyboard editing changed the first visible balance from 20 to 99. The captured request remained at 20, the plan still contained 90 placements, and no second calculation was sent. Its displayed estimate therefore followed the pending edited configuration (22–136 seconds). The unchanged 90-day inputs in the cancellation runs retain their 11–68 second range; cancelled outcomes add no completion calibration sample. The existing session calibration and request-to-rendered-calendar contract are unchanged; these samples do not justify changing the rough initial estimate.

Page heap is Chromium's main-thread performance.memory sample every 50 ms; it excludes the worker heap. Chrome RSS sums the isolated browser process tree at 500 ms intervals and outcome/artifact boundaries, including workers, renderers and shared pages counted in each process's RSS. It is not a unique physical-memory or worker-heap measurement. The post-outcome samples extend through calendar readiness and artifact delivery. Calendar readiness is observed by a 50 ms page timer after the loading overlay disappears, rather than a physical paint measurement.

The observed page timer delays include calendar rendering and reach about 2.0 seconds. Inputs accept edits during worker planning; final calendar rendering remains a visible main-thread pause.

Both cancellation flows activate Cancel with Enter and send one cancel message, with no additional planning request:

Cancellation point	Cancel to outcome (ms)	Cancel to artifacts (ms)	Retained placements
Before a valid plan, controlled startup	61.3	155.0	0
Active optimization, after a valid 90-placement checkpoint	20.6	1979.4	90

The preparation probe queues outgoing worker messages until the keyboard action, then releases the plan and cancel in order. It verifies cancellation before any valid plan, rather than ordinary startup-latency performance. It reports PlanningIncomplete without an incumbent or a false conflict. The active cancellation displays Calculation cancelled — partial plan, retains all 90 placements and delivers its partial-plan explanation. The first keyboard probe omitted the Enter character event and issued no cancel; it was corrected and excluded from cancellation evidence.

No physical reference-phone run is available. Desktop CPU throttling does not establish phone performance or a duration ceiling. The browser reference protocol still uses three measured repetitions; the required Node protocol uses two. These counts are unchanged.

Correctness, static checks and build

NODE_OPTIONS=--max-old-space-size=2048 bunx vitest run --coverage --maxWorkers=1 --exclude='src/app/benchmark/**'
bunx tsc --noEmit -p tsconfig.app.json
bunx tsc --noEmit -p tsconfig.spec.json
bun run lint
bun run format:check
bunx prettier --check 'scripts/ci-*.mjs' '.gitea/workflows/*.yml'
node scripts/ci-policy.test.mjs
node scripts/ci-image.test.mjs
bun run build
git diff --check

All pass. Functional coverage: 633 tests across 33 files, 93.61% statements, 85.99% branches, 95.70% functions and 95.44% lines, in 92.05 seconds with the existing 2 GiB Node heap. The separate benchmark also passes, as recorded above. Coverage settings and thresholds are unchanged. The source changes were also checked with the 266-test focused planner suite during implementation.

A local probe runs the actual aggregate shell body for 12 combinations of benchmark and image outcomes. Failed, cancelled and skipped benchmarks block it; successful benchmarks still preserve the intentional fork/manual image skip. Both images and required depend on benchmark. The aggregate's first failed check exits explicitly, without depending on the runner's shell flags.

The production build retains the existing warning budgets: initial bundle 2.57 MB against 2.50 MB, and calendar-year styles 12.98 kB against 8 kB. Both remain below their unchanged error limits. Sandbox restrictions prevented the first build/policy attempt; reruns with local socket/child-process permissions passed. Semgrep and Trivy are not installed locally, so their security jobs, actual image publication and a live Gitea PR run remain unverified.

Reproduction

Use the recorded Node/Bun versions and frozen lockfile. Run the exact acceptance command above from the integrated baseline checkout for the before measurement, and from this working result for the after measurement. The test prints the entire TSV report, including failures of its unchanged assertions.

For the separate cold diagnostics, measure-core.mjs runs the bundled baseline or optimized core over the unchanged benchmarkScenarios() JSON in fixture order. Generate that JSON in a separate process so the measured Node process does not include Angular fixture-export overhead:

mkdir -p tmp /tmp/vac-issue-78
cat > tmp/issue78-fixtures.ts <<'TS'
import '@angular/compiler';
import { benchmarkScenarios } from '../src/app/benchmark/leave-planner-benchmark';
console.log(JSON.stringify(benchmarkScenarios()));
TS
bun build tmp/issue78-fixtures.ts --target=node --outfile /tmp/vac-issue-78/fixtures.mjs
node /tmp/vac-issue-78/fixtures.mjs > /tmp/vac-issue-78/scenarios.json

The diagnostic runner supplies calculationDate: '2024-01-01', captures outcome/explanation/final-state callbacks, observes preparation progress and samples timer delay and process heap every 50 ms:

bun build src/app/services/leave-planner.core.ts --target=node --outfile /tmp/vac-issue-78/optimized-core.mjs
node --max-old-space-size=2048 /tmp/vac-issue-78/measure-core.mjs optimized
node --max-old-space-size=2048 /tmp/vac-issue-78/compare-configurations.mjs

The baseline bundle and complete requests, results and profiling sources are kept beside those runners. CPU/allocation profiling and the cancelled repeated- work probe are diagnostics only; neither counts as a completed benchmark run.

Save the following as /tmp/vac-issue-78/measure-core.mjs to recreate the cold measurements and complete comparison snapshots. Build baseline-core.mjs from the recorded baseline with the same bun build command, then run with baseline; use optimized for this implementation.

import { readFileSync, writeFileSync, mkdirSync } from 'node:fs';
const engine = process.argv[2] || 'baseline';
const { calculatePlanningAsyncInternal } = await import(`./${engine}-core.mjs`);
const directory = new URL(`./${engine}-measurements/`, import.meta.url);
mkdirSync(directory, { recursive: true });
const scenarios = JSON.parse(readFileSync(new URL('./scenarios.json', import.meta.url)));
for (const scenario of scenarios.filter(
	(item) => !process.argv[3] || item.name === process.argv[3],
)) {
	let artifacts;
	let outcomeAt;
	let preparationMs;
	let peakHeapUsedBytes = 0;
	let maxTimerDelayMs = 0;
	const start = performance.now();
	let lastTick = start;
	const sample = setInterval(() => {
		const tick = performance.now();
		maxTimerDelayMs = Math.max(maxTimerDelayMs, tick - lastTick - 50);
		lastTick = tick;
		peakHeapUsedBytes = Math.max(peakHeapUsedBytes, process.memoryUsage().heapUsed);
	}, 50);
	const outcome = await calculatePlanningAsyncInternal(
		{
			...scenario.request,
			calculationDate: '2024-01-01',
			progress: (stage) => {
				if (stage !== 'preparation') preparationMs ??= performance.now() - start;
			},
		},
		undefined,
		(value) => {
			artifacts = value;
		},
		() => {
			outcomeAt = performance.now() - start;
		},
	);
	clearInterval(sample);
	const measurement = {
		scenario: scenario.name,
		diagnostic: true,
		warmups: 0,
		repetitions: 1,
		preparationMs,
		outcomeMs: outcomeAt,
		planningWorkMs: artifacts.timings?.planningWorkMs,
		wallMs: performance.now() - start,
		maxTimerDelayMs,
		peakHeapUsedBytes,
		peakRssKiB: process.resourceUsage().maxRSS,
		outcome: outcome.kind,
		placements: outcome.placements?.length,
		lastStage: outcome.lastCompletedStage,
	};
	writeFileSync(
		new URL(scenario.name.replaceAll(' ', '-') + '.json', directory),
		JSON.stringify({
			measurement,
			outcome,
			explanation: artifacts.explanation,
			finalState: artifacts.finalState,
		}),
	);
	console.log(JSON.stringify(measurement));
}