vac-optimizer/docs/benchmark

Mobile leave-planner benchmark

Open the deployed application at /benchmark on the reference phone. Tap Run benchmark, wait for the five scenarios to finish, then tap Copy report and save the tab-separated output with the date and build identifier.

The harness measures preparation, planning-to-outcome latency, later trace delivery, and total wall time separately. It also reports planning work, trace capture, trace materialization, and trace transfer independently, plus the largest extra delay of a 50 ms UI heartbeat. It records the outcome, last completed quality stage, placement count, determinism, trace status, and trace candidate count. PlanningIncomplete is not a conflict: its last completed stage and incumbent are retained.

Protocol

• Record phone model, OS version, browser/version, application build, and battery level.
• Use a cool phone, unplugged, with other applications closed. Record whether the device became warm or throttled during the run.
• Tap Run benchmark once. It first runs every scenario as an unreported warm-up, then records three measured repetitions per scenario. Do not interact with the page while a run is active. The first measured result is the determinism baseline; later repetitions report whether they match it.
• The original target was below five seconds for representative and loaded scenarios, with a ten-second planning ceiling for every scenario. The owner accepted six to seven seconds for the loaded scenario on 2026-09-25. These are release measurements, not CI timing assertions.
• Keep the copied report unchanged, including any incomplete outcomes or hardware limitations. No mobile-device measurement is claimed by the source tree until a human records it here.

The browser page uses the same createPlanningCalculation(request) context as the calendar and adds no solver or backend dependency. The ten-second release ceiling applies to planning work; trace delivery is reported separately and cannot replace a valid planning outcome.

planning_ms is the observed latency until the outcome is usable; planning_work_ms excludes trace capture. trace_capture_ms accounts for instrumentation during planning, trace_materialization_ms accounts for building the stable explanation and finishing the trace after the outcome. trace_transfer_ms measures worker transfer and receiver assembly work; trace_delivery_ms measures elapsed time until the complete trace is ready. max_ui_delay_ms includes planning, trace delivery, and report preparation. Trace readiness has no ten-second ceiling and never permits truncating actual candidate evaluations.

Hardware status

No reference-phone run has been recorded. The project owner elected to close the release tickets on 2026-09-25 using the emulated run below, accepting its six-to-seven-second loaded planning time and the observed post-result trace UI pause. This is an explicit release exception, not a reference-phone measurement or proof of physical-device responsiveness.

Emulated browser run — 2026-09-25

Chromium 149.0.7827.55 on an AMD Ryzen 7 7800X3D (x86_64), headless at 412×915 pixels, device scale 2.625, touch enabled, and 4× CPU throttling. One full warm-up was excluded, followed by three measured repetitions. All five scenarios returned within ten seconds of planning time and complete results matched across repetitions. The loaded year took 5.74–6.17 seconds, above the five-second target. A 100 ms main-thread timer observed a maximum extra delay of 5,950.9 ms. Worker event timestamps place the four loaded-year pauses of 4.0–6.0 seconds after their planning results, around delivery of full trace artifacts. This desktop emulation does not establish reference-phone performance or responsiveness.

scenario	repetition	preparation_ms	planning_ms	trace_delivery_ms	wall_ms	outcome	last_stage	placements	deterministic	trace_status	trace_candidates
representative year	1	0.0	443.7	624.7	1069.1	Plan	dates	30	baseline	ready	29451
representative year	2	0.1	490.0	609.3	1099.4	Plan	dates	30	yes	ready	29451
representative year	3	0.1	465.5	614.2	1079.8	Plan	dates	30	yes	ready	29451
typed obligations	1	0.0	157.0	261.2	418.2	Plan	dates	10	baseline	ready	13103
typed obligations	2	0.0	162.6	255.6	418.2	Plan	dates	10	yes	ready	13103
typed obligations	3	0.0	163.4	258.2	421.6	Plan	dates	10	yes	ready	13103
chained earned entitlements	1	0.0	45.3	1.0	47.0	Plan	dates	3	baseline	ready	27
chained earned entitlements	2	0.1	44.5	0.9	45.5	Plan	dates	3	yes	ready	27
chained earned entitlements	3	0.1	44.4	0.8	45.4	Plan	dates	3	yes	ready	27
deliberate conflict	1	0.0	92.8	0.9	93.8	PlanningConflict		0	baseline	ready	366
deliberate conflict	2	0.0	44.7	0.2	44.9	PlanningConflict		0	yes	ready	366
deliberate conflict	3	0.0	43.5	0.6	44.1	PlanningConflict		0	yes	ready	366
loaded year	1	0.3	5901.5	5677.7	11581.0	Plan	dates	144	baseline	ready	865117
loaded year	2	0.8	6167.5	7582.4	13752.4	Plan	dates	144	yes	ready	865117
loaded year	3	0.6	5738.3	6219.5	11962.5	Plan	dates	144	yes	ready	865117

Exact-trace verification — 2026-10-03

Working changes based on 0ce33cc9d6fe7381965a9162571abe205d0b7350, Chromium 149.0.7827.55 on the same AMD Ryzen 7 7800X3D desktop, 412×915 pixels, device scale 2.625, touch enabled, and 4× CPU throttling. One warm-up preceded three recorded repetitions. This is desktop emulation; no physical-phone result is claimed, and the earlier release exception is not used as evidence for this implementation.

All five scenarios matched their pre-change outcomes in a separate core comparison, including traced versus untraced runs. Every repeated browser outcome matched its first repetition. Each loaded run returned 144 placements and retained all 868,919 actual trace candidate records with a ready trace.

Loaded planning work took 5.81–6.18 seconds, below its ten-second budget. Observed outcome latency was 10.36–10.85 seconds, including trace capture; this latency is distinct from the planning-work budget. Complete trace delivery followed 24.09–25.10 seconds later. No evaluation was truncated to meet a timing target. The worst extra delay of the 50 ms heartbeat was 946.8 ms across the loaded runs, including report preparation, compared with 4.65–5.49 seconds before acknowledged delivery in the same final-trace harness.

The worker stages snapshots during planning and delivers at most sixteen 64-operation packets before waiting for receiver acknowledgements. The receiver applies bounded batches, releases consumed packets, and yields between tasks. The worker stops after its terminal packet; benchmark helpers keep completed trace references out of the next repetition. These changes avoid the observed message-flood pauses and repeated-run renderer crashes. The complete trace still requires substantial memory: a separate unthrottled core profile retained approximately 1.06 GB of heap for the loaded trace. These results do not establish physical-device performance.

A separate rendered-calendar run at the same emulated mobile size used twelve balances of twelve days each. It produced 144 placements, 113 gained non-working days, and a longest work sequence of five. Full trace navigation and date comparison completed without browser errors. Maximum extra heartbeat delays were 487.4 ms during planning and calendar rendering, 380.8 ms during trace exploration, and 540.3 ms during comparison. Keyboard navigation, dialog focus restoration, background comparison without an implicit modal, and responsive widths of 375, 768, and 1440 pixels also passed.

The final suite passes 201 tests across 23 files, application and specification type checks, ESLint, and the production build. Build warnings remain for the 2.55 MB initial bundle (2.50 MB warning budget) and 13.41 kB calendar stylesheet (8.00 kB warning budget); hard budgets are not exceeded.

scenario	repetition	preparation_ms	planning_ms	trace_delivery_ms	wall_ms	outcome	last_stage	placements	deterministic	trace_status	trace_candidates	max_ui_delay_ms	planning_work_ms	trace_capture_ms	trace_materialization_ms	trace_transfer_ms
representative year	1	0.0	622.4	716.9	1340.3	Plan	dates	30	baseline	ready	29296	18.2	265.0	256.4	7.4	166.4
representative year	2	0.0	593.5	695.8	1289.4	Plan	dates	30	yes	ready	29296	14.5	255.3	233.8	7.5	184.1
representative year	3	0.0	592.0	721.0	1313.2	Plan	dates	30	yes	ready	29296	11.5	256.1	234.4	9.2	191.8
typed obligations	1	0.1	259.7	324.3	584.6	Plan	dates	10	baseline	ready	13005	11.8	106.8	56.6	3.7	70.2
typed obligations	2	0.1	266.2	330.6	596.9	Plan	dates	10	yes	ready	13005	11.7	110.6	60.3	4.1	81.9
typed obligations	3	0.1	259.6	341.8	601.5	Plan	dates	10	yes	ready	13005	15.7	112.3	54.7	4.0	75.9
chained earned entitlements	1	0.1	94.5	3.0	97.6	Plan	dates	3	baseline	ready	15	0.8	3.4	1.3	0.9	0.3
chained earned entitlements	2	0.0	100.2	2.5	102.7	Plan	dates	3	yes	ready	15	0.2	3.6	1.4	0.7	0.1
chained earned entitlements	3	0.0	96.0	1.5	97.5	Plan	dates	3	yes	ready	15	0.1	3.2	1.5	0.7	0.1
deliberate conflict	1	0.0	94.4	2.4	96.8	PlanningConflict		0	baseline	ready	4	0.2	0.9	0.8	0.3	0.2
deliberate conflict	2	0.0	93.2	1.0	95.0	PlanningConflict		0	yes	ready	4	0.2	1.1	0.5	0.2	0.0
deliberate conflict	3	0.1	92.8	1.6	95.2	PlanningConflict		0	yes	ready	4	1.4	1.0	0.7	0.2	0.1
loaded year	1	0.0	10852.1	25102.6	36002.9	Plan	dates	144	baseline	ready	868919	946.8	6014.8	4730.6	948.3	8862.1
loaded year	2	0.1	10364.4	24816.9	35216.6	Plan	dates	144	yes	ready	868919	791.2	5813.1	4440.6	296.7	8693.3
loaded year	3	0.1	10807.8	24089.4	34933.6	Plan	dates	144	yes	ready	868919	721.7	6176.0	4526.6	978.7	8408.0