nessar
f1f08356df
fix(benchmark): update deterministic checks and improve measurement accuracy fix(service): refine candidate recording and enhance balance assignment logic test(service): add test for balance recording with preference assignment
75 lines
4.2 KiB
Markdown
75 lines
4.2 KiB
Markdown
# Mobile leave-planner benchmark
|
||
|
||
Open the deployed application at `/benchmark` on the reference phone. Tap
|
||
**Run benchmark**, wait for the five scenarios to finish, then tap **Copy
|
||
report** and save the tab-separated output with the date and build identifier.
|
||
|
||
The harness measures preparation, planning-to-outcome latency, later trace
|
||
delivery, and total wall time separately. It records the outcome, last
|
||
completed quality stage, placement count, determinism, trace status, and trace
|
||
candidate count. `PlanningIncomplete` is not a conflict: its last completed
|
||
stage and incumbent are retained.
|
||
|
||
## Protocol
|
||
|
||
- Record phone model, OS version, browser/version, application build, and
|
||
battery level.
|
||
- Use a cool phone, unplugged, with other applications closed. Record whether
|
||
the device became warm or throttled during the run.
|
||
- Tap **Run benchmark** once. It first runs every scenario as an unreported
|
||
warm-up, then records three measured repetitions per scenario. Do not interact
|
||
with the page while a run is active. The first measured result is the
|
||
determinism baseline; later repetitions report whether they match it.
|
||
- The original target was below five seconds for representative and loaded
|
||
scenarios, with a ten-second planning ceiling for every scenario. The owner
|
||
accepted six to seven seconds for the loaded scenario on 2026-09-25. These
|
||
are release measurements, not CI timing assertions.
|
||
- Keep the copied report unchanged, including any incomplete outcomes or
|
||
hardware limitations. No mobile-device measurement is claimed by the source
|
||
tree until a human records it here.
|
||
|
||
The browser page uses the same `createPlanningCalculation(request)` context as
|
||
the calendar and adds no solver or backend dependency. The ten-second release
|
||
ceiling applies to planning latency; trace delivery is reported separately and
|
||
cannot replace a valid planning outcome.
|
||
|
||
## Hardware status
|
||
|
||
No reference-phone run has been recorded. The project owner elected to close
|
||
the release tickets on 2026-09-25 using the emulated run below, accepting its
|
||
six-to-seven-second loaded planning time and the observed post-result trace UI
|
||
pause. This is an explicit release exception, not a reference-phone measurement
|
||
or proof of physical-device responsiveness.
|
||
|
||
## Emulated browser run — 2026-09-25
|
||
|
||
Chromium 149.0.7827.55 on an AMD Ryzen 7 7800X3D (x86_64), headless at
|
||
412×915 pixels, device scale 2.625, touch enabled, and 4× CPU throttling.
|
||
One full warm-up was excluded, followed by three measured repetitions. All
|
||
five scenarios returned within ten seconds of planning time and complete
|
||
results matched across repetitions. The loaded year took 5.74–6.17 seconds,
|
||
above the five-second target. A 100 ms main-thread timer observed a maximum
|
||
extra delay of 5,950.9 ms. Worker event timestamps place the four loaded-year
|
||
pauses of 4.0–6.0 seconds after their planning results, around delivery of
|
||
full trace artifacts. This desktop emulation does not establish reference-phone
|
||
performance or responsiveness.
|
||
|
||
```tsv
|
||
scenario repetition preparation_ms planning_ms trace_delivery_ms wall_ms outcome last_stage placements deterministic trace_status trace_candidates
|
||
representative year 1 0.0 443.7 624.7 1069.1 Plan dates 30 baseline ready 29451
|
||
representative year 2 0.1 490.0 609.3 1099.4 Plan dates 30 yes ready 29451
|
||
representative year 3 0.1 465.5 614.2 1079.8 Plan dates 30 yes ready 29451
|
||
typed obligations 1 0.0 157.0 261.2 418.2 Plan dates 10 baseline ready 13103
|
||
typed obligations 2 0.0 162.6 255.6 418.2 Plan dates 10 yes ready 13103
|
||
typed obligations 3 0.0 163.4 258.2 421.6 Plan dates 10 yes ready 13103
|
||
chained earned entitlements 1 0.0 45.3 1.0 47.0 Plan dates 3 baseline ready 27
|
||
chained earned entitlements 2 0.1 44.5 0.9 45.5 Plan dates 3 yes ready 27
|
||
chained earned entitlements 3 0.1 44.4 0.8 45.4 Plan dates 3 yes ready 27
|
||
deliberate conflict 1 0.0 92.8 0.9 93.8 PlanningConflict 0 baseline ready 366
|
||
deliberate conflict 2 0.0 44.7 0.2 44.9 PlanningConflict 0 yes ready 366
|
||
deliberate conflict 3 0.0 43.5 0.6 44.1 PlanningConflict 0 yes ready 366
|
||
loaded year 1 0.3 5901.5 5677.7 11581.0 Plan dates 144 baseline ready 865117
|
||
loaded year 2 0.8 6167.5 7582.4 13752.4 Plan dates 144 yes ready 865117
|
||
loaded year 3 0.6 5738.3 6219.5 11962.5 Plan dates 144 yes ready 865117
|
||
```
|