diff --git a/docs/benchmark/README.md b/docs/benchmark/README.md index 9939dcd..b6b5196 100644 --- a/docs/benchmark/README.md +++ b/docs/benchmark/README.md @@ -7,6 +7,9 @@ the benchmark fixtures. Before trace removal, the full functional coverage run p across 34 files under the 2 GB Node heap limit, excluding the separate benchmark. That run does not establish a passing performance protocol. +The post-removal functional integration checks, browser QA and unchanged-input +handoff for #78 are recorded in [the #77 verification report](../trace-free-planning-verification.md). + On 2026-10-15, the #76 worktree based on `ea42561` passed 630 functional tests across 33 files with coverage under the 2 GB Node heap limit. Complete outcome comparisons against that baseline matched for all seven benchmark scenarios. diff --git a/docs/trace-free-planning-verification.md b/docs/trace-free-planning-verification.md new file mode 100644 index 0000000..e4e5a55 --- /dev/null +++ b/docs/trace-free-planning-verification.md @@ -0,0 +1,151 @@ +# Integrated planning verification after trace removal + +Functional verification for [#77](https://gitea.nessar.fr/nessar/vac-optimizer/issues/77), +run on 2026-10-15 against `2f4af14869c47836d2c1008157136f75c9d3e98b` +(`develop`, including the merged #76 implementation). The original checkout was +clean at `d9d9befa1256cd2200aa4778d28b2bc23ba141e6`; both commits have the same +tree. This verification adds two real-worker lifecycle tests and this report. +It does not change production behavior, saved data, placement expectations, +benchmark inputs, CI settings or performance limits. + +## Contract audit + +| Path | Verified retained contract | +| --- | --- | +| Synchronous and cooperative core | Outcomes, valid checkpoints, final-state snapshots, stable explanations and explicit cancellation; no recorder, candidate history or historical replay API. | +| Server calculation context | Immutable submitted configuration, independently delivered outcome and explanation, and displayed-placement date analysis. | +| Browser without Worker | Validation and explicit `worker-unavailable` handling; no synchronous planning or trace compatibility result. | +| Native worker and coordinator | Heartbeat, progress, checkpoint, result, explanation and final artifacts. The coordinator terminates the worker after required delivery with no acknowledgement message or queue. | +| Calendar and remaining balances | Final and valid partial calendars, placement provenance, objective comparisons, earned sources, validity dates and concrete non-placement reasons. | +| Alternative dates | Captured horizon, entitlement, type and rule provenance; unrelated placements stay fixed. Hypothetical comparisons never update the displayed plan. | +| Benchmark callers | Supported calculation outcome, explanation and timings; no trace promises, waits, counts or replacement trace metrics. | + +Production-source searches found no decision-trace, historical analysis, +acknowledgement or replay implementation. Angular hydration's `withEventReplay` +is unrelated to planning history. `PlanningArtifacts` carries actual explanation, +final-state and planning-work timing data, rather than a compatibility shell. + +## Existing deterministic fixtures + +The full functional suite passes without changing any existing expected placement, +balance, metric, objective, chronology or completed-stage assertion. + +| Requirement | Existing regression coverage | +| --- | --- | +| Typed joint obligations and shared provenance | `leave-planner.service.spec.ts`, `leave-planner-explanation.spec.ts` and real-worker tests. | +| Earned-entitlement chains and availability | Integration, explanation, supporting-placement and real-worker tests, including declarations and cancellation. | +| Custom formulas and reordered priorities | Optimization, calendar-objective, work-sequence and supporting-placement tests, including real-worker comparisons. | +| Two- and three-year horizons | Service, calendar and worker tests retain the submitted dates and deterministic completed plans. | +| No active optimization rules | Optimization and migration tests retain declarations and remaining balances without adding placements. | +| Stable explanations and remaining reasons | Entitlements, every contributing rule, ordered objective values, first decisive objective, periods and dated non-placement reasons. | +| Cancellation and failures | Before and after a jointly valid incumbent, stale calculations and date analyses, worker liveness, technical budgets, and local explanation errors. | +| Saved settings | Existing migration and persistence tests preserve user rule choices, balances, declarations and explicitly selected horizons. | + +An additional direct comparison uses the pre-removal core from `ea42561`, with +only its relative imports redirected to the unchanged calendar/date/objective +modules. Both versions receive the same cloned requests and calculation date +`2024-01-01`. Full outcomes and stable explanations are compared with deep +equality; no placement expectations or retained explanation fields are filtered. + +All ten completed comparisons match for both the full outcome and stable +explanation: + +| Fixture | Outcome | Placements | Last completed stage | +| --- | --- | --- | --- | +| Representative year | `Plan` | 30 | `dates` | +| Typed joint obligations | `Plan` | 10 | `dates` | +| Chained earned entitlements | `Plan` | 3 | `dates` | +| Deliberate conflict | `PlanningConflict` | 0 | — | +| Two years | `Plan` | 60 | `dates` | +| Three years | `Plan` | 90 | `dates` | +| Holiday value before non-working-day gain | `Plan` | 3 | `dates` | +| Non-working-day gain before holiday value | `Plan` | 3 | `dates` | +| Custom maximum holiday-week aggregation | `Plan` | 3 | `dates` | +| No active optimization, with a declared day | `Plan` | 0 | `dates` | + +The reordered priorities retain their different placement dates: 6, 7 and 10 May +for holiday value first; 29 April, 10 May and 17 May for gain first. The zero-rule +case retains its declaration. These are correctness comparisons, not timed +benchmark repetitions. An exploratory pre-removal loaded-year replay was stopped +and is not used as passing evidence; its full protocol remains with #78. + +## Correctness and static checks + +Environment: Node `24.18.0`, Bun `1.3.14`, Linux under WSL2. + +```sh +NODE_OPTIONS=--max-old-space-size=2048 bunx vitest run --coverage --maxWorkers=1 --exclude='src/app/benchmark/**' +bunx tsc -p tsconfig.app.json --noEmit +bunx tsc -p tsconfig.spec.json --noEmit +bun run lint +bun run format:check +bunx prettier --check 'scripts/ci-*.mjs' '.gitea/workflows/*.yml' +node scripts/ci-policy.test.mjs +node scripts/ci-image.test.mjs +bun run build +git diff --check +``` + +All checks pass. Functional coverage: **632 tests across 33 files**, 93.56% +statements, 85.90% branches, 95.69% functions and 95.41% lines. The full coverage +run took 95.67 seconds with the existing 2 GB Node heap setting. The focused +real-worker file passes all 15 tests, including the two added lifecycle checks. +Coverage configuration and thresholds are unchanged. + +The production build retains its existing warning budgets: initial bundle +2.57 MB against the 2.50 MB warning and calendar-year styles 12.98 kB against +the 8 kB warning. Both remain below their unchanged error limits. + +## Browser QA + +The production build was served locally with +`NG_ALLOWED_HOSTS=localhost,127.0.0.1 PORT=4177` and exercised in headless +Chromium `149.0.7827.55`, using native module Workers, reduced motion and +375×915, 768×915 and 1440-pixel-wide viewports. The calculation date was fixed +to `2024-01-01`. This is desktop functional evidence, not a phone performance run. + +| Flow | Result | +| --- | --- | +| Completed calendar and explanation | Three CP placements on 6, 7 and 10 May 2024, six gained non-working dates; keyboard disclosure, placement navigation and calendar selection work. | +| Remaining balances | Future RTT availability and unavoidable expiry show their actual dates, reasons and planning horizon without attempt/history controls. Earned-source rendering is also covered by component tests. | +| Date comparison | An alternative on 13 May is hypothetical; placements and selected calendar dates remain unchanged. Escape closes the dialog and restores focus to Compare dates. | +| Immutable settings | Persisting a current CP balance edit from 3 to 99 leaves the displayed three placements unchanged. Select another placement and verify that a new comparison request is sent after the edit; that request still contains the captured quantity 3 and original horizon. | +| Cancellation before a valid plan | At 375 px, hold the native worker script download, activate Cancel with Enter, then release startup. Result is `PlanningIncomplete` / `user-cancelled`, without an incumbent or a false conflict. This controlled startup is not latency evidence. | +| Cancellation after a valid plan | At 375 px, use the unchanged loaded-year request and cancel with Enter after a real checkpoint. All three typed obligation windows remain satisfied, placements plus remaining balances total 72, and the partial explanation is available. Inputs remain editable during planning; Cancel is at least 44 px high. | +| Delayed and stale delivery | At 375 px, hold explanation and artifact messages after a real outcome. The three-placement calendar is usable immediately. Recalculate, then release the old messages; the current result and explanation remain intact. | +| Explanation failure | At 375 px, inject explanation transport and checkpoint-explanation recovery failures. An inline error appears while the completed calendar and all three placements remain available. | +| Responsive and keyboard interaction | No page-level horizontal overflow at 375, 768 or 1440 px. Comparison dialogs fit, retain keyboard focus and restore it on dismissal. No unexpected browser page errors. | +| Removed UI | No advanced trace explorer, historical overlays, replay controls, candidate-status UI or remaining-balance attempt controls. | + +Controlled-clock transport tests additionally verify that heartbeats renew the +30-second inactivity watchdog across repeated windows and that monitoring ends +at outcome delivery. Native-worker tests verify actual heartbeat delivery, +incumbent validity and retained explanations. No automatic engine cutoff is added. + +## Handoff to #78 + +The #70 → #71 → #72 → #73 → #74 → #75 dependency order is preserved. The tracker +already reports those six tickets and #76 closed. This work verifies their +functional integration and does not close or modify any other ticket. + +The benchmark source and its original correctness assertions remain unchanged +from the integrated baseline: + +| File | SHA-256 | +| --- | --- | +| `src/app/benchmark/leave-planner-benchmark.ts` | `16e057fa042c898d203f71e018f183c9fbecb5cd0edbf0ce0f3f25277cb00d3a` | +| `src/app/benchmark/leave-planner-benchmark.spec.ts` | `c7b4c37b6c169b42c4d18dfc3e650b25a2ba47415c97268dafefafbfb0ee3023` | + +All seven scenarios, typed and earned-chain fixtures, two-/three-year horizons, +60-/90-placement expectations, determinism comparisons and final `dates` stage +assertions remain in place. Warm-ups and measured repetitions remain unchanged. +The full 2 GB / 300-second benchmark protocol and any required optimization stay +open in [#78](https://gitea.nessar.fr/nessar/vac-optimizer/issues/78); they are not +a completion gate for #75 or #77. This run makes no performance acceptance or +trace-removal speedup claim. The separate advisory benchmark status must be +resolved by #78 under the existing limits. + +Local run logs, the original spec and checkout record, comparison scripts and +browser screenshots are in `/tmp/vac-issue-77`. The source fixtures and commands +above identify the reproducible correctness checks; temporary QA controls are +not part of the application. diff --git a/src/app/services/leave-planner.worker.spec.ts b/src/app/services/leave-planner.worker.spec.ts index 71fa2fc..f07c219 100644 --- a/src/app/services/leave-planner.worker.spec.ts +++ b/src/app/services/leave-planner.worker.spec.ts @@ -33,6 +33,9 @@ describe('real planning worker cancellation', () => { let simulateLongCalculation = false; let simulatedElapsed = 0; const threads = new Set(); + const sentTypes: string[] = []; + const receivedTypes: string[] = []; + let terminationCount = 0; beforeAll(async () => { directory = mkdtempSync(join(tmpdir(), 'vac-optimizer-worker-')); @@ -65,6 +68,9 @@ parentPort.on('message', (data) => globalThis.self.onmessage({ data })); heartbeats = 0; simulateLongCalculation = false; simulatedElapsed = 0; + sentTypes.length = 0; + receivedTypes.length = 0; + terminationCount = 0; class RealWorker { onmessage: ((event: MessageEvent) => void) | null = null; onerror: (() => void) | null = null; @@ -76,6 +82,7 @@ parentPort.on('message', (data) => globalThis.self.onmessage({ data })); simulatedElapsed = data.elapsed; return; } + receivedTypes.push(data.type); if (holdArtifacts && ['explanation', 'artifacts'].includes(data.type)) { heldArtifactMessages.push(() => this.onmessage?.({ data } as MessageEvent)); return; @@ -86,10 +93,12 @@ parentPort.on('message', (data) => globalThis.self.onmessage({ data })); }); this.thread.on('error', () => this.onerror?.()); } - postMessage(message: unknown) { + postMessage(message: { type: string }) { + sentTypes.push(message.type); this.thread.postMessage(message); } terminate() { + terminationCount++; void this.thread.terminate(); } } @@ -104,6 +113,59 @@ parentPort.on('message', (data) => globalThis.self.onmessage({ data })); }); afterAll(() => rmSync(directory, { recursive: true, force: true })); + it('delivers the outcome and explanation and releases the worker without acknowledgements', async () => { + const calculation = createPlanningCalculation({ + horizon: { start: '2024-01-02', end: '2024-01-04' }, + balances: [{ id: 'cp', type: 'cp', quantity: 1, expiryDate: '2024-01-03' }], + rules: [presets[0]], + }); + const outcome = await calculation.outcome; + expect(outcome).toMatchObject({ + kind: 'Plan', + placements: [{ date: '2024-01-02', balanceId: 'cp', type: 'cp' }], + lastCompletedStage: 'dates', + }); + expect(await calculation.explanation).toMatchObject({ + kind: 'PlanExplanation', + placements: [{ placement: { date: '2024-01-02', balanceId: 'cp' } }], + }); + expect(await calculation.explanationError).toBeUndefined(); + expect(await calculation.timings).toEqual({ planningWorkMs: expect.any(Number) }); + expect(sentTypes).toEqual(['plan']); + expect(receivedTypes.slice(-3)).toEqual(['result', 'explanation', 'artifacts']); + expect(new Set(receivedTypes)).toEqual( + new Set(['heartbeat', 'progress', 'checkpoint', 'result', 'explanation', 'artifacts']), + ); + expect(terminationCount).toBe(1); + }); + + it('ignores a replaced real worker’s late explanation without losing either outcome', async () => { + const request = { + horizon: { start: '2024-01-02', end: '2024-01-04' }, + balances: [{ id: 'cp', type: 'cp', quantity: 1, expiryDate: '2024-01-03' }], + rules: [presets[0]], + }; + holdArtifacts = true; + const first = createPlanningCalculation(request); + const original = await first.outcome; + expect(original.kind).toBe('Plan'); + await vi.waitFor(() => expect(heldArtifactMessages).toHaveLength(2)); + holdArtifacts = false; + const latest = createPlanningCalculation({ ...request, rules: [] }); + const outcome = await latest.outcome; + expect(outcome).toMatchObject({ kind: 'Plan', placements: [] }); + const explanation = await latest.explanation; + expect(explanation).toMatchObject({ kind: 'PlanExplanation', placements: [] }); + await latest.timings; + for (const deliver of heldArtifactMessages) deliver(); + expect(await first.outcome).toEqual(original); + expect(await first.explanation).toBeUndefined(); + expect(await latest.outcome).toEqual(outcome); + expect(await latest.explanation).toEqual(explanation); + expect(await latest.explanationError).toBeUndefined(); + expect(terminationCount).toBe(2); + }); + it('preserves a complete optimization configuration and active order in the real worker snapshot', async () => { const configuration: OptimizationConfiguration = structuredClone( OPTIMIZATION_PRESETS.unplacedAtRisk,