test(benchmark): keep the a871f8f engine as a fixed calibration engine

The differential check's frozen bundle will be rebuilt for #302; the
relative gate keeps timing the unchanged a871f8f bundle so its ratios
stay comparable with the recorded history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
nessarandClaude Opus 5.5 committed 2026-10-18 19:14:28 +02:00
1 parent ba05fe88b3
commit b356117263
4 files changed
+3251 -5

No files matched your search

+5 -3
View File
@@ -18,9 +18,11 @@ The relative gate replaces the 300-second acceptance protocol named in
policy. This is an explicit policy change from #279/#285: runners differ in
speed, so a fixed wall-clock limit was flaky.
Before the protocol, in the same process, the spec times the frozen engine
(`src/app/services/frozen-engine/leave-planner.core.frozen.mjs`, the engine at
a871f8f) on the `two years` scenario, best of three, and prints the result.
Before the protocol, in the same process, the spec times the calibration engine
(`src/app/services/frozen-engine/leave-planner.core.calibration.mjs`, the engine at
a871f8f) on the `two years` scenario, best of three, and prints the result. This
bundle never changes, so ratios stay comparable with the recorded history; the
differential check's frozen bundle can be rebuilt independently.
Each scenario's budget is its multiplier times that calibration; the mean
`planningMs` of its two measured repetitions must not exceed it. A scenario
without a multiplier fails too. `#286` tightens the multipliers once the speed
@@ -1,5 +1,5 @@
import { vi } from '../../testing/vitest-compat';
import { calculatePlanningAsyncInternal } from '../services/frozen-engine/leave-planner.core.frozen.mjs';
import { calculatePlanningAsyncInternal } from '../services/frozen-engine/leave-planner.core.calibration.mjs';
import { createPlanningCalculation } from '../services/leave-planner.service';
import {
BENCHMARK_BUDGET_MULTIPLIERS,
@@ -69,7 +69,7 @@ describe('leave planner benchmark protocol', () => {
),
).toHaveLength(2);
}
// Frozen engine (a871f8f) on 'two years', best of three: the runner-speed yardstick.
// Calibration engine (frozen at a871f8f) on 'two years', best of three: the runner-speed yardstick.
const calibrationRequest = scenarios.find((item) => item.name === 'two years')!.request;
let calibrationMs = Infinity;
for (let run = 0; run < 3; run++) {
@@ -0,0 +1,22 @@
import type {
PlanningArtifactsCallback,
PlanningCheckpoint,
PlanningExplanationCallback,
PlanningOutcome,
PlanningOutcomeCallback,
} from '../leave-planner.core';
export function calculatePlanningInternal(
request: unknown,
checkpointCallback?: undefined,
artifactsCallback?: PlanningArtifactsCallback,
): PlanningOutcome;
export function calculatePlanningAsyncInternal(
request: unknown,
checkpointCallback?: PlanningCheckpoint,
artifactsCallback?: PlanningArtifactsCallback,
outcomeCallback?: PlanningOutcomeCallback,
explanationCallback?: PlanningExplanationCallback,
isCancelled?: () => boolean,
): Promise<PlanningOutcome>;
File diff suppressed because it is too large. Load diff