test(benchmark): keep the a871f8f engine as a fixed calibration engine
The differential check's frozen bundle will be rebuilt for #302; the
relative gate keeps timing the unchanged a871f8f bundle so its ratios
stay comparable with the recorded history.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
ba05fe88b3
commit
b356117263
4 files changed
+3251
-5
No files matched your search
@@ -18,9 +18,11 @@ The relative gate replaces the 300-second acceptance protocol named in
|
||||
policy. This is an explicit policy change from #279/#285: runners differ in
|
||||
speed, so a fixed wall-clock limit was flaky.
|
||||
|
||||
Before the protocol, in the same process, the spec times the frozen engine
|
||||
(`src/app/services/frozen-engine/leave-planner.core.frozen.mjs`, the engine at
|
||||
a871f8f) on the `two years` scenario, best of three, and prints the result.
|
||||
Before the protocol, in the same process, the spec times the calibration engine
|
||||
(`src/app/services/frozen-engine/leave-planner.core.calibration.mjs`, the engine at
|
||||
a871f8f) on the `two years` scenario, best of three, and prints the result. This
|
||||
bundle never changes, so ratios stay comparable with the recorded history; the
|
||||
differential check's frozen bundle can be rebuilt independently.
|
||||
Each scenario's budget is its multiplier times that calibration; the mean
|
||||
`planningMs` of its two measured repetitions must not exceed it. A scenario
|
||||
without a multiplier fails too. `#286` tightens the multipliers once the speed
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { vi } from '../../testing/vitest-compat';
|
||||
import { calculatePlanningAsyncInternal } from '../services/frozen-engine/leave-planner.core.frozen.mjs';
|
||||
import { calculatePlanningAsyncInternal } from '../services/frozen-engine/leave-planner.core.calibration.mjs';
|
||||
import { createPlanningCalculation } from '../services/leave-planner.service';
|
||||
import {
|
||||
BENCHMARK_BUDGET_MULTIPLIERS,
|
||||
@@ -69,7 +69,7 @@ describe('leave planner benchmark protocol', () => {
|
||||
),
|
||||
).toHaveLength(2);
|
||||
}
|
||||
// Frozen engine (a871f8f) on 'two years', best of three: the runner-speed yardstick.
|
||||
// Calibration engine (frozen at a871f8f) on 'two years', best of three: the runner-speed yardstick.
|
||||
const calibrationRequest = scenarios.find((item) => item.name === 'two years')!.request;
|
||||
let calibrationMs = Infinity;
|
||||
for (let run = 0; run < 3; run++) {
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import type {
|
||||
PlanningArtifactsCallback,
|
||||
PlanningCheckpoint,
|
||||
PlanningExplanationCallback,
|
||||
PlanningOutcome,
|
||||
PlanningOutcomeCallback,
|
||||
} from '../leave-planner.core';
|
||||
|
||||
export function calculatePlanningInternal(
|
||||
request: unknown,
|
||||
checkpointCallback?: undefined,
|
||||
artifactsCallback?: PlanningArtifactsCallback,
|
||||
): PlanningOutcome;
|
||||
|
||||
export function calculatePlanningAsyncInternal(
|
||||
request: unknown,
|
||||
checkpointCallback?: PlanningCheckpoint,
|
||||
artifactsCallback?: PlanningArtifactsCallback,
|
||||
outcomeCallback?: PlanningOutcomeCallback,
|
||||
explanationCallback?: PlanningExplanationCallback,
|
||||
isCancelled?: () => boolean,
|
||||
): Promise<PlanningOutcome>;
|
||||
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user