nessar/vac-optimizer
1
0 Fork 0
Code Issues 14 Pull Requests 3 Actions Packages Projects 1 Releases Wiki Activity

test(benchmark): report each scenario as it finishes #281

New Issue
Closed
opened 2026-10-16 22:39:45 +02:00 by nessar · 1 comment
nessar commented 2026-10-16 22:39:45 +02:00
Owner
Copy link
Copy source

Parent

#279

What to build

The benchmark protocol prints each scenario's report row as soon as that scenario's measured repetitions finish, instead of printing everything at the end. A CI benchmark job that times out still shows which scenarios completed and how long they took.

Acceptance criteria

  • Each scenario's rows appear in the job log when the scenario completes, in the existing tab-separated report format.
  • The complete report at the end is still available and unchanged in content.
  • Scenarios, inputs, warm-ups, repetitions, assertions, heap limit and timeout are unchanged.
  • A local run killed partway through shows the rows of the completed scenarios.

Blocked by

## Parent #279 ## What to build The benchmark protocol prints each scenario's report row as soon as that scenario's measured repetitions finish, instead of printing everything at the end. A CI benchmark job that times out still shows which scenarios completed and how long they took. ## Acceptance criteria - [ ] Each scenario's rows appear in the job log when the scenario completes, in the existing tab-separated report format. - [ ] The complete report at the end is still available and unchanged in content. - [ ] Scenarios, inputs, warm-ups, repetitions, assertions, heap limit and timeout are unchanged. - [ ] A local run killed partway through shows the rows of the completed scenarios. ## Blocked by - #278
nessar added the ready-for-agent label 2026-10-16 22:39:45 +02:00
nessar added a new dependency 2026-10-16 22:39:45 +02:00
#278 perf(planning): restore the integrated benchmark gate
nessar added a new dependency 2026-10-16 22:39:46 +02:00
#285 ci(benchmark): gate scenario speed relative to the runner
nessar commented 2026-10-17 02:19:55 +02:00
Author
Owner
Copy link
Copy source

Done in PR #287 (24c3554, then 3bbe989): the benchmark prints each scenario's row as soon as it finishes, now as an aligned Markdown summary table followed by the complete TSV report. A killed CI run (srvhome, run 31) kept the completed scenarios' rows in its log.

Done in PR #287 (24c3554, then 3bbe989): the benchmark prints each scenario's row as soon as it finishes, now as an aligned Markdown summary table followed by the complete TSV report. A killed CI run (srvhome, run 31) kept the completed scenarios' rows in its log.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
bug
dependencies
docker
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
javascript
needs-info
needs-triage
question
ready-for-agent
wontfix
No labels ready-for-agent
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate (renovate) Bot nessar (nessar)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Blocks
#285 ci(benchmark): gate scenario speed relative to the runner
nessar/vac-optimizer
Depends on
#278 perf(planning): restore the integrated benchmark gate
nessar/vac-optimizer
Reference: nessar/vac-optimizer#281