Skip to content

feat: write experiments.jsonl per run with action workflow run_id#45798

Closed
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/ensure-experiment-jsonl-includes-action-workflow-r
Closed

feat: write experiments.jsonl per run with action workflow run_id#45798
pelikhan with Copilot wants to merge 3 commits into
mainfrom
copilot/ensure-experiment-jsonl-includes-action-workflow-r

Conversation

Copilot AI commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Experiment state had no durable, run-correlated JSONL log. assignments.json has no run_id; state.json embeds a runs[] array but is primarily a counter store. Neither is convenient for correlating experiment assignments with a specific GitHub Actions run.

New file: experiments.jsonl

A growing JSONL file committed to the experiments git branch alongside state.json. Each line:

{"run_id":"12345678","timestamp":"2026-07-15T18:00:00.000Z","assignments":{"style":"detailed"}}

Pruned to the last 512 entries (matching MAX_RUN_HISTORY).

Changes

  • pick_experiment.cjs — after picking variants, appends a {run_id, timestamp, assignments} record to experiments.jsonl in the state dir; prunes to 512. runRecord is constructed once and shared with state.runs to avoid duplication.
  • load_experiment_state_from_repo.cjs — fetches experiments.jsonl from the experiments git branch alongside state.json so pick_experiment can append to the existing log (non-fatal on 404).
  • compiler_experiments.go — adds GH_AW_STATE_FILES: state.json,assignments.json,experiments.jsonl to the push_experiments_state job so all three files are committed to the branch.
  • pkg/constants/job_constants.go — adds ExperimentsRunsFilename = "experiments.jsonl" constant (mirrors EvalsResultFilename).
  • experiments-specification.md — adds experiments.jsonl to the definitions table; updates R-STORE-REPO-001/003; adds R-STORE-REPO-006 specifying write/prune behaviour.
  • pick_experiment.test.cjs — 4 new tests: initial write with correct run_id, no write when no assignments, append across successive runs, pruning at MAX_RUN_HISTORY.
  • *.lock.yml — all 256 workflows recompiled.

Copilot AI and others added 3 commits July 15, 2026 18:15
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
… JSONL entries

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
… load warning

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title feat: add experiments.jsonl with action workflow run_id to experiment JSONL entries feat: write experiments.jsonl per run with action workflow run_id Jul 15, 2026
Copilot AI requested a review from pelikhan July 15, 2026 18:33
@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Field Value
Category feature
Risk 🟡 Medium
Priority Score 72/100
Action fast_track

Score breakdown: Impact 35/50 · Urgency 20/30 · Quality 17/20

Rationale: Adds durable experiments.jsonl logging alongside state files. Wide blast radius (49 files, 256 workflow lock files regenerated). Well-described with 4 new tests. CI not yet run. Medium risk.

Generated by 🔧 PR Triage Agent · 36.7 AIC · ⌖ 7.39 AIC · ⊞ 5.6K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants