Closed
feat: write experiments.jsonl per run with action workflow run_id#45798
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
… JSONL entries Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
… load warning Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
feat: add experiments.jsonl with action workflow run_id to experiment JSONL entries
feat: write experiments.jsonl per run with action workflow run_id
Jul 15, 2026
Copilot created this pull request from a session on behalf of
pelikhan
July 15, 2026 18:33
View session
Contributor
🤖 PR Triage
Score breakdown: Impact 35/50 · Urgency 20/30 · Quality 17/20 Rationale: Adds durable
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Experiment state had no durable, run-correlated JSONL log.
assignments.jsonhas norun_id;state.jsonembeds aruns[]array but is primarily a counter store. Neither is convenient for correlating experiment assignments with a specific GitHub Actions run.New file:
experiments.jsonlA growing JSONL file committed to the experiments git branch alongside
state.json. Each line:{"run_id":"12345678","timestamp":"2026-07-15T18:00:00.000Z","assignments":{"style":"detailed"}}Pruned to the last 512 entries (matching
MAX_RUN_HISTORY).Changes
pick_experiment.cjs— after picking variants, appends a{run_id, timestamp, assignments}record toexperiments.jsonlin the state dir; prunes to 512.runRecordis constructed once and shared withstate.runsto avoid duplication.load_experiment_state_from_repo.cjs— fetchesexperiments.jsonlfrom the experiments git branch alongsidestate.jsonso pick_experiment can append to the existing log (non-fatal on 404).compiler_experiments.go— addsGH_AW_STATE_FILES: state.json,assignments.json,experiments.jsonlto thepush_experiments_statejob so all three files are committed to the branch.pkg/constants/job_constants.go— addsExperimentsRunsFilename = "experiments.jsonl"constant (mirrorsEvalsResultFilename).experiments-specification.md— addsexperiments.jsonlto the definitions table; updates R-STORE-REPO-001/003; adds R-STORE-REPO-006 specifying write/prune behaviour.pick_experiment.test.cjs— 4 new tests: initial write with correctrun_id, no write when no assignments, append across successive runs, pruning atMAX_RUN_HISTORY.*.lock.yml— all 256 workflows recompiled.