Prototype BEPv4 Agent lifecycle and providers - #4235
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…oviders-v4 # Conflicts: # baml_language/crates/baml_compiler2_tir/src/builder.rs # baml_language/crates/baml_compiler2_tir/src/inference.rs # baml_language/crates/baml_tests/tests/bytecode_format/snapshots/bytecode_format__bytecode_display_expanded.snap # baml_language/crates/baml_tests/tests/bytecode_format/snapshots/bytecode_format__bytecode_display_expanded_unoptimized.snap
⏭️ Performance benchmarks were skippedPerf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to To run them on this PR, do any of the following, then push a commit (or re-run CI):
|
![cursor[bot]](https://cdn.github.com/in/1210556?s=60&v=4){kind=small}
Binary size checks passed
Generated by |
| } | ||
| } | ||
|
|
||
| function task<T>( |
There was a problem hiding this comment.
These should probably be methods on Task itself:
Task.new and Task.named or something like that
| // The empty root capability. Specific operations require a narrower provider | ||
| // interface so a realtime channel cannot accidentally promise a final T. | ||
| interface Provider { | ||
| function name(self) -> string throws never { "provider" } |
There was a problem hiding this comment.
probably should not default
| } | ||
| } | ||
|
|
||
| interface GenerationProvider requires Provider { |
There was a problem hiding this comment.
What is the difference between CompletionProvider and GenerationProvider? Could probably be deduped
| (input * price.input_per_million + output * price.output_per_million) / 1000000.0 | ||
| } | ||
|
|
||
| function aggregate_usage(responses: ResponseMetadata[]) -> Usage throws never { |
There was a problem hiding this comment.
where is this used? how is it used? does impl of runners call this automatically for the ones we support?
| @@ -0,0 +1,42 @@ | |||
| // Provider-owned continuation state. Concrete implementations retain wire | |||
There was a problem hiding this comment.
messages stuff should probably go in its own namespace like ns_chat or something: it is an implementation part for multi-turn runners not a base part of the framework
…oviders-v4 # Conflicts: # baml_language/crates/baml_cli/src/grep_command.rs
…oviders-v4 # Conflicts: # baml_language/crates/baml_cli/src/check_command.rs # baml_language/crates/baml_cli/src/commands.rs # baml_language/crates/baml_cli/src/describe_command.rs # baml_language/crates/baml_cli/src/format.rs # baml_language/crates/baml_cli/src/generate.rs # baml_language/crates/baml_cli/src/pack_command.rs # baml_language/crates/baml_cli/src/run_command.rs # baml_language/crates/baml_cli/src/test_command.rs
![coderabbitai[bot]](https://cdn.github.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5143590. Configure here.
aaronvg
changed the title
3 participants
Summary
Task<T>+ai.run.Agent<T>+ anAgentProviderwithbegin<T>,step<T>, andsubmitbaml_src_temp2ai_scenariosinto zero-argument runnable examples with assertions intests/, scenario-local utilities, explicit custom runners, goal-oriented comments, and//#graph regionstask.run(...), concrete generic runner dispatch, interface implementations, and method callscanary, taking its newer CLI and type-system work while preserving explicit raw-source loading forbaml_src_temp2Why
The previous prototype allowed normal model work to be described through overlapping Completion, Generation, Provider, and Runner abstractions. That made ownership unclear and risked recursive
provider -> runner -> providerexecution.This design has one ordinary lifecycle:
TDone<T>,Handoff, orBudgetReachedStreaming, realtime sessions, background jobs, batch work, transcription, caches, and external harnesses remain explicit separate lifecycles because they do not obey the same one-step protocol.
Usage
Every public scenario is zero-argument and can be run directly:
The provider configuration and integration tests under
ai_scenarioscover OpenAI Responses only—there is no Chat Completions implementation.Diff size
Against the merged
canary: 28,961 additions and 1,033 deletions across 281 files. Nineteen binary snapshot files are excluded from the line totals.Swift realtime support is present after the canary merge, but canary now contains the same Swift changes, so there is no remaining Swift diff in this PR.
Validation
cargo fmt --all -- --checkcargo build -p baml_clicargo nextest run -p baml_cli -p sys_llm -p sys_ops --no-fail-fast— 989 passedbaml check --from crates/baml_tests/baml_src_temp2— 212 files checkedinfisical run --env=test -- ...— 25 passedai_scenarios.observe_an_agent— passedThe branch is merged with
origin/canaryatfb2527cd2.Note
High Risk
Large architectural shift in how model calls, tools, retries, and conversations behave, plus broad provider and compiler integration; regressions could affect auth, side-effect safety, and resumption semantics.
Overview
Introduces BEP-064 (BEPv4): a single normal lifecycle where
ai.run.AgentdrivesAgentProvider(begin/step/submit), replacing overlapping Completion/Generation/provider loops. Direct LLM calls lower to a default Agent and unwrapDone<T>; streaming, jobs, realtime, and harnesses stay separate lifecycles.Adds the
aistdlib surface inbaml_builtins2:Task<T>, outcomes (Done,BudgetReached,Handoff,Interrupted), conversations, openai.Failure+UnknownError, tool limits, prompt recipes, harness helpers, and internal Agent plumbing (tool dispatch, cancellation, output fingerprints, direct-call lowering).The executable corpus in
baml_src_temp2gains multi-vendorAgentProviderimplementations (OpenAI Responses, Anthropic, Google, Vertex, Claude Code), reorganizedai_scenarios, and provider request tests—grounded by new design docs under_plan/bepv4(guides, error model, namespace layout). Minor comment tweak indedent.rs;.baml/.gitignorenow ignores all files under.baml/.Reviewed by Cursor Bugbot for commit 43c8201. Bugbot is set up for automated code reviews on this repo. Configure here.