Update GPT pricing catalog and default detection/evals model routing via aliases by Copilot · Pull Request #49586 · github/gh-aw

Copilot · 2026-08-01T13:48:26Z

This PR updates GPT pricing entries in the GitHub/OpenAI model catalog and introduces explicit detection / evals aliases for small-model routing. Detection and evals now default to these aliases when no model is configured, instead of relying on implicit fallback behavior.

Pricing catalog updates (GitHub/OpenAI)
- Refreshed GPT pricing values in models.json for both runtime copies:
  - pkg/cli/data/models.json
  - actions/setup/js/models.json
- Updated GPT-5.6 cache-read rates and added missing OpenAI GPT-5.6 entries in the OpenAI provider section so pricing is consistent across providers/catalog consumers.
New built-in model aliases
- Added alias mappings in pkg/workflow/data/model_aliases.json:
  - detection: ["small"]
  - evals: ["small"]
- Keeps job-level model intent explicit while still resolving through existing small-model policy.
Default model behavior changes
- Threat detection now defaults to "detection" when no model is set (both inline and external detection paths).
- Evals now defaults to "evals" when no model is set (replacing the hardcoded "small" fallback).
- Enterprise overrides (GH_AW_DEFAULT_DETECTION_MODEL, GH_AW_DEFAULT_EVALS_MODEL) still take precedence.
Compiler fixture/test alignment
- Updated alias/default expectations and wasm golden outputs to match the new default resolution path.

# before (implicit / hardcoded small fallback)
evals:
  questions:
    - id: builds
      question: Does it build?

safe-outputs:
  threat-detection: true

# after (same config, explicit alias-backed defaults)
# evals model resolves to: "evals" -> "small"
# detection model resolves to: "detection" -> "small"

✨ PR Review Safe Output Test - Run 30715626945

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · sonnet46 · 103.7 AIC · ⌖ 10 AIC · ⊞ 6.3K · ◷
_{Comment /smoke-claude to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-08-01T18:40:16Z

PR Triage

Category: chore
Risk: medium
Priority: low
Score: 43/100 (Impact25/Urgency10/Quality8)
Recommended action: defer

Draft; pricing catalog + model routing alias changes, moderate behavioral impact, still draft.

Generated by 🔧 PR Triage Agent · auto · 73.4 AIC · ⌖ 10.1 AIC · ⊞ 8K · ◷

Copilot

Pull request overview

Updates model pricing and routes default detection/evaluation workloads through dedicated small-model aliases.

Changes:

Refreshes GPT-5.6 pricing in both catalogs.
Adds detection and evals aliases backed by small.
Updates compiler defaults, tests, and generated fixtures.

Show a summary per file

File	Description
`pkg/workflow/threat_detection_test.go`	Updates inline detection expectations.
`pkg/workflow/threat_detection_inline_engine.go`	Adds the `detection` fallback.
`pkg/workflow/threat_detection_external.go`	Adds the external-detector fallback.
`pkg/workflow/evals_steps.go`	Defaults evaluations to `evals`.
`pkg/workflow/evals_config.go`	Documents the new default.
`pkg/workflow/data/model_aliases.json`	Defines detection and evals aliases.
`pkg/workflow/model_aliases_test.go`	Verifies the aliases.
`pkg/cli/data/models.json`	Updates the CLI pricing catalog.
`actions/setup/js/models.json`	Updates the setup-action pricing catalog.
`pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden`	Refreshes generated version metadata.
`pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden`	Refreshes generated version metadata.
`pkg/workflow/testdata/TestWasmGolden_CompileFixtures/playwright-cli-mode.golden`	Refreshes generated version metadata.
`pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden`	Refreshes generated version metadata.
`pkg/workflow/testdata/TestWasmGolden_AllEngines/copilot.golden`	Refreshes generated version metadata.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 14/14 changed files
Comments generated: 3
Review effort level: Balanced

 	// Model is the default LLM model to use for evaluations. Use a model alias such as
 	// "small" or a full model ID. Per-question Model fields override this value.
-	// When empty, the compiler default ("small") is used.
+	// When empty, the compiler default ("evals") is used.


 	if model == "" {
-		model = "small"
+		model = "evals"
 	}


+	if resolvedDetectionModel == "" {
+		resolvedDetectionModel = "detection"
+	}


pelikhan · 2026-08-01T19:34:26Z

@copilot merge main and recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

…pricing-table Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-08-01T19:47:36Z

@copilot merge main and recompile

Done — merged main (f84e968) and recompiled all 266 workflows successfully (266 succeeded, 0 errors).

github-actions · 2026-08-01T19:51:41Z

✅ Smoke Copilot Auto completed successfully!

github-actions · 2026-08-01T19:51:46Z

🚀 Smoke Gemini MISSION COMPLETE! Gemini has spoken. ✨

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

Details

The threat detection results could not be parsed.

Review the workflow run logs for details.

github-actions · 2026-08-01T19:51:50Z

🚀 Smoke Pi MISSION COMPLETE! Pi delivered. 🥧

No PR comment/label action taken: workflow lacks pull_request triggering context; issue created with smoke test results.

github-actions · 2026-08-01T19:51:52Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

Details

The threat detection results could not be parsed.

Review the workflow run logs for details.

github-actions · 2026-08-01T19:52:14Z

✅ All tools validated successfully! Agent Container Smoke Test confirms agent container is ready.

github-actions · 2026-08-01T19:52:28Z

📰 BREAKING: Smoke Copilot - AOAI (Entra) is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-08-01T19:52:31Z

✅ Smoke Copilot Small completed successfully!

github-actions · 2026-08-01T19:52:31Z

✅ Smoke OTEL completed successfully!

github-actions · 2026-08-01T19:52:45Z

📰 BREAKING: Smoke Copilot - AOAI (apikey) is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-08-01T19:52:50Z

✅ Smoke Copilot MAI completed successfully!

github-actions · 2026-08-01T19:53:09Z

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-08-01T19:55:15Z

Small names, clear routes now—
detection, evals aliased,
pricing tuned in sync.

Inspired by the PR's new detection/evals aliases replacing hardcoded fallbacks, alongside refreshed GPT pricing data.

Generated by 🌸 Smoke Copilot Auto for #49586 · auto · 10.5 AIC · ⌖ 4.6 AIC · ⊞ 5.9K · ◷
_{Add label smoke to run again}

github-actions · 2026-08-01T19:55:17Z

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

Details

The threat detection results could not be parsed.

Review the workflow run logs for details.

Smoke Test Results

GitHub MCP Testing: ✅
Web Fetch Testing: ✅
File Writing Testing: ✅
Bash Tool Testing: ✅
Build gh-aw: ❌

Overall Status: FAIL

✨ Smoke Gemini — Powered by Gemini · agent · 11.1 AIC · ⊞ 9.4K · ◷
_{Comment /smoke-gemini to run again}

github-actions · 2026-08-01T19:55:44Z

Agent Container Tool Check

Tool	Status	Version
bash	✅	5.2.21
sh	✅	available
git	✅	2.54.0
jq	✅	1.7
yq	✅	4.53.3
curl	✅	8.5.0
gh	✅	2.96.0
node	✅	24.18.0
python3	✅	3.11.15
go	✅	1.24.13
java	✅	21.0.11 (Temurin)
dotnet	✅	10.0.302

Result: 12/12 tools available ✅

Overall status: PASS

🔧 Tool validation by Agent Container Smoke Test · auto · 17.2 AIC · ⌖ 3.58 AIC · ⊞ 8.2K · ◷
_{Comment /smoke-test-tools to run again}

github-actions · 2026-08-01T19:56:23Z

Smoke test summary:

Overall status: PASS
25 commits in the last 24h; themes: workflow/safety fixes, docs/deps, lint/testing improvements.
Run: https://github.com/github/gh-aw/actions/runs/30715680941

Generated by ⚡ Smoke Copilot MAI · mai10 · 3.41 AIC · ⌖ 1.4 AIC · ⊞ 19.7K · ◷
_{Comment /smoke-copilot-mai to run again}
_{Add label smoke to run again}

github-actions · 2026-08-01T19:56:44Z

Commit pushed: 7b78dd2

Generated by Changeset Generator · gpt54 · 5.35 AIC · ⌖ 14.1 AIC · ⊞ 12.3K

github-actions · 2026-08-01T20:00:35Z

Warning

threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

Details

The threat detection results could not be parsed.

Review the workflow run logs for details.

Comment Memory

Quiet builds hum on
GitHub lights flicker through dusk
Smoke test keeps its watch

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex · gpt54 · 16.3 AIC · ⊞ 11.8K · ◷
_{Comment /smoke-codex to run again}

github-actions · 2026-08-01T20:04:24Z

Smoke Test Copilot - Results

PR: Update GPT pricing catalog and default detection/evals model routing via aliases

github(gh-proxy) merged PRs: ✅
mcpscripts pr query: ✅
Serena find_symbol: ❌ (timeout)
Playwright: ✅
web-fetch: ✅
file+bash: ✅
discussion query+comment: ✅
make build: ✅
artifact upload: ✅
create_discussion: ✅
dispatch_workflow: ✅
PR review tools: ✅
comment memory: ✅
file-summarizer: ❌
check_run: ✅
LSP TS count: ✅

Overall: FAIL

cc @pelikhan @Copilot

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · auto · 42.8 AIC · ⌖ 3.66 AIC · ⊞ 8.7K · ◷
_{Comment /smoke-copilot to run again}
_{Add label smoke to run again}

github-actions · 2026-08-01T20:04:27Z

Comment Memory

Copilot testing
autumn code review whispers
green checks bloom in light

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · auto · 42.8 AIC · ⌖ 3.66 AIC · ⊞ 8.7K · ◷
_{Comment /smoke-copilot to run again}
_{Add label smoke to run again}

github-actions

Smoke test automated review (test).

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · auto · 42.8 AIC · ⌖ 3.66 AIC · ⊞ 8.7K
_{Comment /smoke-copilot to run again}
_{Add label smoke to run again}

github-actions

💥 Automated smoke test review - all systems nominal! Run 30715626945

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

accounts.google.com
android.clients.google.com
clients2.google.com
contentautofill.googleapis.com
safebrowsingohttpgateway.googleapis.com
www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

💥 [THE END] — Illustrated by Smoke Claude · sonnet46 · 103.7 AIC · ⌖ 10 AIC · ⊞ 6.3K
_{Comment /smoke-claude to run again}

Comments that could not be inline-anchored

.changeset/patch-update-model-pricing-and-default-routing.md:5

📝 Smoke Test Review Comment #1 — Run 30715626945: The changeset description clearly captures the intent of this PR. Consider also noting the compiler fixture updates in the summary for full traceability.

.github/workflows/ab-testing-advisor.lock.yml:840

🔍 Smoke Test Review Comment #2 — Run 30715626945: The model routing update here looks consistent with the new alias-based approach. Verify that all lock files have been recompiled consistently across all workflow variants.

github-actions · 2026-08-01T20:05:31Z

Smoke Test 30715680366: FAIL
✅ GitHub ❌ mcpscripts ❌ Serena ❌ Playwright ✅ Web-fetch ✅ File ✅ Azure ❌ Discussion ✅ Build ❌ Artifact ❌ Disc Create ❌ Dispatch ❌ PR Review ❌ Memory ❌ Sub-agent ❌ Check run

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

dc.services.visualstudio.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "dc.services.visualstudio.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot - AOAI (Entra) · o40mini · 25.4 AIC · ⌖ 3.72 AIC · ⊞ 18.8K · ◷
_{Comment /smoke-copilot-aoai-entra to run again}
_{Add label smoke to run again}

github-actions · 2026-08-03T05:28:31Z

🎉 This pull request is included in a new release.

Release: v0.84.3

Update GPT pricing and add detection/evals model aliases

536e6a3

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI assigned Copilot and pelikhan Aug 1, 2026

Copilot AI changed the title ~~Update GPT pricing and add detection/evals model defaults~~ Update GPT pricing catalog and default detection/evals model routing via aliases Aug 1, 2026

Copilot created this pull request from a session on behalf of pelikhan August 1, 2026 13:48 View session

Copilot finished work on behalf of pelikhan August 1, 2026 13:48

Copilot AI requested a review from pelikhan August 1, 2026 13:48

github-actions Bot added pr-action:defer pr-agent:github-copilot pr-priority:low pr-risk:medium pr-type:chore labels Aug 1, 2026

pelikhan marked this pull request as ready for review August 1, 2026 19:24

Copilot AI review requested due to automatic review settings August 1, 2026 19:24

Copilot started reviewing on behalf of pelikhan August 1, 2026 19:24 View session

Copilot AI reviewed Aug 1, 2026

View reviewed changes

Copilot started work on behalf of pelikhan August 1, 2026 19:35 View session

Copilot AI and others added 2 commits August 1, 2026 19:40

chore: merge main and recompile workflows

2b7bd52

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Merge remote-tracking branch 'origin/main' into copilot/update-model-…

f84e968

…pricing-table Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan August 1, 2026 19:47

pelikhan added the smoke label Aug 1, 2026

github-actions Bot temporarily deployed to aoai-model August 1, 2026 19:51 Inactive

github-actions Bot removed the smoke label Aug 1, 2026

github-actions Bot temporarily deployed to aoai-model August 1, 2026 19:52 Inactive

Add changeset

7b78dd2

github-actions Bot added the needs-review label Aug 1, 2026

This was referenced Aug 1, 2026

[aw] Smoke Antigravity produced no safe outputs #49621

Closed

[aw] Changeset Generator is missing required data #49622

Closed

pelikhan merged commit 14122b3 into main Aug 1, 2026

pelikhan deleted the copilot/update-model-pricing-table branch August 1, 2026 20:02

github-actions Bot reviewed Aug 1, 2026

View reviewed changes

github-actions Bot mentioned this pull request Aug 1, 2026

Smoke Test: Claude - 30715626945 #49627

Closed

github-actions Bot reviewed Aug 1, 2026

View reviewed changes

github-actions Bot added the smoke-copilot-aoai-entra label Aug 1, 2026

github-actions Bot mentioned this pull request Aug 1, 2026

Smoke Test: MAI-Code-1-Flash - 30716154729 #49638

Closed

github-actions Bot mentioned this pull request Aug 4, 2026

[gh-aw-parity] gh-aw threat-detection changes to review - 2026-08-04 github/gh-aw-threat-detection#732

Open

Uh oh!

Conversation

Copilot AI commented Aug 1, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026

PR Triage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

pelikhan commented Aug 1, 2026

Uh oh!

Copilot AI commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026

Smoke Test Results

Uh oh!

github-actions Bot commented Aug 1, 2026

Agent Container Tool Check

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 1, 2026

Comment Memory

Uh oh!

github-actions Bot commented Aug 1, 2026

Smoke Test Copilot - Results

Uh oh!

github-actions Bot commented Aug 1, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Comments that could not be inline-anchored

Uh oh!

github-actions Bot commented Aug 1, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

Reviewers

Copilot AI commented Aug 1, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading

github-actions Bot commented Aug 1, 2026 •

edited

Loading