Skip to content

test(mcp): diagnose startup stalls (do not merge)#41947

Draft
Skn0tt wants to merge 3 commits into
microsoft:mainfrom
Skn0tt:skn0tt-diagnose-mcp-startup-stalls
Draft

test(mcp): diagnose startup stalls (do not merge)#41947
Skn0tt wants to merge 3 commits into
microsoft:mainfrom
Skn0tt:skn0tt-diagnose-mcp-startup-stalls

Conversation

@Skn0tt

@Skn0tt Skn0tt commented Jul 23, 2026

Copy link
Copy Markdown
Member

Caution

Temporary diagnostic PR. Do not merge as-is.

This adds a PWTEST_MCP_STARTUP_TRACE-gated JSONL timeline around MCP CLI, daemon, browser, registry and dashboard startup. The test fixture keeps each timeline outside the per-test output directory, records PID liveness during cleanup and uploads the raw files even when a test times out.

The MCP workflow temporarily runs the full serial suite on 10 fresh Ubuntu Chrome, Windows Chrome and Windows Edge runners each. Unrelated PR workflows are skipped for this branch so only the diagnostic matrix consumes CI.

Copilot-Session: b62093c2-7f1f-46d8-9f9a-be7e30a965ab
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Skn0tt and others added 2 commits July 23, 2026 16:24
Split Chromium connection and navigation startup into individual diagnostic
phases so the next sampled CI run can identify the exact stalled operation.
Limit matrix concurrency to reduce hosted-runner fleet pressure.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b62093c2-7f1f-46d8-9f9a-be7e30a965ab
Stop traced Ubuntu jobs before the hosted VM loses communication, and retain
resource telemetry alongside the partial startup timelines.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b62093c2-7f1f-46d8-9f9a-be7e30a965ab
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP startup diagnostic (temporary)"

1 failed
❌ [msedge] › mcp/cli-core.spec.ts:140 › uncheck @mcp-startup-diagnostic-windows-latest-msedge-sample-4

12629 passed, 1310 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟡 The one failure is uncertain — likely a flake, but I can't prove it

Hi, I'm the Playwright bot and I took a first look at the CI failures.

The latest report has a single red: [msedge] › mcp/cli-core.spec.ts:140 › uncheck. The PR's product changes don't reach that code path, and it's a classic snapshot race that got rescued on retry within this PR's own run — but I found no occurrence of it failing off this branch, so I can't certify it as pre-existing.

Details

This PR adds PWTEST_MCP_STARTUP_TRACE-gated startup tracing plus MCP test-fixture changes, and reruns the MCP suite 10× serially on fresh runners. startupTrace is a no-op file-append unless the env var is set, and the frames.ts edits only wrap goto()/navigation — nothing in the uncheck / setChecked / aria-snapshot path is touched.

Uncertain

  • [msedge] › mcp/cli-core.spec.ts:140 › uncheck — fails with expect(received).toContain(expected), expecting - checkbox [active] [ref=e2] but getting - checkbox [checked] [ref=e2] — i.e. the checkbox still reads [checked] in the snapshot, a state/timing race rather than a logic error. The instrumented code path (navigation/startup tracing) isn't the one under test, so a real regression is unlikely. But per our flake rule I need the same test failing where this PR can't be responsible, and I don't have it: across the results DB uncheck is green in 192/192 msedge runs (0 final failures) and never fails on main or other PRs in the window. The only failed attempt on record is on this branch (run 30023210154), and it was rescued on retry (…,failed,passed,…). The 10× serial reruns here plausibly just surfaced a latent snapshot race, but without off-branch evidence I'm calling it uncertain, not a confirmed flake.

To be sure either way: a rerun of the MCP suite on this test (or the same 10× harness on an unmodified main) — if uncheck flakes there too, it's pre-existing.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant