Skip to content

test(expo): Improve Maestro E2E reliability and runtime - #9259

Merged
wobsoriano merged 14 commits into
mainfrom
rob/maestro-improvement
Jul 28, 2026
Merged

test(expo): Improve Maestro E2E reliability and runtime#9259
wobsoriano merged 14 commits into
mainfrom
rob/maestro-improvement

Conversation

@wobsoriano

@wobsoriano wobsoriano commented Jul 27, 2026

Copy link
Copy Markdown
Member

Description

Improves Expo Maestro E2E reliability and caches native builds. Runtime drops from ~18m to ~11m on iOS and ~14m on Android when uncached, and ~8m/~10m on cache hits.

Checklist

  • pnpm test runs as expected.
  • pnpm build runs as expected.
  • (If applicable) JSDoc comments have been added or updated for any package exports
  • (If applicable) Documentation has been updated

Type of change

  • 🐛 Bug fix
  • 🌟 New feature
  • 🔨 Breaking change
  • 📖 Refactoring / dependency upgrade / documentation
  • other:

@changeset-bot

changeset-bot Bot commented Jul 27, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5d3e40f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@clerk/expo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
clerk-js-sandbox Ready Ready Preview, Comment Jul 28, 2026 1:13am
swingset Ready Ready Preview, Comment Jul 28, 2026 1:13am

Request Review

@pkg-pr-new

pkg-pr-new Bot commented Jul 27, 2026

Copy link
Copy Markdown

Open in StackBlitz

@clerk/astro

npm i https://pkg.pr.new/@clerk/astro@9259

@clerk/backend

npm i https://pkg.pr.new/@clerk/backend@9259

@clerk/chrome-extension

npm i https://pkg.pr.new/@clerk/chrome-extension@9259

@clerk/clerk-js

npm i https://pkg.pr.new/@clerk/clerk-js@9259

@clerk/electron

npm i https://pkg.pr.new/@clerk/electron@9259

@clerk/electron-passkeys

npm i https://pkg.pr.new/@clerk/electron-passkeys@9259

@clerk/eslint-plugin

npm i https://pkg.pr.new/@clerk/eslint-plugin@9259

@clerk/expo

npm i https://pkg.pr.new/@clerk/expo@9259

@clerk/expo-google-signin

npm i https://pkg.pr.new/@clerk/expo-google-signin@9259

@clerk/expo-passkeys

npm i https://pkg.pr.new/@clerk/expo-passkeys@9259

@clerk/express

npm i https://pkg.pr.new/@clerk/express@9259

@clerk/fastify

npm i https://pkg.pr.new/@clerk/fastify@9259

@clerk/hono

npm i https://pkg.pr.new/@clerk/hono@9259

@clerk/localizations

npm i https://pkg.pr.new/@clerk/localizations@9259

@clerk/nextjs

npm i https://pkg.pr.new/@clerk/nextjs@9259

@clerk/nuxt

npm i https://pkg.pr.new/@clerk/nuxt@9259

@clerk/react

npm i https://pkg.pr.new/@clerk/react@9259

@clerk/react-router

npm i https://pkg.pr.new/@clerk/react-router@9259

@clerk/shared

npm i https://pkg.pr.new/@clerk/shared@9259

@clerk/tanstack-react-start

npm i https://pkg.pr.new/@clerk/tanstack-react-start@9259

@clerk/testing

npm i https://pkg.pr.new/@clerk/testing@9259

@clerk/ui

npm i https://pkg.pr.new/@clerk/ui@9259

@clerk/upgrade

npm i https://pkg.pr.new/@clerk/upgrade@9259

@clerk/vue

npm i https://pkg.pr.new/@clerk/vue@9259

commit: 5d3e40f

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The native-build workflow labels E2E matrix runs, extends their timeout, validates matching React versions, removes fixture-app artifact transfers, and installs built iOS and Android applications directly from fixture outputs. Maestro flows clear keychain state, retry warmup, and record execution results. The sign-in form handles email-code verification for additional sign-in statuses, and the fixture updates React versions. A changeset records the update.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: improving Expo Maestro E2E reliability and runtime.
Description check ✅ Passed The description is directly related to the changes, mentioning Expo Maestro E2E reliability and native build caching.

Comment @coderabbitai help to get the list of available commands.

@wobsoriano wobsoriano closed this Jul 27, 2026
@wobsoriano wobsoriano reopened this Jul 27, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@integration/tests/expo-native/run-flows.sh`:
- Around line 53-55: Condense the warmup comment immediately above the warmup
logic to one terse line, preserving both the cold-start/a11y-tree rationale and
the instruction to abort on repeated failure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: ea16be0f-4bb7-4e33-8f6f-c93c97ee38b5

📥 Commits

Reviewing files that changed from the base of the PR and between ef889c2 and 5227cba.

📒 Files selected for processing (5)
  • .github/workflows/expo-native-build.yml
  • integration/tests/expo-native/flows/subflows/_warmup.yaml
  • integration/tests/expo-native/flows/subflows/open-app.yaml
  • integration/tests/expo-native/run-android-flows.sh
  • integration/tests/expo-native/run-flows.sh
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • clerk/clerk_go (manual)
  • clerk/dashboard (manual)
  • clerk/accounts (manual)
  • clerk/backoffice (manual)
  • clerk/clerk (manual)
  • clerk/clerk-docs (manual)
  • clerk/cloudflare-workers (manual)
  • clerk/clerk-ios (auto-detected)
  • clerk/clerk-android (auto-detected)
  • clerk/cli (auto-detected)
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/expo-native-build.yml

Comment on lines +53 to +55
# Warmup: parse the JS bundle + populate the a11y tree once so the first real
# flow doesn't flake on cold-start; force-stop afterwards so the first
# launchApp clearState doesn't race a still-foregrounded app process.
maestro test ${debug_args+"${debug_args[@]}"} flows/subflows/_warmup.yaml || true
# flow doesn't flake on cold-start. A repeated failure means the app is not
# runnable, so abort instead of timing out and retrying every real flow.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Condense the warmup comment.

Keep the rationale, but reduce this three-line block to one terse line.

Proposed fix
-# Warmup: parse the JS bundle + populate the a11y tree once so the first real
-# flow doesn't flake on cold-start. A repeated failure means the app is not
-# runnable, so abort instead of timing out and retrying every real flow.
+# Retry warmup once; abort if the app is still not runnable.

As per coding guidelines, “keep warranted comments to one terse line rather than a verbose block.”

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Warmup: parse the JS bundle + populate the a11y tree once so the first real
# flow doesn't flake on cold-start; force-stop afterwards so the first
# launchApp clearState doesn't race a still-foregrounded app process.
maestro test ${debug_args+"${debug_args[@]}"} flows/subflows/_warmup.yaml || true
# flow doesn't flake on cold-start. A repeated failure means the app is not
# runnable, so abort instead of timing out and retrying every real flow.
# Retry warmup once; abort if the app is still not runnable.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@integration/tests/expo-native/run-flows.sh` around lines 53 - 55, Condense
the warmup comment immediately above the warmup logic to one terse line,
preserving both the cold-start/a11y-tree rationale and the instruction to abort
on repeated failure.

Source: Coding guidelines

@wobsoriano wobsoriano changed the title test(expo): Run Maestro tests after native builds test(expo): Improve Maestro E2E reliability and runtime Jul 28, 2026
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

API Changes Report

Generated by Break Check on 2026-07-28T01:13:42.753Z

Summary

Metric Count
Packages analyzed 19
Packages with changes 0
🔴 Breaking changes 0
🟡 Non-breaking changes 0
🟢 Additions 0

No API Changes Detected

All packages have stable APIs with no detected changes.


Report generated by Break Check

Last ran on 5d3e40f.

@wobsoriano
wobsoriano merged commit 1397d63 into main Jul 28, 2026
59 checks passed
@wobsoriano
wobsoriano deleted the rob/maestro-improvement branch July 28, 2026 02:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants