test: Add network envelope snapshot tests - #8638
Merged
Merged
Conversation
Capture complete normalized transaction and failed-request envelopes from the server-backed networking integration suite.
philprime
added
the
run-full-ci
Use the SDK serializer and deserializer with partial fixture assertions. Remove baseline regeneration and standalone tests for test-only normalization logic.
Account for macOS and SDK V10 payload differences, and filter captured envelopes by item type to avoid unrelated telemetry.
philprime
marked this pull request as ready for review
August 3, 2026 08:22
philprime
requested review from
NinjaLikesCheez,
itaybre,
noahsmartin and
philipphofmann
as code owners
August 3, 2026 08:22
![sentry[bot]](https://cdn.github.com/in/12637?s=60&v=4){kind=small}
Use placeholders for SDK and locale-dependent values, and allow repeated expectation fulfillment from task cancellation or matching envelopes.
9 tasks
philipphofmann
approved these changes
Aug 3, 2026
philipphofmann
left a comment
philipphofmann
left a comment
Member
There was a problem hiding this comment.
Thanks for adding this 😃, LGTM 🚀
Address review feedback on the network envelope tests. Rename the helper and fixtures to snapshot terminology, since that is the established name for this testing concept. Compare snapshots strictly in both directions. Recursing over the fixture keys alone never visited keys that only exist in the envelope, so additive regressions passed silently. This immediately surfaced four uncovered keys: sdk.features, sdk.integrations, payload.extra and payload.tags. Aggregate all mismatches into one failure that lists every difference and prints the actual envelope JSON, so it is clear both what changed and how to update the snapshot. Replace clearTestState with SentrySDK.close, which uninstalls the integrations and thereby disables the network tracker. Also serve the test server on port 8081 in the make targets. The tests and scripts/start-test-server.sh use 8081, but the targets bound the default 8080, so they could not serve them.
philprime
enabled auto-merge (squash)
August 3, 2026 15:04
![cursor[bot]](https://cdn.github.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2c06824. Configure here.
📲 Install BuildsiOS
|
Bugbot was right: the strict comparison failed on V10 because failed-request capture assigns cookies to the request and to contexts.response, and both were absent from the snapshot. Give V10 its own -v10 snapshot per envelope instead of carving exceptions into the comparison. Both variants are then compared just as strictly, and the V10 deltas are visible in one file: cookies, infer_ip=auto, and no device.locale. This removes the SDK_V10 special cases for infer_ip and device.locale, leaving a single conditional for file selection. Add TEST_PLAN to the macOS test targets, which could not run the test-server plan at all before.
philprime
changed the title
Contributor
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| a8fc1c8 | 1237.62 ms | 1261.57 ms | 23.95 ms |
| 07d6099 | 1224.06 ms | 1247.13 ms | 23.06 ms |
| 6ae3681 | 1226.09 ms | 1257.47 ms | 31.38 ms |
| 871ce4d | 1223.43 ms | 1257.17 ms | 33.75 ms |
| 7087caa | 1212.33 ms | 1261.69 ms | 49.36 ms |
| d5bcdd8 | 1224.61 ms | 1252.73 ms | 28.13 ms |
| 66cfde1 | 1226.84 ms | 1255.50 ms | 28.66 ms |
| b927860 | 1213.67 ms | 1257.94 ms | 44.27 ms |
| 821fd52 | 1225.70 ms | 1256.73 ms | 31.03 ms |
| f82ca9b | 1227.06 ms | 1264.67 ms | 37.61 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| a8fc1c8 | 24.14 KiB | 1.23 MiB | 1.20 MiB |
| 07d6099 | 24.14 KiB | 1.17 MiB | 1.15 MiB |
| 6ae3681 | 24.14 KiB | 1.19 MiB | 1.17 MiB |
| 871ce4d | 24.14 KiB | 1.22 MiB | 1.20 MiB |
| 7087caa | 24.14 KiB | 1.17 MiB | 1.15 MiB |
| d5bcdd8 | 24.14 KiB | 1.22 MiB | 1.20 MiB |
| 66cfde1 | 24.14 KiB | 1.20 MiB | 1.17 MiB |
| b927860 | 24.14 KiB | 1.19 MiB | 1.17 MiB |
| 821fd52 | 24.14 KiB | 1.26 MiB | 1.23 MiB |
| f82ca9b | 24.14 KiB | 1.17 MiB | 1.14 MiB |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add committed fixture comparisons for envelopes produced by network instrumentation. The server-backed integration suite captures a successful-request transaction envelope and an HTTP 400 failed-request event envelope through the production transport adapter.
The tests round-trip captured envelopes through the SDK serializer and deserializer, then assert the stable fixture fields recursively. Placeholder values accept runtime-generated data such as IDs, timestamps, machine context, and stack traces. Concrete network request data, response context, spans, exception details, SDK metadata, envelope headers, and item headers remain comparison-sensitive.
This addresses the outstanding full-envelope validation discussed in #8502. Existing networking integration tests assert selected fields but do not detect regressions elsewhere in the payload.
Validated with formatting, static analysis, an iOS build, and the complete five-test
Sentry_TestServerplan.#skip-changelog
Closes #8648