Skip to content

Improve reusable TestCase and Swift Testing integration - #2

Draft
kudit wants to merge 29 commits into
mainfrom
agent/improve-test-infrastructure
Draft

Improve reusable TestCase and Swift Testing integration#2
kudit wants to merge 29 commits into
mainfrom
agent/improve-test-infrastructure

Conversation

@kudit

@kudit kudit commented Jul 26, 2026

Copy link
Copy Markdown
Owner

What changed

This draft PR keeps the work in two reviewable commits:

  1. Consolidate TestCase execution and preserve diagnostics

    • Routes execute() and the live-UI run() path through one lifecycle implementation.
    • Preserves async setup/test/teardown behavior without attempting to await inside defer.
    • Adds .parallel and .serialized execution modes for tests that mutate process-global state.
    • Introduces TestFailure, retaining file, function, line, column, and detailed expectation values.
    • Logs expectation failures at .ERROR before throwing so command-line runs retain useful diagnostics.
  2. Add reusable Swift Testing module adapter

    • Adds a Compatibility Testing Library product in the same package and .swiftpm project.
    • Adds ModuleTestEntry, one Swift Testing argument per Compatibility TestCase.
    • Uses CustomTestStringConvertible for readable Xcode names such as Device › Battery Tests › Reports current battery state.
    • Uses CustomTestArgumentEncodable for stable argument identity and selective reruns.
    • Lets consuming modules reduce their adapter to ModuleTestEntry.entries(including:) and entry.execute().

Design decisions

  • TestCase remains @MainActor because it owns observable UI progress and a mutable test closure. A sendable execution snapshot keeps detached execution safe and prevents the UI model from crossing actors.
  • main {} remains distinct from await MainActor.run {}: the former schedules fire-and-forget work, while the latter awaits completion and can return a value.
  • The broader main/background/sleep/WASM overload consolidation belongs to follow-up PR Consolidate threading APIs and add WebAssembly async fallbacks #3.

Open draft review items

Before this PR is marked ready for review:

  • make .serialized execution truly exclusive against both serialized and parallel reusable tests;
  • mark the debug-configuration test serialized;
  • restore debug configuration with test-local synchronous defer rather than automatically resetting all Compatibility settings around every test;
  • compile and run the package tests in Xcode and SwiftPM across the supported platform matrix.

Validation so far

  • New Swift files were syntax-parsed with Swift 6.2.1.
  • Full package compilation and platform validation remain required before merge.

Dependent follow-up

After this API is finalized and released, create a Device pull request that imports CompatibilityTesting and replaces its duplicated Entry/module lookup adapter with ModuleTestEntry.

@kudit
kudit force-pushed the agent/improve-test-infrastructure branch from 20d8a11 to c6c86f1 Compare July 26, 2026 02:08
kudit added 27 commits July 27, 2026 10:32
Fix test infrastructure availability
Simplified code duplication and context description.
Refactor lastPathComponent to support Windows-style paths and remove Foundation dependency.
Added guidelines for collaborative coding workflow to improve interaction with maintainers.
Added testing requirements and TODOs for release preparation.
fix @available where macOS 12 was paired with watchOS 8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant