Add deterministic NativeAOT regression smoke test for GC restricted reverse P/Invoke assert path

[Copilot]

Resolves #110683.

This adds a focused NativeAOT regression test for the GC restricted callout path where background GC can invoke managed [UnmanagedCallersOnly] callbacks and hit Thread::InlineTryFastReversePInvoke assertion behavior prior to the runtime fix. The change is test-only and is intended to reliably drive the failing pre-fix path, not modify product behavior.

• Test added

  • src/tests/nativeaot/SmokeTests/GcRestrictedCalloutReversePInvoke/
  • New smoke test project plus program harness.

• Repro wiring

  • Registers tracker support via ComWrappers.RegisterForTrackerSupport(...).
  • Creates a tracker object through MockReferenceTrackerRuntime so GC restricted tracker callbacks are active (GCStopCollection path).

• Deterministic triggering

  • Forces repeated concurrent Gen2 collections (GC.Collect(..., blocking:false)).
  • Runs bounded allocation pressure on a background thread.
  • Uses timeout + synchronization (SpinWait + elapsed bound) instead of unbounded probabilistic loops.

• Scope/guards

  • Configured as process-isolated NativeAOT smoke test.
  • Excludes Apple mobile and Android targets where the required native dependency is not available in this leg.
  • Keeps coverage broad otherwise (not win-x86-only).

• Expectation documented

  • Source comment references dotnet/runtime#110683 and states expected behavior:
    • release: test should complete,
    • assertion-enabled runtime (before product fix): can fail on the target assertion path.

• Environment limitation observed while validating command

  • In this environment, Release NativeAOT targeted test build failed due to missing Microsoft.DotNet.ILCompiler.SingleEntry.targets; Checked configuration was used for targeted execution.

GC.Collect(2, GCCollectionMode.Forced, blocking: false, compacting: false);

if (GC.GetGCMemoryInfo().Generation == 2 &&
    GC.GetGCMemoryInfo().Concurrent)
{
    // Concurrent Gen2 observed while tracker GC callbacks are registered.
    // This is the targeted repro envelope for dotnet/runtime#110683.
}

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib
See info in area-owners.md if you want to be subscribed.

[MichalStrehovsky]

@copilot now fix the assert failure by weakening the assertion as described in #110683.

[Copilot]

@copilot now fix the assert failure by weakening the assertion as described in #110683.

Fixed in 4413efc — the NativeAOT reverse P/Invoke fast path now asserts IsGCSpecial() for DoNotTriggerGc threads instead of requiring ThreadStore::IsTrapThreadsRequested().

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/tests/Interop/COM/ComWrappers/GcRestrictedCalloutReversePInvoke/GcRestrictedCalloutReversePInvoke.csproj:21

• This project uses xUnit ([Fact] in Program.cs) but unlike the other ComWrappers interop test projects in this directory it doesn’t reference $(TestLibraryProjectPath). As-is, this will likely fail to compile due to missing xUnit/test infrastructure references.

  <ItemGroup>
    <Compile Include="../Common.cs" />
    <Compile Include="Program.cs" />
  </ItemGroup>

  <ItemGroup>
    <CMakeProjectReference Include="../MockReferenceTrackerRuntime/CMakeLists.txt" />
  </ItemGroup>
</Project>

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[MichalStrehovsky]

We now have a test that hits the assert without this fix and works with the fix. Is the test valuable enough to keep?

Cc @VSadov

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

src/tests/Interop/COM/ComWrappers/GcRestrictedCalloutReversePInvoke/GcRestrictedCalloutReversePInvoke.csproj:16

• This test project uses xUnit ([Fact] in Program.cs) but the csproj is missing the standard that all other ComWrappers test projects in this directory use. Without it, the project may not compile or may miss test infrastructure references.

  <ItemGroup>
    <Compile Include="../Common.cs" />
    <Compile Include="Program.cs" />
  </ItemGroup>

src/tests/Interop/COM/ComWrappers/GcRestrictedCalloutReversePInvoke/GcRestrictedCalloutReversePInvoke.csproj:5

• This project is described as a NativeAOT smoke test in the PR metadata, but it’s currently added under src/tests/Interop/COM/ComWrappers (not src/tests/nativeaot/SmokeTests). If it’s intended to run in the NativeAOT smoke test legs, it likely needs to live under the nativeaot test tree (or have additional wiring) to be discovered/executed there.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <CLRTestPriority>0</CLRTestPriority>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <!-- Test interacts with GC collections, also needed for CLRTestTargetUnsupported below -->

src/tests/Interop/COM/ComWrappers/GcRestrictedCalloutReversePInvoke/Program.cs:63

• Use xUnit assertions for test failures instead of throwing Exception. This keeps failures consistent with the rest of the ComWrappers tests and produces cleaner output in test reports.

        if (!observedConcurrentGen2)
        {
            throw new Exception($"Timed out after {timeout} waiting for a concurrent Gen2 GC. Initial count: {initialGen2Collections}, final count: {GC.CollectionCount(2)}.");
        }

src/coreclr/nativeaot/Runtime/thread.inl:208

• The PR description states the change is test-only and does not modify product behavior, but this hunk changes a runtime assertion condition in Thread::InlineTryFastReversePInvoke. Please update the PR description/scope accordingly (or split the runtime change into the appropriate fix PR if this PR is meant to be test-only).

    if (IsDoNotTriggerGcSet())
    {
        // We expect this scenario only when EE is stopped or we're on a GC worker thread.
        ASSERT(ThreadStore::IsTrapThreadsRequested() || IsGCSpecial());
        // no need to do anything