Skip to content

[wasm][coreCLR] R2R: VM-side load of a flat webcil composite image - #131016

Merged
lewing merged 3 commits into
dotnet:mainfrom
lewing:lewing-wasm-r2r-composite-load
Jul 23, 2026
Merged

[wasm][coreCLR] R2R: VM-side load of a flat webcil composite image#131016
lewing merged 3 commits into
dotnet:mainfrom
lewing:lewing-wasm-r2r-composite-load

Conversation

@lewing

@lewing lewing commented Jul 18, 2026

Copy link
Copy Markdown
Member

What

Minimal, OS-neutral VM plumbing so the runtime can load a flat webcil-in-wasm composite R2R image. crossgen2 emits the wasm startup R2R composite as a flat webcil image, but the R2R/PE-decoder load path assumes a mapped PE — a named RTR_HEADER export and IsMapped() RVA addressing — so today the VM can't find the R2R header or address sections in a flat webcil composite. Four small touch-points close that gap:

File Change
vm/nativeimage.cpp webcil composite exposes no named RTR_HEADER export → fall back to the decoder's R2R header
vm/readytoruninfo.cpp treat flat webcil as RVA-addressable directly from base (IsWebcilFormat() alongside IsMapped())
vm/peimagelayout.inl route IsComponentAssembly() through the webcil decoder (DECODER_DISPATCH)
vm/peimagelayout.cpp guard against re-applying base relocations to a host-probed shared buffer

Guarded on TARGET_WASM / FEATURE_PORTABLE_ENTRYPOINTS, so the same code serves browser and wasi from one path.

Why

Part of #130524. Addresses the composite-loadability open question in #130519 ("whether the composite container is fully browser-loadable today or needs work"). This is the VM-consumer half — the producer/packaging side stays with #130519.

Feedback wanted — the relocation guard

The one design question in this PR is the peimagelayout.cpp guard. Host-probed webcil R2R images are backed by a single shared in-memory buffer, not a copy-on-write file mapping. The assembly binder can open such an image more than once (identity probe + load) as independent PEImage/layout objects over the same buffer, and each open would re-run ApplyBaseRelocations on that memory. WASM table-index relocations are additive (index += tableBase), so a second application doubles tableBase and pushes indirect-call indices out of the function table.

This PR guards it with a VM-local set of already-relocated base addresses (SetSHash<TADDR>). Because the duplicate opens are distinct objects sharing one buffer, a per-PEImageLayout/per-PEImage flag wouldn't dedup them — the state has to key on the shared resource. It is intentionally not synchronized (relies on the current single-threaded wasm runtime) and is marked INTERIM in-code.

The cleaner production direction is probably to relocate the host-probed buffer once at the point it's handed to the runtime and skip webcil in ApplyBaseRelocations entirely — but that reaches into the host/probe boundary, which is out of scope for this VM-only load change. Guidance on the preferred layer here is the main thing I'm looking for.

Scope / out of scope

Out: composite production/packaging (#130519); the CoreLib-R2R execution gates — QCall/PInvoke, class-init, etc. (#129850); SIMD instruction-set reporting; the host-side merge/probe mechanism.

Testing

  • Compiles under ./build.sh clr -os wasi -c Release (the TARGET_WASM branches are only exercised by a wasm build).
  • Validated on the WASI R2R prototype: a composite System.Private.CoreLib R2R image (~49k R2R functions) loads and dispatches through exactly these four changes.
  • Cannot run standalone yet — a loaded CoreLib R2R image needs the execution gates tracked in Running wasm with R2R'd System.Private.CoreLib #129850 to get past startup. This PR is the load prerequisite for that work, landing incrementally behind the wasm guards like the rest of the wasm R2R bring-up.

Note

This PR (code and description) was drafted with GitHub Copilot assistance.

Copilot AI review requested due to automatic review settings July 18, 2026 17:49
@lewing lewing added the arch-wasm WebAssembly architecture label Jul 18, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke
See info in area-owners.md if you want to be subscribed.

@lewing
lewing force-pushed the lewing-wasm-r2r-composite-load branch from a884ea1 to d21c292 Compare July 18, 2026 17:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds CoreCLR VM support for loading a flat Webcil-in-Wasm composite ReadyToRun (R2R) image by adjusting a few PE/R2R assumptions (exported RTR header, RVA addressing, component detection routing) and adding a WASM-specific safeguard against double-applying relocations over a shared host-provided buffer.

Changes:

  • Teach composite-owner lookup to treat Webcil images as RVA-addressable (alongside mapped PE images).
  • Route IsComponentAssembly() through the active decoder via DECODER_DISPATCH.
  • Add a WASM-only guard to avoid reapplying base relocations to a shared Webcil buffer.
  • Fall back to GetReadyToRunHeader() when RTR_HEADER export is absent (WASM/Webcil composite).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/coreclr/vm/nativeimage.cpp Adds a WASM/Webcil-specific fallback to obtain the R2R header via the decoder when no RTR_HEADER export exists.
src/coreclr/vm/readytoruninfo.cpp Adjusts composite-owner string resolution to treat Webcil as “mapped-like” for RVA addressing.
src/coreclr/vm/peimagelayout.inl Switches IsComponentAssembly() to decoder dispatch for PE/Webcil parity.
src/coreclr/vm/peimagelayout.cpp Adds a WASM-only guard to prevent double relocation of a shared host-probed Webcil buffer.

Comment thread src/coreclr/vm/readytoruninfo.cpp Outdated
Copilot AI review requested due to automatic review settings July 18, 2026 17:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/coreclr/vm/readytoruninfo.cpp:543

  • For Webcil images, ReadyToRun section "VirtualAddress" is still an RVA that must be translated through the Webcil section table (like WebcilDecoder::GetRvaData does). Using GetBase()+virtualAddress assumes an identity RVA->file-offset mapping that WebcilDecoder does not guarantee, and will point into the Webcil header/incorrect data when PointerToRawData != VirtualAddress.

Suggested: keep the direct base+RVA path only for mapped PE images, and use GetRvaData for Webcil.

    LPCUTF8 ownerCompositeExecutableName = NULL;
    if (pLayout->IsMapped() || pLayout->IsWebcilFormat())
    {
        // Mapped PE images and (flat) webcil images are RVA-addressable directly from the base:
        // webcil is flat-mapped so the R2R section VirtualAddress maps 1:1 onto GetBase()+VA.

Comment thread src/coreclr/vm/peimagelayout.cpp Outdated
Copilot AI review requested due to automatic review settings July 18, 2026 18:04
@lewing
lewing force-pushed the lewing-wasm-r2r-composite-load branch from d21c292 to b5705a1 Compare July 18, 2026 18:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment thread src/coreclr/vm/nativeimage.cpp Outdated
Comment thread src/coreclr/vm/peimagelayout.cpp Outdated
Copilot AI review requested due to automatic review settings July 18, 2026 18:28
@lewing
lewing force-pushed the lewing-wasm-r2r-composite-load branch from b5705a1 to 7e433ae Compare July 18, 2026 18:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread src/coreclr/vm/peimagelayout.cpp
crossgen2 emits the wasm startup R2R composite as a flat webcil-in-wasm
image, but the R2R/PE-decoder load path assumes a mapped PE (a named
RTR_HEADER export and IsMapped() RVA addressing), so the VM cannot locate
the R2R header or address sections in a flat webcil composite. Add the
minimal OS-neutral (TARGET_WASM / FEATURE_PORTABLE_ENTRYPOINTS) plumbing to
load it:

- nativeimage.cpp: fall back to the decoder's R2R header when there is no
  named RTR_HEADER export.
- readytoruninfo.cpp: treat flat webcil as RVA-addressable from base.
- peimagelayout.inl: route IsComponentAssembly() through the webcil decoder.
- peimagelayout.cpp: guard against re-applying base relocations to a
  host-probed shared buffer (additive wasm table-index relocs would double).

Applies to both wasm targets (browser and wasi). Part of dotnet#130524, addresses
the composite-loadability open question in dotnet#130519.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae
Copilot AI review requested due to automatic review settings July 18, 2026 20:46
@lewing
lewing force-pushed the lewing-wasm-r2r-composite-load branch from 7e433ae to c21bec4 Compare July 18, 2026 20:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/coreclr/vm/peimagelayout.cpp:261

  • The relocation de-dup set is accessed without synchronization, but the WASM build can be configured with multithreading (e.g., WasmEnableThreads => FEATURE_MULTITHREADING). In that configuration, concurrent loads of the same shared webcil buffer could race on SetSHash mutation, or even both pass the Contains() check and apply relocations twice. Consider taking a lightweight lock when IsWebcilFormat() is true and holding it for the duration of ApplyBaseRelocations so at most one thread can relocate+record a given base at a time.
    typedef SetSHash< TADDR,
                      NoRemoveSHashTraits< NonDacAwareSHashTraits< SetSHashTraits<TADDR> > > > RelocatedWebcilSet;
    static RelocatedWebcilSet s_relocatedWebcilBases;
    const bool isHostProbedWebcil = IsWebcilFormat();
    const TADDR webcilBase = isHostProbedWebcil ? (TADDR)GetBase() : (TADDR)0;

The peimagelayout.inl change routes PEImageLayout::IsComponentAssembly()
through DECODER_DISPATCH to WebcilDecoder::IsComponentAssembly(), but that
was stubbed to return FALSE, leaving the dispatch inert: readytoruninfo.cpp
gates composite R2R load on IsComponentAssembly(), so every webcil composite
was reported as non-component. Implement it from the R2R COMPONENT flag,
matching the PE decoder path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae
Copilot AI review requested due to automatic review settings July 20, 2026 01:06
@lewing

lewing commented Jul 20, 2026

Copy link
Copy Markdown
Member Author

Added src/coreclr/inc/webcildecoder.h (new commit): implement WebcilDecoder::IsComponentAssembly() from the R2R COMPONENT flag.

The peimagelayout.inl change here routes PEImageLayout::IsComponentAssembly() through DECODER_DISPATCH to the webcil decoder, but that method was stubbed return FALSE — so the dispatch was inert (every webcil composite reported as non-component). readytoruninfo.cpp gates the composite R2R load on IsComponentAssembly(), so this is required for composite detection to actually work (matches the PE decoder path). Verified building under ./build.sh clr -os wasi -c Release.

Note

Comment drafted with GitHub Copilot assistance.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Comment thread src/coreclr/vm/peimagelayout.cpp
Comment thread src/coreclr/vm/peimagelayout.cpp
Comment thread src/coreclr/inc/webcildecoder.h

@davidwrighton davidwrighton left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good. just a few more things to fix.

Comment thread src/coreclr/vm/nativeimage.cpp Outdated
Comment thread src/coreclr/vm/peimagelayout.cpp
Comment thread src/coreclr/vm/peimagelayout.cpp Outdated
Resolves davidwrighton's review feedback on the flat webcil composite R2R load:

- nativeimage.cpp: on WASM, don't call GetExport("RTR_HEADER") at all. PE R2R
  images that rely on that export can't be loaded on WASM, so obtain the R2R
  header from the decoder via HasReadyToRunHeader()/GetReadyToRunHeader(). The
  NULL-header check still fails genuinely non-R2R images.

- peimagelayout.cpp: make the webcil relocation de-dup MT-safe. Take a new
  CrstWebcilImageRelocation lock before the Contains check and hold it for the
  rest of ApplyBaseRelocations, so concurrent loads of the same shared buffer
  (WASM can be built with FEATURE_MULTITHREADING) can't both relocate it.
  Record the base right after the check rather than at method end, and drop the
  misleading "throw-safety" comment: a throw partway through relocation is
  unrecoverable for the additive relocation model regardless of when we record.

- peimage.cpp / peimagelayout.{h,cpp}: add PEImageLayout::Startup() to Init the
  new Crst, called from PEImage::Startup().

- CrstTypes.def / crsttypes_generated.h: add the CrstWebcilImageRelocation type
  (Unordered leaf), regenerated via CrstTypeTool.

Validated with a browser-wasm coreclr build (clr subset): peimagelayout.cpp,
nativeimage.cpp, and peimage.cpp compile clean (0 warnings, 0 errors).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53517266-f0ba-4d8e-9fa2-a9e6fc16753b
Copilot AI review requested due to automatic review settings July 23, 2026 02:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/coreclr/vm/nativeimage.cpp:219

  • On TARGET_WASM this path now uses the decoder header whenever HasReadyToRunHeader() is true, regardless of image format. That means a PE composite that is missing the expected RTR_HEADER export could be silently accepted on WASM (since HasReadyToRunHeader() also works for PE), weakening validation beyond the stated webcil-only intent. Gate the decoder-based header lookup on IsWebcilFormat() and keep the export path for PE layouts.
#ifdef TARGET_WASM
        // On WebAssembly the runtime only loads flat webcil composites, which do not expose a named
        // "RTR_HEADER" export the way PE R2R images do; obtain the R2R header from the decoder instead.
        // PE R2R images (which rely on the export) cannot be loaded on WASM, so the export path is never
        // taken here. A genuinely non-R2R image still fails validation below via the NULL header check.
        if (peLoadedImage->HasReadyToRunHeader())
            *header = peLoadedImage->GetReadyToRunHeader();
#else // TARGET_WASM
        *header = (READYTORUN_HEADER *)peLoadedImage->GetExport("RTR_HEADER");
#endif // TARGET_WASM

@lewing
lewing marked this pull request as ready for review July 23, 2026 18:58
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@lewing
lewing enabled auto-merge (squash) July 23, 2026 19:21
@lewing

lewing commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/ba-g remaining jobs are stuck infra

@lewing
lewing merged commit 71830fd into dotnet:main Jul 23, 2026
104 of 110 checks passed
@lewing
lewing deleted the lewing-wasm-r2r-composite-load branch July 23, 2026 22:31
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-rc1 milestone Jul 24, 2026
hez2010 pushed a commit to hez2010/runtime that referenced this pull request Jul 26, 2026
…otnet#131016)

## What

Minimal, OS-neutral VM plumbing so the runtime can **load a flat
webcil-in-wasm composite R2R image**. crossgen2 emits the wasm startup
R2R composite as a flat webcil image, but the R2R/PE-decoder load path
assumes a mapped PE — a named `RTR_HEADER` export and `IsMapped()` RVA
addressing — so today the VM can't find the R2R header or address
sections in a flat webcil composite. Four small touch-points close that
gap:

| File | Change |
|---|---|
| `vm/nativeimage.cpp` | webcil composite exposes no named `RTR_HEADER`
export → fall back to the decoder's R2R header |
| `vm/readytoruninfo.cpp` | treat flat webcil as RVA-addressable
directly from base (`IsWebcilFormat()` alongside `IsMapped()`) |
| `vm/peimagelayout.inl` | route `IsComponentAssembly()` through the
webcil decoder (`DECODER_DISPATCH`) |
| `vm/peimagelayout.cpp` | guard against re-applying base relocations to
a host-probed shared buffer |

Guarded on `TARGET_WASM` / `FEATURE_PORTABLE_ENTRYPOINTS`, so the **same
code serves browser and wasi** from one path.

## Why

Part of dotnet#130524. Addresses the composite-loadability open question in
dotnet#130519 ("whether the composite container is fully browser-loadable
today or needs work"). This is the VM-*consumer* half — the
*producer*/packaging side stays with dotnet#130519.

## Feedback wanted — the relocation guard

The one design question in this PR is the `peimagelayout.cpp` guard.
Host-probed webcil R2R images are backed by a **single shared in-memory
buffer**, not a copy-on-write file mapping. The assembly binder can open
such an image more than once (identity probe + load) as **independent**
`PEImage`/layout objects over the *same* buffer, and each open would
re-run `ApplyBaseRelocations` on that memory. WASM table-index
relocations are additive (`index += tableBase`), so a second application
doubles `tableBase` and pushes indirect-call indices out of the function
table.

This PR guards it with a VM-local set of already-relocated base
addresses (`SetSHash<TADDR>`). Because the duplicate opens are distinct
objects sharing one buffer, a per-`PEImageLayout`/per-`PEImage` flag
wouldn't dedup them — the state has to key on the shared resource. It is
intentionally **not** synchronized (relies on the current
single-threaded wasm runtime) and is marked INTERIM in-code.

The cleaner production direction is probably to **relocate the
host-probed buffer once at the point it's handed to the runtime and skip
webcil in `ApplyBaseRelocations` entirely** — but that reaches into the
host/probe boundary, which is out of scope for this VM-only load change.
Guidance on the preferred layer here is the main thing I'm looking for.

## Scope / out of scope

**Out:** composite *production*/packaging (dotnet#130519); the CoreLib-R2R
execution gates — QCall/PInvoke, class-init, etc. (dotnet#129850); SIMD
instruction-set reporting; the host-side merge/probe mechanism.

## Testing

- Compiles under `./build.sh clr -os wasi -c Release` (the `TARGET_WASM`
branches are only exercised by a wasm build).
- Validated on the WASI R2R prototype: a composite
`System.Private.CoreLib` R2R image (~49k R2R functions) loads and
dispatches through exactly these four changes.
- **Cannot run standalone yet** — a loaded CoreLib R2R image needs the
execution gates tracked in dotnet#129850 to get past startup. This PR is the
load prerequisite for that work, landing incrementally behind the wasm
guards like the rest of the wasm R2R bring-up.

> [!NOTE]
> This PR (code and description) was drafted with GitHub Copilot
assistance.

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae
Copilot-Session: 53517266-f0ba-4d8e-9fa2-a9e6fc16753b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch-wasm WebAssembly architecture area-VM-coreclr

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants