[wasm][coreCLR] R2R: VM-side load of a flat webcil composite image - #131016
Conversation
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Tagging subscribers to this area: @agocke |
a884ea1 to
d21c292
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds CoreCLR VM support for loading a flat Webcil-in-Wasm composite ReadyToRun (R2R) image by adjusting a few PE/R2R assumptions (exported RTR header, RVA addressing, component detection routing) and adding a WASM-specific safeguard against double-applying relocations over a shared host-provided buffer.
Changes:
- Teach composite-owner lookup to treat Webcil images as RVA-addressable (alongside mapped PE images).
- Route
IsComponentAssembly()through the active decoder viaDECODER_DISPATCH. - Add a WASM-only guard to avoid reapplying base relocations to a shared Webcil buffer.
- Fall back to
GetReadyToRunHeader()whenRTR_HEADERexport is absent (WASM/Webcil composite).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| src/coreclr/vm/nativeimage.cpp | Adds a WASM/Webcil-specific fallback to obtain the R2R header via the decoder when no RTR_HEADER export exists. |
| src/coreclr/vm/readytoruninfo.cpp | Adjusts composite-owner string resolution to treat Webcil as “mapped-like” for RVA addressing. |
| src/coreclr/vm/peimagelayout.inl | Switches IsComponentAssembly() to decoder dispatch for PE/Webcil parity. |
| src/coreclr/vm/peimagelayout.cpp | Adds a WASM-only guard to prevent double relocation of a shared host-probed Webcil buffer. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/coreclr/vm/readytoruninfo.cpp:543
- For Webcil images, ReadyToRun section "VirtualAddress" is still an RVA that must be translated through the Webcil section table (like WebcilDecoder::GetRvaData does). Using GetBase()+virtualAddress assumes an identity RVA->file-offset mapping that WebcilDecoder does not guarantee, and will point into the Webcil header/incorrect data when PointerToRawData != VirtualAddress.
Suggested: keep the direct base+RVA path only for mapped PE images, and use GetRvaData for Webcil.
LPCUTF8 ownerCompositeExecutableName = NULL;
if (pLayout->IsMapped() || pLayout->IsWebcilFormat())
{
// Mapped PE images and (flat) webcil images are RVA-addressable directly from the base:
// webcil is flat-mapped so the R2R section VirtualAddress maps 1:1 onto GetBase()+VA.
d21c292 to
b5705a1
Compare
b5705a1 to
7e433ae
Compare
crossgen2 emits the wasm startup R2R composite as a flat webcil-in-wasm image, but the R2R/PE-decoder load path assumes a mapped PE (a named RTR_HEADER export and IsMapped() RVA addressing), so the VM cannot locate the R2R header or address sections in a flat webcil composite. Add the minimal OS-neutral (TARGET_WASM / FEATURE_PORTABLE_ENTRYPOINTS) plumbing to load it: - nativeimage.cpp: fall back to the decoder's R2R header when there is no named RTR_HEADER export. - readytoruninfo.cpp: treat flat webcil as RVA-addressable from base. - peimagelayout.inl: route IsComponentAssembly() through the webcil decoder. - peimagelayout.cpp: guard against re-applying base relocations to a host-probed shared buffer (additive wasm table-index relocs would double). Applies to both wasm targets (browser and wasi). Part of dotnet#130524, addresses the composite-loadability open question in dotnet#130519. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae
7e433ae to
c21bec4
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/coreclr/vm/peimagelayout.cpp:261
- The relocation de-dup set is accessed without synchronization, but the WASM build can be configured with multithreading (e.g., WasmEnableThreads => FEATURE_MULTITHREADING). In that configuration, concurrent loads of the same shared webcil buffer could race on SetSHash mutation, or even both pass the Contains() check and apply relocations twice. Consider taking a lightweight lock when IsWebcilFormat() is true and holding it for the duration of ApplyBaseRelocations so at most one thread can relocate+record a given base at a time.
typedef SetSHash< TADDR,
NoRemoveSHashTraits< NonDacAwareSHashTraits< SetSHashTraits<TADDR> > > > RelocatedWebcilSet;
static RelocatedWebcilSet s_relocatedWebcilBases;
const bool isHostProbedWebcil = IsWebcilFormat();
const TADDR webcilBase = isHostProbedWebcil ? (TADDR)GetBase() : (TADDR)0;
The peimagelayout.inl change routes PEImageLayout::IsComponentAssembly() through DECODER_DISPATCH to WebcilDecoder::IsComponentAssembly(), but that was stubbed to return FALSE, leaving the dispatch inert: readytoruninfo.cpp gates composite R2R load on IsComponentAssembly(), so every webcil composite was reported as non-component. Implement it from the R2R COMPONENT flag, matching the PE decoder path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae
|
Added The Note Comment drafted with GitHub Copilot assistance. |
davidwrighton
left a comment
There was a problem hiding this comment.
Looks pretty good. just a few more things to fix.
Resolves davidwrighton's review feedback on the flat webcil composite R2R load:
- nativeimage.cpp: on WASM, don't call GetExport("RTR_HEADER") at all. PE R2R
images that rely on that export can't be loaded on WASM, so obtain the R2R
header from the decoder via HasReadyToRunHeader()/GetReadyToRunHeader(). The
NULL-header check still fails genuinely non-R2R images.
- peimagelayout.cpp: make the webcil relocation de-dup MT-safe. Take a new
CrstWebcilImageRelocation lock before the Contains check and hold it for the
rest of ApplyBaseRelocations, so concurrent loads of the same shared buffer
(WASM can be built with FEATURE_MULTITHREADING) can't both relocate it.
Record the base right after the check rather than at method end, and drop the
misleading "throw-safety" comment: a throw partway through relocation is
unrecoverable for the additive relocation model regardless of when we record.
- peimage.cpp / peimagelayout.{h,cpp}: add PEImageLayout::Startup() to Init the
new Crst, called from PEImage::Startup().
- CrstTypes.def / crsttypes_generated.h: add the CrstWebcilImageRelocation type
(Unordered leaf), regenerated via CrstTypeTool.
Validated with a browser-wasm coreclr build (clr subset): peimagelayout.cpp,
nativeimage.cpp, and peimage.cpp compile clean (0 warnings, 0 errors).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53517266-f0ba-4d8e-9fa2-a9e6fc16753b
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/coreclr/vm/nativeimage.cpp:219
- On TARGET_WASM this path now uses the decoder header whenever
HasReadyToRunHeader()is true, regardless of image format. That means a PE composite that is missing the expectedRTR_HEADERexport could be silently accepted on WASM (sinceHasReadyToRunHeader()also works for PE), weakening validation beyond the stated webcil-only intent. Gate the decoder-based header lookup onIsWebcilFormat()and keep the export path for PE layouts.
#ifdef TARGET_WASM
// On WebAssembly the runtime only loads flat webcil composites, which do not expose a named
// "RTR_HEADER" export the way PE R2R images do; obtain the R2R header from the decoder instead.
// PE R2R images (which rely on the export) cannot be loaded on WASM, so the export path is never
// taken here. A genuinely non-R2R image still fails validation below via the NULL header check.
if (peLoadedImage->HasReadyToRunHeader())
*header = peLoadedImage->GetReadyToRunHeader();
#else // TARGET_WASM
*header = (READYTORUN_HEADER *)peLoadedImage->GetExport("RTR_HEADER");
#endif // TARGET_WASM
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
/ba-g remaining jobs are stuck infra |
…otnet#131016) ## What Minimal, OS-neutral VM plumbing so the runtime can **load a flat webcil-in-wasm composite R2R image**. crossgen2 emits the wasm startup R2R composite as a flat webcil image, but the R2R/PE-decoder load path assumes a mapped PE — a named `RTR_HEADER` export and `IsMapped()` RVA addressing — so today the VM can't find the R2R header or address sections in a flat webcil composite. Four small touch-points close that gap: | File | Change | |---|---| | `vm/nativeimage.cpp` | webcil composite exposes no named `RTR_HEADER` export → fall back to the decoder's R2R header | | `vm/readytoruninfo.cpp` | treat flat webcil as RVA-addressable directly from base (`IsWebcilFormat()` alongside `IsMapped()`) | | `vm/peimagelayout.inl` | route `IsComponentAssembly()` through the webcil decoder (`DECODER_DISPATCH`) | | `vm/peimagelayout.cpp` | guard against re-applying base relocations to a host-probed shared buffer | Guarded on `TARGET_WASM` / `FEATURE_PORTABLE_ENTRYPOINTS`, so the **same code serves browser and wasi** from one path. ## Why Part of dotnet#130524. Addresses the composite-loadability open question in dotnet#130519 ("whether the composite container is fully browser-loadable today or needs work"). This is the VM-*consumer* half — the *producer*/packaging side stays with dotnet#130519. ## Feedback wanted — the relocation guard The one design question in this PR is the `peimagelayout.cpp` guard. Host-probed webcil R2R images are backed by a **single shared in-memory buffer**, not a copy-on-write file mapping. The assembly binder can open such an image more than once (identity probe + load) as **independent** `PEImage`/layout objects over the *same* buffer, and each open would re-run `ApplyBaseRelocations` on that memory. WASM table-index relocations are additive (`index += tableBase`), so a second application doubles `tableBase` and pushes indirect-call indices out of the function table. This PR guards it with a VM-local set of already-relocated base addresses (`SetSHash<TADDR>`). Because the duplicate opens are distinct objects sharing one buffer, a per-`PEImageLayout`/per-`PEImage` flag wouldn't dedup them — the state has to key on the shared resource. It is intentionally **not** synchronized (relies on the current single-threaded wasm runtime) and is marked INTERIM in-code. The cleaner production direction is probably to **relocate the host-probed buffer once at the point it's handed to the runtime and skip webcil in `ApplyBaseRelocations` entirely** — but that reaches into the host/probe boundary, which is out of scope for this VM-only load change. Guidance on the preferred layer here is the main thing I'm looking for. ## Scope / out of scope **Out:** composite *production*/packaging (dotnet#130519); the CoreLib-R2R execution gates — QCall/PInvoke, class-init, etc. (dotnet#129850); SIMD instruction-set reporting; the host-side merge/probe mechanism. ## Testing - Compiles under `./build.sh clr -os wasi -c Release` (the `TARGET_WASM` branches are only exercised by a wasm build). - Validated on the WASI R2R prototype: a composite `System.Private.CoreLib` R2R image (~49k R2R functions) loads and dispatches through exactly these four changes. - **Cannot run standalone yet** — a loaded CoreLib R2R image needs the execution gates tracked in dotnet#129850 to get past startup. This PR is the load prerequisite for that work, landing incrementally behind the wasm guards like the rest of the wasm R2R bring-up. > [!NOTE] > This PR (code and description) was drafted with GitHub Copilot assistance. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2902374a-f352-4c46-bcec-8a35c97733ae Copilot-Session: 53517266-f0ba-4d8e-9fa2-a9e6fc16753b
What
Minimal, OS-neutral VM plumbing so the runtime can load a flat webcil-in-wasm composite R2R image. crossgen2 emits the wasm startup R2R composite as a flat webcil image, but the R2R/PE-decoder load path assumes a mapped PE — a named
RTR_HEADERexport andIsMapped()RVA addressing — so today the VM can't find the R2R header or address sections in a flat webcil composite. Four small touch-points close that gap:vm/nativeimage.cppRTR_HEADERexport → fall back to the decoder's R2R headervm/readytoruninfo.cppIsWebcilFormat()alongsideIsMapped())vm/peimagelayout.inlIsComponentAssembly()through the webcil decoder (DECODER_DISPATCH)vm/peimagelayout.cppGuarded on
TARGET_WASM/FEATURE_PORTABLE_ENTRYPOINTS, so the same code serves browser and wasi from one path.Why
Part of #130524. Addresses the composite-loadability open question in #130519 ("whether the composite container is fully browser-loadable today or needs work"). This is the VM-consumer half — the producer/packaging side stays with #130519.
Feedback wanted — the relocation guard
The one design question in this PR is the
peimagelayout.cppguard. Host-probed webcil R2R images are backed by a single shared in-memory buffer, not a copy-on-write file mapping. The assembly binder can open such an image more than once (identity probe + load) as independentPEImage/layout objects over the same buffer, and each open would re-runApplyBaseRelocationson that memory. WASM table-index relocations are additive (index += tableBase), so a second application doublestableBaseand pushes indirect-call indices out of the function table.This PR guards it with a VM-local set of already-relocated base addresses (
SetSHash<TADDR>). Because the duplicate opens are distinct objects sharing one buffer, a per-PEImageLayout/per-PEImageflag wouldn't dedup them — the state has to key on the shared resource. It is intentionally not synchronized (relies on the current single-threaded wasm runtime) and is marked INTERIM in-code.The cleaner production direction is probably to relocate the host-probed buffer once at the point it's handed to the runtime and skip webcil in
ApplyBaseRelocationsentirely — but that reaches into the host/probe boundary, which is out of scope for this VM-only load change. Guidance on the preferred layer here is the main thing I'm looking for.Scope / out of scope
Out: composite production/packaging (#130519); the CoreLib-R2R execution gates — QCall/PInvoke, class-init, etc. (#129850); SIMD instruction-set reporting; the host-side merge/probe mechanism.
Testing
./build.sh clr -os wasi -c Release(theTARGET_WASMbranches are only exercised by a wasm build).System.Private.CoreLibR2R image (~49k R2R functions) loads and dispatches through exactly these four changes.Note
This PR (code and description) was drafted with GitHub Copilot assistance.