Skip to content

Define bounded lists#640

Open
Felix-El wants to merge 4 commits into
WebAssembly:mainfrom
Felix-El:main
Open

Define bounded lists#640
Felix-El wants to merge 4 commits into
WebAssembly:mainfrom
Felix-El:main

Conversation

@Felix-El

Copy link
Copy Markdown

Add bounded lists (list<T, ..N>)

Closes #385.

@cpetig cpetig left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly indentation doubts

Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated

@lukewagner lukewagner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This is looking generally good, a few comments:

Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/canonical-abi/definitions.py Outdated
Comment thread design/mvp/Binary.md Outdated
`none` case of an optional immediate.)
* 🔧 for fixed-sized lists the length of the list must be larger than 0 to pass
validation.
* 🔧 for fixed-sized lists (`0x67`) the length of the list must be larger than

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-existing, but it looks like the grammar already covers this twice: once by using <u32> (unsigned) and once with the (if maxlen > 0). We could also remove the (if maxlen > 0). But should we specify a maximum for maxlen?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For u32, zero is valid and the maxlen > 0 checks forbids zero. What would be a legitimate upper bound... i32::MAX?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already (just recently added) a MAX_LIST_BYTE_LENGTH = 228-1. That's just bytes, but even still, it seems like a reasonable upper bound.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, defined that bound in BINARY.md but where would we check this in definitions.py ?

@cpetig

cpetig commented Jul 2, 2026

Copy link
Copy Markdown

Associated with Implement WIT Fixed-Length-Lists

Felix-El and others added 3 commits July 14, 2026 08:27
Cherry-picked the essence of cpetig's commit d2874eb
from https://github.com/cpetig/component-model/tree/bounded-lists, adapted to
the current codebase (ptr_type/opts threading, updated class names).

Bounded strings are intentionally excluded.

Co-authored-by: Christof Petig <christof.petig@arcor.de>
- Add trap_if(actual_len > maybe_length) to lift_flat_list, mirroring
  the existing trap in load_list's heap path
- Add over-length trap tests for both flat and heap lifting
- Add alignment test for bounded list of U32 (verifies 3-byte padding
  after U8 length prefix)
- fix memory bounds checking
- improve integration with existing list load/store code
- fix indentation
- avoid default argument
- more readable load/store recipe
Replace the expanded-element flat representation with a pointer-based
one for direct fixed and bounded length lists. Add contains_direct_list
and type_has_direct_list_result helpers to detect when result types
contain such lists, forcing memory-based returns. Update test expectations
and add new roundtrip tests for the pointer-based flat ABI.
@Felix-El

Copy link
Copy Markdown
Author

Here is another shot at an ABI tweak for efficient passing of fixed/bounded lists. It is already implemented in the commit I just pushed. @cpetig 's ideas with refinement by discussion, my write-up; more versatile / less invasive compared to my previous proposal.


Flat ABI for Fixed and Bounded Length Lists

Motivation

The current flat ABI for fixed length lists expands all N element slots
into the core function's parameter/result list.
For large N this causes parameter overflow (exceeding
MAX_FLAT_PARAMS or MAX_FLAT_RESULTS), forcing an early fallback to
memory passing for the entire argument list when even one or two lists
are large.

If the current approach was naively adopted for bounded length lists
(which are still theory at this time), expanded form would even waste
flat-register space on uninitialized elements for partially-populated lists.

A second, independent motivation is that fixed and bounded length lists
are, unlike tuples, indexed at runtime and registers in core WebAssembly
cannot be indexed efficiently. Therefore, the expanded-element flat representation
forces register/memory (de)composition on either guest, adding to overhead.

This proposal replaces the expanded-element flat representation with a
pointer-based one for direct fixed and bounded length lists in parameters —
lists which are eligible for flat representation (not nested, immediately or
transitively, in another list).

Flat representation

Type Flat representation
list<T> (unbounded) [ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed) [ptr] — one pointer to N contiguous elements
list<T, ..N> (bounded) [ptr, len] — pointer to len contiguous elements + actual length

Memory representation

Type Memory representation
list<T> (unbounded) [ptr, len] — pointer to len contiguous elements + actual length (unchanged, for reference)
list<T, N> (fixed) [e0, e1, ...]N contiguous elements
list<T, ..N> (bounded) [len*, e0, e1, ...]N contiguous elements, uninitialized beyond actual length

bounded actual len: smallest integer type that can represent values up to (including) N

Lowering and Lifting

Parameters

Below explanation is for bounded length lists which carry a runtime length.
The flow is similar for fixed length lists - length just would not be passed
as it is statically known.

If all core params fit within MAX_FLAT_PARAMS, they are passed flat, otherwise
we fall back to a in-memory tuple representation (param area). This is not new.

  1. Caller

    • assume list L is already constructed in guest memory as L_ptr and L_len (from API use)
    • if passing flat: pass L_ptr and L_len directly, classic unbounded list ownership semantics
    • if passing in memory: move list elements inplace into param area
  2. Host, before call forwarded

    • if passing flat:
      • allocate twin list L2 in the exporting guest (optimization: bulk alloc, cache, etc.)
      • move elements between backing memories (import -> export)
      • pass L2_ptr and L2_len = L_len, classic unbounded list ownership semantics
    • if passing in memory:
      • move elements from inbound param area to outbound param area
  3. Callee

    • if passing flat: take L2_ptr and L2_len from core params
    • if passing in memory: read L2_len from param area, let L2_ptr point to first inplace element in param area
    • either way: construct list from L2_ptr, L2_len for API use

Results

Memory-passing is enforced when even a single direct fixed/bounded length list is in the result type -
that's a new second switchover reason beyond exceeding MAX_FLAT_RESULTS.
A return area (ret area) is used to forward all results as an in-memory tuple.

  1. Caller

    • result area is preallocated with enough room to hold all fixed/bounded length lists inline
    • result area pointer is passed as out core param
  2. Host, after call forwarded

    • copy inplace elements from returned export-side ret area into import-side ret area provided by out param
  3. Callee

    • construct the list by moving elements from ret area for API use

Fused allocation

When multiple direct lists exist among parameters and passing flat, the host might service
their backing (export) memory needs from a single allocation as an optimization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bounded lists and strings

4 participants