In the review of #50327 it was noted (#50327 (comment)) that the bulk conversion does not consult a user-defined to_pylist override on ExtensionArray subclasses for nested children.
This was already inconsistent before #50327:
ListScalar.as_py called child_values.to_pylist(), so an override on a list child's class was honored;
StructScalar.as_py converted fields via per-element Scalars, so an override on a struct child's class was ignored.
Top-level overrides dispatch normally through Python method resolution in both the old and the new code, and ExtensionScalar.as_py / __arrow_ext_scalar_class__ — the documented customization point — is honored on all paths.
This issue is to decide the contract:
- document that nested
to_pylist overrides are not consulted (only ExtensionScalar.as_py is), or
- detect custom
to_pylist overrides and route nested conversion through them (sketch in the linked discussion).
In the review of #50327 it was noted (#50327 (comment)) that the bulk conversion does not consult a user-defined
to_pylistoverride onExtensionArraysubclasses for nested children.This was already inconsistent before #50327:
ListScalar.as_pycalledchild_values.to_pylist(), so an override on a list child's class was honored;StructScalar.as_pyconverted fields via per-element Scalars, so an override on a struct child's class was ignored.Top-level overrides dispatch normally through Python method resolution in both the old and the new code, and
ExtensionScalar.as_py/__arrow_ext_scalar_class__— the documented customization point — is honored on all paths.This issue is to decide the contract:
to_pylistoverrides are not consulted (onlyExtensionScalar.as_pyis), orto_pylistoverrides and route nested conversion through them (sketch in the linked discussion).