macOS arm64 wheel ships only ISA-L base functions (no NEON/PMULL assembly), ~5x slower than the manylinux aarch64 wheel on the same CPU

[geoff-davis]

Summary

The isal 1.8.0 macOS arm64 wheel from PyPI appears to be built without any of ISA-L's optimized aarch64 assembly. Every hot function resolves to the plain-C _base implementation, and measured throughput is ~4–6x below the manylinux aarch64 wheel running on the same CPU (Apple M3, Linux container, no emulation).

Symbol evidence

macOS arm64 wheel (isal_zlib.cpython-312-darwin.so):

$ nm isal_zlib.cpython-312-darwin.so | grep -icE "neon|pmull"
0
$ nm isal_zlib.cpython-312-darwin.so | grep -E "crc32_gzip|deflate_body"
_crc32_gzip_refl
_crc32_gzip_refl_base
_isal_deflate_body
_isal_deflate_body_base

manylinux2014_aarch64 wheel, same version (isal-1.8.0-cp312-cp312-manylinux2014_aarch64...whl):

$ nm isal/isal_zlib.cpython-312-*.so | grep -iE "neon|pmull" | head
accum32_neon
adler32_neon
crc32_gzip_refl_pmull
crc32_ieee_norm_pmull
...

Measured impact (Apple M3, CPython 3.12, isal 1.8.0)

Gzip decompression of an 8 MiB realistic-entropy JSONL payload (37% compression ratio, compressed by stdlib zlib level 6):

wheel	decompress	crc32 (8 MiB)
macOS arm64 (host)	0.21 GB/s	0.48 GB/s
manylinux aarch64 (container on same machine)	2.40 GB/s	(not measured)

For comparison on the same host, stdlib zlib decompresses the same payload at 0.92 GB/s and zlib-ng at 0.63 GB/s — so the macOS wheel is also several times slower than every alternative it is meant to outperform.

Reproduce

import time, zlib
from isal import isal_zlib

data = open("/dev/urandom", "rb").read(4 * 1024 * 1024) + b"B" * (4 * 1024 * 1024)
c = zlib.compressobj(6, zlib.DEFLATED, 31)
payload = c.compress(data) + c.flush()

for name, mod in (("zlib", zlib), ("isal", isal_zlib)):
    best = float("inf")
    for _ in range(7):
        d = mod.decompressobj(wbits=31)
        t0 = time.perf_counter()
        d.decompress(payload)
        best = min(best, time.perf_counter() - t0)
    print(name, f"{len(data)/best/1e9:.2f} GB/s")

Environment

• isal 1.8.0 (PyPI wheel), CPython 3.12.8
• macOS (Darwin 25.5.0), Apple M3
• Wheel filename: isal-1.8.0-cp312-cp312-macosx_*_arm64.whl

Presumably the macOS build isn't assembling ISA-L's aarch64/*.S sources (they may need explicit handling with the Xcode toolchain), so the multibinary dispatchers only ever see the base functions. Related: #252 asks for arm64 to be listed as supported — worth noting that until this is fixed, the macOS arm64 wheel is likely slower than stdlib zlib for most workloads.

[rhpvorderman]

I just checked the build process and this is probably due to using make -f Makefile.unx rather than using autoconf and autotools.

Can you check the conda-forge build of ISA-L? https://github.com/conda-forge/isa-l-feedstock
That uses autoconf and autotools.

The igzip application included should be much faster than normal gzip. If not, something is really off.

[geoff-davis]

Checked the conda-forge build as suggested — your hypothesis is confirmed. All on the same Apple M3, same 8 MiB realistic-entropy fixture as above, python-isal 1.8.0 in both cases:

Symbols. conda-forge libisal.2.dylib (autotools build): 33 NEON/PMULL symbols (adler32_neon, crc32_gzip_refl_pmull, …); the conda-forge python extension dynamically links it (@rpath/libisal.2.dylib). The PyPI macOS arm64 wheel statically embeds the base-only build: 0 such symbols.

Throughput, conda-forge vs PyPI wheel:

	PyPI wheel	conda-forge	delta
isal_zlib decompress (gzip, 37% ratio)	0.21 GB/s	0.52 GB/s	2.5x
isal_zlib.crc32 (8 MiB)	0.48 GB/s	28.2 GB/s	59x

The crc32 delta is the clean PMULL signal, since it's pure checksum with no inflate logic in the way.

On the igzip canary: igzip -d -c from the conda-forge build measures ~2x slower than macOS system gzip -d -c on a 40 MiB fixture (0.08 s vs 0.04 s, output verified identical). That's not evidence of a bad build — Apple ships a hardware-accelerated zlib/gzip stack on Apple Silicon (its zlib.crc32 measures ~43 GB/s here), so "faster than normal gzip" is not a reliable health check on macOS arm64 specifically. The crc32 microbenchmark (~28 GB/s with PMULL vs ~0.5 GB/s base) is a much sharper one.

So: switching the macOS wheel build from make -f Makefile.unx to autotools (as conda-forge does) should recover the aarch64 assembly. Happy to test a candidate wheel if useful.