Skip to content

musl release asset ships without tree-sitter grammars — silently falls back to line-based chunking #83

Description

@amondnet

Summary

The csp-linux-x64-musl asset of v0.1.9 cannot load any tree-sitter grammar at runtime, so it falls back to line-based chunking. The csp-linux-x64 (gnu) asset from the same release loads them fine. Search still returns the right files, but chunks are cut at line boundaries instead of syntax boundaries, so results are noticeably worse.

Reproduce

Same query, same corpus (a small TypeScript project), the two Linux x64 assets of v0.1.9:

# musl asset — on Ubuntu 22.04
csp search "webhook signature verification" ./src --top-k 1
csp: could not load tree-sitter grammar for 'typescript': Language 'typescript' not found. Falling back to line-based chunking for this language.
{"query":"...","results":[{"chunk":{"location":"webhook.ts:31-56", ...
# gnu asset — on Ubuntu 24.04, no warning
csp search "webhook signature verification" ./src --top-k 1
{"query":"...","results":[{"chunk":{"location":"webhook.ts:27-33", ...
asset warning top hit
csp-linux-x64 (gnu) none webhook.ts:27-33 — the doc comment plus the function signature, a syntax-bounded unit
csp-linux-x64-musl could not load tree-sitter grammar webhook.ts:31-56 — starts mid-doc-comment and stops mid-function body

Assets used (v0.1.9):

  • csp-linux-x64-musl sha256 e7023df4333555b908a0706cf788c3c42130ece74cb1a554be9bf7376a211899
  • csp-linux-x64 sha256 f4c94645fe0e3aef6e79c61f8b0c0c53daa4647c74f69efe095f243c9c2067ce

Expected

The musl asset should register the same grammars as the gnu asset. It is the only Linux asset that runs on hosts with glibc < 2.39 (the gnu assets are built on ubuntu-latest = 24.04 and fail to load on Ubuntu 22.04 with version 'GLIBC_2.39' not found), so on those hosts the musl build is the only option — and it is the degraded one.

Possible cause

.github/workflows/release-rust.yml builds the two targets through different toolchains:

if [[ "${{ matrix.target }}" == *-musl ]]; then
  cargo zigbuild --release --locked -p code-search-please --target "${{ matrix.target }}"
else
  cargo build --release --locked -p code-search-please --target "${{ matrix.target }}"
fi

Same package, same default features — so this looks like a link-time rather than a feature-flag difference. If grammars are registered through life-before-main constructors or a custom link section (the inventory/ctor pattern), a fully static musl link can drop or never run them, which would present exactly like this: the binary works, the registry is empty, every lookup misses. Worth checking whether the grammar registry is populated at startup in the musl binary, and whether --no-gc-sections or an explicit registration list changes it.

Not a duplicate of

Environment

  • csp 0.1.9
  • musl asset run on cloudflare/sandbox:0.12.4 (Ubuntu 22.04.5, glibc 2.35), linux/amd64
  • gnu asset run on ubuntu:24.04 (glibc 2.39), linux/amd64

Context

Found while baking csp into a code-review bot's container image. That image is built on the Cloudflare Sandbox base (Ubuntu 22.04), so it has to use the musl asset.

Metadata

Metadata

Assignees

No one assigned

    Labels

    p2Priority 2 - Mediumtype:bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions