Reject sharded-checkpoint shard paths that escape the checkpoint folder

[AbdullahRasheed45]

Closes #4067 (the path-traversal half of the report)

The problem

load_checkpoint_in_model builds its list of shard files from the weight_map of a sharded checkpoint's *.index.json:

checkpoint_files = sorted(list(set(index.values())))
checkpoint_files = [os.path.join(checkpoint_folder, f) for f in checkpoint_files]

Those values come from whoever authored the checkpoint. os.path.join follows .. segments, and discards the folder entirely when the second argument is absolute, so a crafted index makes the loader open files outside the checkpoint directory. It is reachable through plain load_checkpoint_in_model(model, "<dir>") and load_checkpoint_and_dispatch(...) — no trust_remote_code, no opt-in flag.

Reproduced on main with a checkpoint whose weight_map points at ../outside/payload.safetensors, and again with an absolute path: both loaded the outside file successfully.

This is not remote code execution — safetensors goes through safe_open and the torch branch already passes weights_only=True. The impact is an out-of-directory file open, which doubles as an existence and parse oracle for paths on the host.

The change

Shard names are validated before they are joined: absolute paths are rejected, and the normalized result must stay inside the checkpoint folder.

The containment check deliberately compares normalized path strings rather than realpath. That distinction matters: the Hugging Face hub cache stores snapshots/<rev>/model.safetensors as a symlink into ../../blobs/<sha>, so a realpath-based check would resolve outside the snapshot directory and reject perfectly legitimate cached checkpoints. Normalizing the string blocks .. traversal without following symlinks.

Verification

Both attack vectors are now refused with a clear ValueError. I also checked the layouts that must keep working, since a fix here can easily be too strict:

layout	before	after
../outside/payload.safetensors	loads outside file	rejected
absolute path outside the folder	loads outside file	rejected
plain model-00001.safetensors	loads	loads
nested shards/model-00001.safetensors	loads	loads
hub-cache style symlink to ../../blobs/<sha>	loads	loads
./model-00001.safetensors	loads	loads

Tests added to tests/test_modeling_utils.py: one covering the escaping names (relative, nested-relative, absolute), one covering the nested-subdirectory and symlinked-shard layouts so the guard cannot be tightened into breaking the hub cache. The first fails on main.

pytest tests/test_modeling_utils.py → 43 passed, 3 skipped. ruff (0.13.1, as pinned in setup.py) format and check are clean.

Not covered here

The report also describes a DoS where a shard pointed at a FIFO makes torch.load block forever. That needs a separate decision about rejecting non-regular files, which has its own compatibility surface, so I left it out rather than bundling it. Happy to follow up if you'd like it handled the same way.

A previous attempt at this (#4070) was closed by the stale bot rather than on review, and the vulnerability is still present on main.