Bug: GarbageCollector deletes live files on Windows due to backslash/forward-slash path mismatch in file-protocol stores

[lochhh]

Bug Report

Description

On Windows, dj.gc.GarbageCollector.collect() treats every live row's storage file as orphaned, and dry_run=False deletes files that are still referenced by live rows.

Root cause

StorageBackend._full_path() (datajoint/storage.py, protocol == "file" branch) builds the path with:

​python return str(Path(location) / path) ​

str(Path(...)) renders OS-native separators (backslashes on Windows). But fsspec's LocalFileSystem.walk() (used by GarbageCollector.list_hash_paths/list_schema_paths) always returns POSIX-style forward-slash paths.

gc.py computes full_root = backend._full_path("") (backslash-separated) and then does file_path.replace(full_root, "") where file_path comes from fs.walk() (forward-slash). The .replace() silently no-ops on Windows, so the "relative" paths returned by list_hash_paths/list_schema_paths are actually still full absolute paths. These never match the (correctly relative) referenced-path set built from row metadata, so _is_covered() always returns False, i.e. every stored object looks orphaned.

Reproducibility

Include:

• WIN 11
• Python 3.12
• DataJoint Version (master branch) 2.3.1

The test introduced in 03436ff fails with

FAILED tests/unit/test_storage_adapter.py::TestStorageBackendPluginDelegation::test_file_protocol_full_path_uses_forward_slashes - AssertionError: assert 'data\blobs...\cd\hash123' == 'data/blobs/s...ab/cd/hash123'

[lochhh]

Correction to root cause

Apologies, turns out I was on v2.3.1, where gc.py computed relative paths via:
file_path.replace(full_root, "")
That is confirmed broken on Windows as described: backslash full_root vs forward-slash file_path → .replace() no-ops → _is_covered() always False → mass false-positive orphans.

This issue no longer exists on 2.3.2, given that gc.py's list_hash_paths/list_schema_paths/objectref.walk() now use index-based slicing:
relative_path = file_path[len(full_root):].lstrip("/")