Parallelize opt-in Scroll pushes

[IOHelpMe]

Summary

• Add opt-in bounded parallel publication through SCROLL_PUSH_JOBS.
• Keep the default at SCROLL_PUSH_JOBS=1, so release and PR publication remain serial unless a caller explicitly opts in.
• Preserve the explicit catalog and enforce a complete category phase before any artifact push starts.
• Add CI coverage for deterministic dry runs, the serial default, concurrency bounds, the category barrier, complete job reaping, and failure propagation.

The local druid-local orchestration opts into four workers (SCROLL_PUSH_JOBS=4). Production release/CI behavior in this repository remains unchanged at one worker.

Why

Local seeding publishes 18 categories and 94 artifacts to Harbor. These independent registry pushes were performed serially, so registry/network latency dominated setup time. A bounded worker pool overlaps only those independent pushes; it does not omit validation, categories, artifacts, database seeding, or any other setup step.

Measured impact

Measured on the same Windows/WSL local Harbor stack with the complete 112-item catalog:

Mode	Time
Serial (SCROLL_PUSH_JOBS=1)	79.5 s
Four workers (SCROLL_PUSH_JOBS=4)	25.4 s
Savings	54.1 s (68.1%)

Safety and failure handling

• Reject zero, negative, and non-integer job counts before invoking druid.
• Never exceed the configured worker count.
• Reap every started background process before returning.
• Return a failure if any push in the phase fails.
• Prevent artifact publication when the category phase fails.
• Preserve deterministic catalog output in dry-run mode.

Validation

• bash -n scripts/push.sh scripts/tests/push-parallel.test.sh scripts/tests/fixtures/fake-druid.sh
• bash ./scripts/tests/push-parallel.test.sh
• BASH_COMPAT=32 bash ./scripts/tests/push-parallel.test.sh
• ./scripts/validate_all_scrolls.sh
• Real local seed against Harbor: all 18 categories and 94 artifacts published successfully with four workers.

[druid-infra]

Error: This repo is not allowlisted for Atlantis.

[IOHelpMe]

CI-Nachbesserung 776f891: Der erste Run scheiterte nicht an der Parallelisierung, Registry oder Fork-Secrets, sondern an der Test-Pipeline grep | head unter set -o pipefail. head schloss die Pipe früh; grep erhielt EPIPE und lieferte Exit 2. Der Test nutzt nun grep -m 1 und beendet den Producer selbst sauber. Lokal erneut grün: make build-tree, vollständige Scroll-Validierung und der Paralleltest mit exakt 18 Kategorien sowie 94 Artefakten; Fehlerweitergabe und vollständiges Reaping bleiben abgedeckt. Der Push hat einen neuen GitHub-Run ausgelöst.

[IOHelpMe]

Finaler Status für 776f891: Der erneute build-deploy-Run ist vollständig SUCCESS. Damit sind Build-Tree, vollständige Scroll-Validierung und der bounded-parallelism-Test inklusive 18 Kategorien, 94 Artefakten, Fehlerweitergabe und vollständigem Worker-Reaping auch in GitHub CI grün.