Start install downloads before local-only checks - #23381
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimises brew install by overlapping network transfers (bottle manifests and, once confirmed, bottles) with local-only preinstall work, improving wall-clock install time and making the reordering visible via phase timing instrumentation.
Changes:
- Start enqueuing bottle manifest downloads as soon as formula installers are constructed (including ask-mode), so manifest downloads run concurrently.
- For confirmed installs, enqueue the formula’s bottle download earlier in
FormulaInstaller#prelude_fetchand avoid requeueing it later inenqueue_fetch. - Add a dedicated
preinstall_checksphase timing and expand/adjust unit tests to cover the new queueing behaviour.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| Library/Homebrew/utils/phase_timings.rb | Instruments perform_preinstall_checks_once as a separate preinstall_checks phase. |
| Library/Homebrew/test/formula_installer_spec.rb | Adds coverage for early bottle enqueueing and ensures enqueue_fetch doesn’t requeue an already-enqueued bottle. |
| Library/Homebrew/test/cmd/install_spec.rb | Updates expectations for the new prelude fetch behaviour and adds an ask-mode queue drain test. |
| Library/Homebrew/install.rb | Adds a metadata_only path to prelude_fetch_formulae to support ask-mode manifest-only prefetching. |
| Library/Homebrew/formula_installer.rb | Splits metadata-only vs full prelude fetch work; enqueues bottle early and avoids duplicate enqueue/reporting later. |
| Library/Homebrew/cmd/install.rb | Reorders flow so early prefetch can overlap local checks; drains manifest downloads before printing the ask-mode plan. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
MikeMcQuaid
force-pushed
the
install-earlier-downloads
branch
3 times, most recently
from
July 31, 2026 19:53
9e70a6c to
2210c2a
Compare
MikeMcQuaid
enabled auto-merge
August 1, 2026 13:30
p-linnane
approved these changes
Aug 2, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Aug 2, 2026
- `brew install` only started network transfers after preinstall checks and, in ask mode, fetched each bottle manifest serially while computing the dry-run plan. - Enqueue bottle manifests on the shared download queue right after building formula installers in both ask and no-ask modes so transfers overlap preinstall checks and dependant scanning and manifests for multiple formulae download concurrently. Ask mode drains them under a `Downloading bottle manifests` heading before printing the plan; warm runs enqueue nothing and stay silent. - Once downloads are confirmed (`--yes` installs, reinstall, upgrade and other `Install.fetch_formulae` callers), also enqueue the formula's own bottle in `FormulaInstaller#prelude_fetch`: the blob URL needs neither the manifest nor dependency resolution, so both transfer concurrently and staging joins the same queue cycle. - Skip the `enqueue_fetch` requeue for bottles the prelude fetch already enqueued so a completed early download is not reported a second time. - Give `DownloadQueue#fetch` `only:` and `heading:` so a heading is always printed before any queue output and never for empty fetches: dependency resolution inside `Install.fetch_formulae` waits on just the bottle manifests it needs (under `Downloading bottle manifests`) and the cask source pre-fetch on just its cask files, keeping other in-flight downloads queued and unreported so bottles only ever appear under the `Fetching downloads for:` heading, which now prints lazily from the fetch that reports them (`brew upgrade` enqueues before it knows that heading's contents). This replaces `Install.show_combined_fetch_downloads_heading`, `brew upgrade`'s manual manifest heading predicate and its dead `show_downloads_heading` plumbing. - Instrument `Install.perform_preinstall_checks_once` as a `preinstall_checks` phase timing to keep the reordering visible. - Archive-cold `brew install hello`: the first transfer starts at ~375ms instead of ~442ms, the bottle no longer waits for the manifest round trip (~672ms before, ~375ms now) and `--yes` wall time drops around 30%. Two-formula ask-mode installs drop one full manifest round trip (~1240ms to ~1030ms).
MikeMcQuaid
force-pushed
the
install-earlier-downloads
branch
from
August 3, 2026 07:54
2210c2a to
284cd0e
Compare
MikeMcQuaid
enabled auto-merge
August 3, 2026 07:57
Member
|
@MikeMcQuaid could it be that this caused the failure in https://github.com/Homebrew/homebrew-core/actions/runs/30799099950/job/91641096768?pr=296731? |
7 tasks
8 tasks
MikeMcQuaid
added a commit
that referenced
this pull request
Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on version-bump PRs: the bumped formula's stale bottle block derives its manifest URL from the new version, whose bottle has not been published yet, and `DownloadQueue#fetch` treats every bottle manifest failure as fatal. - Before that change the default ask-mode plan fetched manifests synchronously via `Formula#fetch_bottle_tab`, which rescues download errors so dependency resolution falls back to a full install, and the installer's memoisation then kept the queue from retrying the download. - Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are still reported but neither raise nor mark the fetch or run as failed. Use it for the metadata-only drains (`fetch_formulae`'s bottle manifest waits, `brew install`'s ask-mode drain and `brew upgrade`'s tab prefetch), restoring the synchronous path's tolerance. Manifest failures in fetches that pour bottles remain fatal.
MikeMcQuaid
added a commit
that referenced
this pull request
Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on version-bump PRs: the bumped formula's stale bottle block derives its manifest URL from the new version, whose bottle has not been published yet, and `DownloadQueue#fetch` treats every bottle manifest failure as fatal. - Before that change the default ask-mode plan fetched manifests synchronously via `Formula#fetch_bottle_tab`, which rescues download errors so dependency resolution falls back to a full install, and the installer's memoisation then kept the queue from retrying the download. - Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are still reported but neither raise nor mark the fetch or run as failed. Use it for the metadata-only drains (`fetch_formulae`'s bottle manifest waits, `brew install`'s ask-mode drain and `brew upgrade`'s tab prefetch), restoring the synchronous path's tolerance. Manifest failures in fetches that pour bottles remain fatal.
8 tasks
MikeMcQuaid
added a commit
that referenced
this pull request
Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on version-bump PRs: the bumped formula's stale bottle block derives its manifest URL from the new version, whose bottle has not been published yet, and `DownloadQueue#fetch` treats every bottle manifest failure as fatal. - Before that change the default ask-mode plan fetched manifests synchronously via `Formula#fetch_bottle_tab`, which rescues download errors so dependency resolution falls back to a full install, and the installer's memoisation then kept the queue from retrying the download. - Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are still reported but neither raise nor mark the fetch or run as failed. Use it for the metadata-only drains (`fetch_formulae`'s bottle manifest waits, `brew install`'s ask-mode drain and `brew upgrade`'s tab prefetch), restoring the synchronous path's tolerance. Manifest failures in fetches that pour bottles remain fatal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
brew installonly started network transfers after preinstall checks and, in ask mode, fetched each bottle manifest serially while computing the dry-run plan.Fetching bottle manifestsheading before printing the plan; warm runs enqueue nothing and stay silent.--yesinstalls, reinstall, upgrade and otherInstall.fetch_formulaecallers), also enqueue the formula's own bottle inFormulaInstaller#prelude_fetch: the blob URL needs neither the manifest nor dependency resolution, so both transfer concurrently and staging joins the same queue cycle.enqueue_fetchrequeue for bottles the prelude fetch already enqueued so a completed early download is not reported a second time.Install.perform_preinstall_checks_onceas apreinstall_checksphase timing to keep the reordering visible.brew install hello: the first transfer starts at ~375ms instead of ~442ms, the bottle no longer waits for the manifest round trip (~672ms before, ~375ms now) and--yeswall time drops around 30%. Two-formula ask-mode installs drop one full manifest round trip (~1240ms to ~1030ms).brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?Claude Fable 5 max with local review and testing.