Skip to content

Start install downloads before local-only checks - #23381

Merged
MikeMcQuaid merged 1 commit into
mainfrom
install-earlier-downloads
Aug 3, 2026
Merged

Start install downloads before local-only checks#23381
MikeMcQuaid merged 1 commit into
mainfrom
install-earlier-downloads

Conversation

@MikeMcQuaid

Copy link
Copy Markdown
Member
  • brew install only started network transfers after preinstall checks and, in ask mode, fetched each bottle manifest serially while computing the dry-run plan.
  • Enqueue bottle manifests on the shared download queue right after building formula installers in both ask and no-ask modes so transfers overlap preinstall checks and dependant scanning and manifests for multiple formulae download concurrently. Ask mode drains them under a Fetching bottle manifests heading before printing the plan; warm runs enqueue nothing and stay silent.
  • Once downloads are confirmed (--yes installs, reinstall, upgrade and other Install.fetch_formulae callers), also enqueue the formula's own bottle in FormulaInstaller#prelude_fetch: the blob URL needs neither the manifest nor dependency resolution, so both transfer concurrently and staging joins the same queue cycle.
  • Skip the enqueue_fetch requeue for bottles the prelude fetch already enqueued so a completed early download is not reported a second time.
  • Instrument Install.perform_preinstall_checks_once as a preinstall_checks phase timing to keep the reordering visible.
  • Archive-cold brew install hello: the first transfer starts at ~375ms instead of ~442ms, the bottle no longer waits for the manifest round trip (~672ms before, ~375ms now) and --yes wall time drops around 30%. Two-formula ask-mode installs drop one full manifest round trip (~1240ms to ~1030ms).

  • Have you followed our Contributing guidelines?
  • Have you checked for other open Pull Requests for the same change?
  • Have you explained what your changes do? Performance claims (e.g. "this is faster") must include Hyperfine benchmarks.
  • Have you explained why you'd like these changes included, not just what they do?
  • For bug fixes, have you given step-by-step brew commands to reproduce the bug?
  • Have you written new tests (excluding integration tests)? Here's an example.
  • Have you successfully run brew lgtm (style, typechecking and tests) locally?

  • I did not use AI/LLM to create this PR, or I disclosed the tool/model below and reviewed its output; I did not attribute commits to AI and will answer maintainer questions and review comments myself without AI/LLM.

Claude Fable 5 max with local review and testing.


Copilot AI review requested due to automatic review settings July 31, 2026 15:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimises brew install by overlapping network transfers (bottle manifests and, once confirmed, bottles) with local-only preinstall work, improving wall-clock install time and making the reordering visible via phase timing instrumentation.

Changes:

  • Start enqueuing bottle manifest downloads as soon as formula installers are constructed (including ask-mode), so manifest downloads run concurrently.
  • For confirmed installs, enqueue the formula’s bottle download earlier in FormulaInstaller#prelude_fetch and avoid requeueing it later in enqueue_fetch.
  • Add a dedicated preinstall_checks phase timing and expand/adjust unit tests to cover the new queueing behaviour.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Library/Homebrew/utils/phase_timings.rb Instruments perform_preinstall_checks_once as a separate preinstall_checks phase.
Library/Homebrew/test/formula_installer_spec.rb Adds coverage for early bottle enqueueing and ensures enqueue_fetch doesn’t requeue an already-enqueued bottle.
Library/Homebrew/test/cmd/install_spec.rb Updates expectations for the new prelude fetch behaviour and adds an ask-mode queue drain test.
Library/Homebrew/install.rb Adds a metadata_only path to prelude_fetch_formulae to support ask-mode manifest-only prefetching.
Library/Homebrew/formula_installer.rb Splits metadata-only vs full prelude fetch work; enqueues bottle early and avoids duplicate enqueue/reporting later.
Library/Homebrew/cmd/install.rb Reorders flow so early prefetch can overlap local checks; drains manifest downloads before printing the ask-mode plan.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Library/Homebrew/install.rb
Comment thread Library/Homebrew/test/cmd/install_spec.rb Outdated
@MikeMcQuaid
MikeMcQuaid force-pushed the install-earlier-downloads branch 3 times, most recently from 9e70a6c to 2210c2a Compare July 31, 2026 19:53
@MikeMcQuaid
MikeMcQuaid enabled auto-merge August 1, 2026 13:30
@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue Aug 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Aug 2, 2026
- `brew install` only started network transfers after preinstall
  checks and, in ask mode, fetched each bottle manifest serially
  while computing the dry-run plan.
- Enqueue bottle manifests on the shared download queue right after
  building formula installers in both ask and no-ask modes so
  transfers overlap preinstall checks and dependant scanning and
  manifests for multiple formulae download concurrently. Ask mode
  drains them under a `Downloading bottle manifests` heading before
  printing the plan; warm runs enqueue nothing and stay silent.
- Once downloads are confirmed (`--yes` installs, reinstall, upgrade
  and other `Install.fetch_formulae` callers), also enqueue the
  formula's own bottle in `FormulaInstaller#prelude_fetch`: the blob
  URL needs neither the manifest nor dependency resolution, so both
  transfer concurrently and staging joins the same queue cycle.
- Skip the `enqueue_fetch` requeue for bottles the prelude fetch
  already enqueued so a completed early download is not reported a
  second time.
- Give `DownloadQueue#fetch` `only:` and `heading:` so a heading is
  always printed before any queue output and never for empty fetches:
  dependency resolution inside `Install.fetch_formulae` waits on just
  the bottle manifests it needs (under `Downloading bottle manifests`)
  and the cask source pre-fetch on just its cask files, keeping other
  in-flight downloads queued and unreported so bottles only ever
  appear under the `Fetching downloads for:` heading, which now
  prints lazily from the fetch that reports them (`brew upgrade`
  enqueues before it knows that heading's contents). This replaces
  `Install.show_combined_fetch_downloads_heading`, `brew upgrade`'s
  manual manifest heading predicate and its dead
  `show_downloads_heading` plumbing.
- Instrument `Install.perform_preinstall_checks_once` as a
  `preinstall_checks` phase timing to keep the reordering visible.
- Archive-cold `brew install hello`: the first transfer starts at
  ~375ms instead of ~442ms, the bottle no longer waits for the
  manifest round trip (~672ms before, ~375ms now) and `--yes` wall
  time drops around 30%. Two-formula ask-mode installs drop one full
  manifest round trip (~1240ms to ~1030ms).
@MikeMcQuaid
MikeMcQuaid force-pushed the install-earlier-downloads branch from 2210c2a to 284cd0e Compare August 3, 2026 07:54
@MikeMcQuaid
MikeMcQuaid enabled auto-merge August 3, 2026 07:57
@MikeMcQuaid
MikeMcQuaid added this pull request to the merge queue Aug 3, 2026
Merged via the queue into main with commit d610afe Aug 3, 2026
45 checks passed
@MikeMcQuaid
MikeMcQuaid deleted the install-earlier-downloads branch August 3, 2026 08:39
@SMillerDev

Copy link
Copy Markdown
Member

MikeMcQuaid added a commit that referenced this pull request Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on
  version-bump PRs: the bumped formula's stale bottle block derives
  its manifest URL from the new version, whose bottle has not been
  published yet, and `DownloadQueue#fetch` treats every bottle
  manifest failure as fatal.
- Before that change the default ask-mode plan fetched manifests
  synchronously via `Formula#fetch_bottle_tab`, which rescues
  download errors so dependency resolution falls back to a full
  install, and the installer's memoisation then kept the queue from
  retrying the download.
- Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are
  still reported but neither raise nor mark the fetch or run as
  failed. Use it for the metadata-only drains (`fetch_formulae`'s
  bottle manifest waits, `brew install`'s ask-mode drain and
  `brew upgrade`'s tab prefetch), restoring the synchronous path's
  tolerance. Manifest failures in fetches that pour bottles remain
  fatal.
MikeMcQuaid added a commit that referenced this pull request Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on
  version-bump PRs: the bumped formula's stale bottle block derives
  its manifest URL from the new version, whose bottle has not been
  published yet, and `DownloadQueue#fetch` treats every bottle
  manifest failure as fatal.
- Before that change the default ask-mode plan fetched manifests
  synchronously via `Formula#fetch_bottle_tab`, which rescues
  download errors so dependency resolution falls back to a full
  install, and the installer's memoisation then kept the queue from
  retrying the download.
- Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are
  still reported but neither raise nor mark the fetch or run as
  failed. Use it for the metadata-only drains (`fetch_formulae`'s
  bottle manifest waits, `brew install`'s ask-mode drain and
  `brew upgrade`'s tab prefetch), restoring the synchronous path's
  tolerance. Manifest failures in fetches that pour bottles remain
  fatal.
MikeMcQuaid added a commit that referenced this pull request Aug 3, 2026
- After #23381, homebrew-core CI fails installing dependencies on
  version-bump PRs: the bumped formula's stale bottle block derives
  its manifest URL from the new version, whose bottle has not been
  published yet, and `DownloadQueue#fetch` treats every bottle
  manifest failure as fatal.
- Before that change the default ask-mode plan fetched manifests
  synchronously via `Formula#fetch_bottle_tab`, which rescues
  download errors so dependency resolution falls back to a full
  install, and the installer's memoisation then kept the queue from
  retrying the download.
- Add `DownloadQueue#fetch(allow_failures:)`: failed downloads are
  still reported but neither raise nor mark the fetch or run as
  failed. Use it for the metadata-only drains (`fetch_formulae`'s
  bottle manifest waits, `brew install`'s ask-mode drain and
  `brew upgrade`'s tab prefetch), restoring the synchronous path's
  tolerance. Manifest failures in fetches that pour bottles remain
  fatal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants