Retarget x64 CI to the NVMe 1ES pool and images - #142
Draft
David-Engel wants to merge 7 commits into
Draft
Conversation
Switch every x64 Windows/Linux job in the Rust CI pipelines from the RUST-1ES-POOL-WUS3 pool (RUST-1ES-UBUSLIM / RUST-Win22-Sql25-1P images) to the NVMe-capable RUST-1ES-POOL-WUS3-NVME pool and the RUST-1ES-UBUSLIM-NVME / RUST-Win22-Sql25-1P-NVME images. The old pool and images are being retired, so the NVMe pool becomes the default everywhere rather than an opt-in override. DRY refactor for the two files with many references: - validation-stages.yml: adds poolName / linuxImage / windowsImage parameters (NVMe defaults) as a single source of truth; every inline x64 pool block now references them. - sync-container-images.yml: adds poolName / imageName pipeline variables; every job pool block now references them. All other pipelines (OneBranch build, benchmark, sync-github-main-to-stable) and the shared templates (sql-host, kerberos, test-matrix, test-matrix-alpine, private-link) are renamed directly to the NVMe names. ARM jobs (RUST-1ES-POOL-ARM-WUS3, RUST-UBUNTU-ARM64, RUST-WINSRV-ARM) are unchanged - no NVMe ARM images exist.
Temporarily reduces .pipeline/validation-pipeline.yml (the PR validation entry, def "mssql-rs Pull request validation") to a single DiskInfo stage with a Linux and a Windows job that print disk hardware/layout and exit — no build/test — to confirm the NVMe disk topology cheaply. Also enriches the reusable diagnostics for when full validation is restored: - print-disk-info.sh: lsblk TRAN/ROTA, /dev/nvme*, per-mount transport. - PrintAgentDebugInfo.ps1: Get-PhysicalDisk BusType, Get-Disk, Get-Volume. - validation-stages.yml Build_Linux: run print-disk-info.sh. REVERT: restore the validation-stages.yml template call in validation-pipeline.yml (kept commented at the bottom) and drop the DiskInfo stage.
Disk fix verified on rebuilt images (Linux root 194G / Windows C: 200G, both local NVMe). Restore the full validation-stages.yml matrix. The reusable disk diagnostics (print-disk-info.sh, PrintAgentDebugInfo.ps1, Build_Linux disk step) are kept.
The SSRP named-instance test (test_ssrp_named_instance_resolution) connects to HOST\SQLDEV as sa. sql-setup-template.yml only configured SQLDEV's sa password when the MSSQL$SQLDEV service was already Running, and silently skipped otherwise. On freshly-built images the service can be stopped / manual-start at setup time, so sa was never set and the test failed with 'Login failed for user sa'. Now set the service to Automatic, start it, and wait for Running before configuring, instead of skipping.
Generate-SqlCertificates.ps1 used a bare Restart-Service, which fails intermittently on freshly-provisioned ephemeral NVMe images with 'cannot be stopped' (CouldNotStopService) because SQL Server is still in startup/recovery. Wait for a steady state, then stop/start with retries via Restart-SqlServiceSafely.
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. 🔗 Quick Links |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switches every x64 Windows/Linux job in the Rust CI pipelines from the
RUST-1ES-POOL-WUS3pool (RUST-1ES-UBUSLIM/RUST-Win22-Sql25-1Pimages) to the NVMe-capableRUST-1ES-POOL-WUS3-NVMEpool and theRUST-1ES-UBUSLIM-NVME/RUST-Win22-Sql25-1P-NVMEimages. The NVMe images/pool were newly provisioned (v6D4ads_v6SKU with a local NVMe temp disk); the old pool and images are being retired, so NVMe becomes the default everywhere rather than an opt-in override.DRY refactor (single source of truth per file)
validation-stages.yml— addspoolName/linuxImage/windowsImageparameters (NVMe defaults); every inline x64 pool block now references them, so a future retarget is a 1–3 line change.sync-container-images.yml— addspoolName/imageNamepipeline variables; all 11 job pool blocks now reference them.Straight rename
OneBranch/stages.yml,benchmark-pipeline.yml,sync-github-main-to-stable.ymlsql-host-template.yml,kerberos-test-template.yml,test-matrix-template.yml,test-matrix-template-alpine.yml,private-link-smoke-template.ymlrun-benchmarks.ps1comment andarm-sql-host-design.mdUnchanged (intentional)
ARM jobs (
RUST-1ES-POOL-ARM-WUS3,RUST-UBUNTU-ARM64,RUST-WINSRV-ARM) — no NVMe ARM images exist.Linked work item
AB#46593
Testing
yaml.safe_load.Breaking changes / migration notes
The NVMe pool uses a 4-vCPU
D4ads_v6SKU vs. the previous 8-vCPU pool. Functionally equivalent, but absolute perf/benchmark numbers will shift; PR-vs-target benchmark comparisons stay valid since both sides run on the same pool. The 1ES images/pool/subnet themselves are provisioned via Bicep that is intentionally kept out of this public repo (contains internal subscription/tenant IDs).