perf: parallelize mage preflight checks (~39% faster)#7772
Open
jongio wants to merge 1 commit intoAzure:mainfrom
Open
perf: parallelize mage preflight checks (~39% faster)#7772jongio wants to merge 1 commit intoAzure:mainfrom
jongio wants to merge 1 commit intoAzure:mainfrom
Conversation
jongio
requested review from
JeffreyCA,
hemarina,
rajeshkamal5050,
tg-msft,
vhvb1989,
wbreza and
weikanglim
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR speeds up the developer inner loop by restructuring mage preflight into a two-wave execution model, running independent checks concurrently while keeping test execution serialized.
Changes:
- Parallelizes “Wave 1” preflight checks (format, fix, copyright, lint, cspell, build) using
wg.Go() - Keeps “Wave 2” tests sequential and skips them when the build check fails
- Adds command/script output capture helpers to support non-interleaved parallel output and a deterministic summary
jongio
force-pushed
the
feature/faster-preflight
branch
from
66dea45 to
d0fa070
Compare
jongio
changed the title
Reorganize preflight into two waves, both parallelized: - Wave 1 (parallel): gofmt, go fix, copyright, lint, cspell, build Build step also pre-builds azd + azd-record binaries - Wave 2 (parallel): unit tests + playback tests run concurrently using CLI_TEST_SKIP_BUILD=true since binaries are pre-built Adds runCaptureAll, runCaptureShellScript, and runPlaybackTestsWithEnv helpers. Summary prints in deterministic order. Closes Azure#7766 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jongio
force-pushed
the
feature/faster-preflight
branch
from
d0fa070 to
13b695e
Compare
Collaborator
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Parallelizes
mage preflightfor a ~39% speedup by running checks AND tests concurrently.Before/After
Design
Wave 1 (parallel): gofmt, go fix, copyright, golangci-lint, cspell (Go), cspell (misc), go build — all 7 run concurrently. The build step also pre-builds both
azdandazd-recordbinaries.Wave 2 (parallel): Unit tests and playback tests run concurrently with
CLI_TEST_SKIP_BUILD=true, using the pre-built binaries from Wave 1. Previously these were sequential due to a shared binary build race — pre-building eliminates that.Changes
Preflight()to usewg.Go()(Go 1.26) for both wavesazd+azd-recordbinaries in Wave 1 build stepCLI_TEST_SKIP_BUILD=truefor Wave 2 to prevent test binary rebuild racesrunCaptureAll,runCaptureShellScript,runPlaybackTestsWithEnvhelpersCloses #7766