CI: upgrade PyTorch images to ROCm 7.14 - #989
Merged
Merged
Conversation
Use pinned PyTorch 2.12 release images for test and publish workflows, and invalidate the MLIR cache for the new runtime. Co-authored-by: Cursor <cursoragent@cursor.com>
coderfeli
added
multi-gpu
performance
The PyTorch 2.12 runtime image omits HIP headers and CMake metadata, so initialize the matching ROCm devel package and expose its toolchain paths. Co-authored-by: Cursor <cursoragent@cursor.com>
Avoid replacing the PyTorch 2.12 image's matched Triton build with a ROCm 7.2 binary that segfaults when loaded after PyTorch. Co-authored-by: Cursor <cursoragent@cursor.com>
Build an AMD-only MLIR toolchain once, then package current and baseline FlyDSL wheels with the required tools for reuse across GPU jobs. This avoids repeated 2 GB MLIR downloads, preserves benchmark and FileCheck coverage, and lets trusted PR runs cache cold LLVM builds.
Phil-amd
force-pushed
the
ci/upgrade-rocm-7.14
branch
from
August 11, 2026 01:35
197f7f9 to
30ff7ca
Compare
Make baseline artifacts optional, prune stale worktrees, and validate ROCDL code generation and transported tools before GPU tests. Keep the cold LLVM checkout a plain shallow fetch. A blob-filtered sparse checkout made the server build an uncached pack for an arbitrary SHA and then lazily re-fetched the tree during checkout: measured on CI, `--depth 1` alone takes ~100s while `--depth 1 --filter=blob:none` did not finish within 100 minutes. Discard a leftover promisor clone instead of unsetting its config, which would leave the missing blobs unrecoverable. Cap the LLVM build step at 45 minutes so a stall releases the GPU runner early. Centralize build profiles, ROCm paths, and Python version discovery. Compress artifacts and keep multi-GPU jobs tied to their producer.
Phil-amd
force-pushed
the
ci/upgrade-rocm-7.14
branch
from
August 11, 2026 02:32
30ff7ca to
4e1a69d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use pinned PyTorch 2.12 release images for test and publish workflows, and invalidate the MLIR cache for the new runtime.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist