Repository navigation
Conversation
Omega is the primary component of GOMEGA and COMEGA cases, but there was no PES_SPEC_FILE entry for it, so these cases fell back to a generic 1-node layout. On pm-gpu with gnugpu, that is 4 MPI tasks (one per GPU) for all components, leaving MPAS-Seaice, the data models and the coupler on only 4 CPU cores. PEM tests then halved this to 2 tasks, for which there is no MPAS-Seaice partition file. Add components/omega/cime_config/config_pes.xml and point PES_SPEC_FILE to it. On pm-gpu, muller-gpu and alvarez-gpu, use 64 MPI tasks per node for the CPU components and 4 tasks with a stride of 16 for Omega, so that pm-gpu_set_affinity_npergpu.sh gives each Omega task its own GPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
xylar
changed the title
Author
TestingI ran these coupled tests on pm-gpu with gnugpu, at 48fecaa (before the Aurora commit):
In the EC30to60E2r2 🤖 Generated with Claude Code |
Author
|
Running Omega concurrently with MPAS-Seaice (by adding
🤖 Generated with Claude Code |
Author
Testing on auroraTested the merge of #671 (
aurora, intel, mpichPolaris
|
Author
Testing on pm-cpu and pm-gpuTested the merge of #671 (
pm-cpu, gnu, mpichPolaris
|
1 task done
xylar
force-pushed
the
omega/add-gomega-gpu-pe-layout
branch
from
October 31, 2026 18:04
6a671d9 to
48fecaa
Compare
xylar
changed the title
Author
|
I dropped the Aurora commit (6a671d9), so the branch is back at 48fecaa. On Aurora, intel and intelgpu share one machine name, and An Aurora layout should wait until CIME can match on compiler. That should be a small change, since Posted by Claude Code on @xylar's behalf. The testing, analysis and wording above are AI-authored; please check them accordingly. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a PE layout for Omega cases on pm-gpu that runs MPAS-Seaice, the data models and the coupler on 64 CPU tasks while Omega keeps one task per GPU.
GOMEGA and COMEGA cases had no PE layout, so on pm-gpu every component ran on 4 tasks, one per GPU.
PEMthen halved MPAS-Seaice to 2 tasks, which has no partition file (#654). On pm-gpu, muller-gpu and alvarez-gpu, this PR uses 64 MPI tasks per node and puts Omega on 4 of them, 16 apart, so the existingpm-gpu_set_affinity_npergpu.shgives each its own GPU. The pm-gpu ne256 coupled layouts do the same for EAMxx.For reviewers to consider:
gnu) build of Omega on pm-gpu also gets only 4 Omega tasks, sinceconfig_pes.xmlcan't select by compiler.ERSrun time is dominated by restart and history writes on one PIO task per node, which this doesn't change.xmllintrejectspstridagainstconfig_pes.xsd, as it does forconfig_pesall.xml.Fixes #654
Checklist
Documentation:
Linting
Building
Testing1
aurora, intel, mpich
aurora, intelgpu, mpich
frontier, craygnu, mpich
frontier, craygnu-mphipcc, mpich
pm-cpu, gnu, mpich
pm-gpu, gnugpu, mpich
Testingwith the following:have been run on and indicate that are all passing.
has passed, using the Polaris
e3sm_submodules/Omegabaseline-pfor both the baseline (Polarise3sm_submodules/Omega) and the PR build(such as
ERS_DorPEM_Dtests) have been run and indicate that they are all passing🤖 Generated with Claude Code
Footnotes
Remove rows for any configurations you do not run (or ask someone else to run).
At a minimum, PRs are expected to pass tests on at least one CPU and at least one GPU platform for both standalone tests (CTests and
omega_pr) and coupled CIME tests. ↩