TL/UCP: add onesided a2a peer ordering and fragmentation - #1345
wfaderhold21 wants to merge 1 commit into
Conversation
|
🤖 CI Triage Agent — TL;DR: The codestyle check failed because the commit title Full analysisSummary: The "codestyle" job's commit-title lint step exited 1 because the PR's commit message title is too long. Root cause: The commit-title validation in Implicated commit: File: Suggested fix: Reword the commit title to 50 characters or fewer while keeping a valid header prefix, then amend/force-push. Example (48 chars): Related: none
|
b20948d to
cc211ea
Compare
TEST/GTEST: add onesided alltoall ordering test - Fix single_onesided tune to @onesided (@1 selected bruck after the bruck algorithm was inserted into the enum) - Add test_alltoall_onesided.ordering covering seq/stride/ilv/full peer orders crossed with fragmentation (nfrags 1/4) - Give the onesided gtest context a simulated multi-node topology so ilv/full orders and the mixed GET/PUT progress path see a real local/remote split TL/UCP: remove ilv onesided a2a peer ordering - The 2026-09-10 pareto sweep (job 11769, 256 ranks) showed ilv dominated by both full and stride on bandwidth and congestion at every repeat and percent_bw: 82.17-82.53 GiB/s vs 84.65-85.89 at 4 MiB/peer, ~6% more CNP/s - Drop UCC_TL_UCP_ALLTOALL_ONESIDED_ORDER_ILV from the enum and the ALLTOALL_ONESIDED_ORDER config (now seq, stride, or full) - Simplify the mixed GET/PUT progress selection in ucc_tl_ucp_alltoall_onesided_init: AUTO + full alone triggers the mixed path (ilv shared the same local/remote split layout, so no dedicated branch existed in build_peer_order) - Shrink test_alltoall_onesided.ordering to seq/stride/full (6 cases); the ordering gtest passes 150/150 on thor - Rejecting ORDER=ilv now fails cleanly at config parse time No behavioral change to the remaining seq/stride/full layouts. TL/UCP: require split peer layout for mixed under seq/stride - alltoall_onesided_build_peer_order now takes the selected alg. The mixed GET/PUT progress path classifies local vs. remote by position in the order (the first/last n_local slots), so it requires the local/remote split layout. seq/stride with the mixed alg previously fell into the plain ring-walk layout, which leaves n_local = 0; the mixed progress path then treated every peer as remote and issued all ops as PUT, so the requested mixed (local-GET/remote-PUT) behavior silently degraded to all-PUT. They now get the split layout, which keeps the requested walk order within each class. - Move the pos/is_local/is_local_peer locals in mixed_progress out of the loops, align a local in finalize, and drop the obsolete fragmentation-heuristic comment in _init. No change to the seq/stride/full layouts under pure PUT/GET. TEST/GTEST: seed simulated proc info from local probe - set_simulated_proc_info now starts from ucc_local_proc before overriding the topology coordinates. ucc_context_create_proc_info copies the whole struct into the context id (and allgathers it to every rank), and topology construction reads cpu_vendor/cpu_model, so leaving the remaining fields indeterminate was undefined. Only the topology coordinates are now overridden. TEST/GTEST: add onesided alltoall mixed-alg ordering test - Add test_alltoall_onesided_mixed.ordering: ALG=mixed crossed with ORDER in {stride, full} and NFRAGS in {1, 4}. The auto tune in the existing ordering test only reaches the mixed GET/PUT progress path via `full` (seq/stride route to GET), so the explicit mixed path under the non-full orders was untested. That is the path that depends on build_peer_order's local/remote split requirement, so it is the one that needs a permanent case. Seq is omitted: it is the default order and is covered by the auto path.
|
🤖 CI Triage Agent — TL;DR: The codestyle lint failed on the commit-title check, not on code: four commits in PR #1345 have titles longer than 50 characters and/or use the unsupported Full analysisSummary: The Root cause:
The two commits using Implicated commit: [REDACTED:Hex High Entropy String] ( File: .github/workflows/codestyle.yaml:39-53 (checker); offending commit titles on branch Suggested fix: Reword the four commits with
Note specifically that Related: PR #1345 (#1345); sibling PR #1344 touches the same onesided-alltoall ordering area.
|
278b27b to
c28ac9b
Compare
What
Adds tuning knobs to onesided alltoall (peer ordering and message fragmentation) and schedules SHM and remote communication in the algorithm.
Why ?
In the current algorithm, performance tends to drop significantly when the number of tokens calculated at init time reaches 1 or lower. This changes the order and rate of message delivery to move performance closer to SOL.
How ?
Measured performance for messages >= 128 kB shows up to 15% improved bandwidth on IB (Gaia) and 8% improved bandwidth on RoCE.