Repository navigation
[CuTe DSL] SM100 pipelines: constant leader rank and pair mask when the cluster is one 2CTA pair - #3909
[CuTe DSL] SM100 pipelines: constant leader rank and pair mask when the cluster is one 2CTA pair#3909Johnsonms wants to merge 1 commit into
Conversation
…he cluster is one 2CTA pair PipelineTmaUmma, PipelineAsyncUmma, PipelineUmmaAsync and PipelineTmaMultiConsumersAsync derive the leading CTA rank and the multicast / peer masks from the CTA rank at runtime. When the cluster is exactly one 2CTA pair (cluster shape (2, 1, 1, 1)) both CTAs get the same values, rank 0 and mask 0b11, so return those constants instead. Every arrival on these pipelines then carries no rank arithmetic (seven mapa computations fold to a literal in the FlashAttention-4 SM100 backward, 8k causal -1.7% kernel cycles, non-causal -1.1%). Other cluster shapes keep the generic path.
|
Validation after opening the PR (B200, DSL 4.8.0 package with this change applied, FlashAttention-4 SM100 backward on Dao-AILab/flash-attention main 68bd095c, code unchanged; details in #3907). Kernel cycles ( Paper-style TFLOPS (-lgc 1965, hdim 128, 16 heads, 32k tokens per shape), causal 1k / 2k / 4k / 8k / 16k / 32k: 550 / 796 / 1009 / 1166 / 1279 / 1247 -> 555 / 807 / 1014 / 1175 / 1276 / 1259. Tests: flash-attention |
PipelineTmaUmma,PipelineAsyncUmma,PipelineUmmaAsync(andPipelineTmaMultiConsumersAsyncthrough the shared helper) derive the leading CTA rank and the multicast / peer masks fromblock_idx_in_clusterat kernel start and keep them in registers for every arrival. When the cluster is exactly one 2CTA pair, cluster shape(2, 1, 1, 1), both CTAs get the same values: leader rank 0 and pair mask0b11. This PR returns those constants in that case (_is_single_pair_cluster), so the arrivals carry no rank arithmetic; other cluster shapes keep the generic path.Measured on the FlashAttention-4 SM100 backward (Dao-AILab/flash-attention
main, B200, DSL 4.8.0 package with this change applied,ncu --clock-control none): sevenmapacomputations fold to a literal and the stack allocation drops 16 -> 0 bytes; kernel cycles 8k causal 3.548 M -> 3.486 M (-1.7%), 32k causal 12.53 M -> 12.32 M (-1.7%), 8k non-causal 6.216 M -> 6.151 M (-1.0%), hdim 256 causal 10.09 M -> 9.61 M (-4.7%), non-causal 18.26 M -> 17.64 M (-3.4%). Outputs and dK / dV bitwise identical, dQ within its fp32 reduce-order noise; flash-attention'stests/cute/test_flash_attn.pyhdim 128 / 256 subsets pass.examples/.../fp16_gemm_4.py(preferred cluster (2, 4, 1): generic path, fallback (2, 1, 1): constant path) passes verification.Context: this is one of the two source-side items behind the DSL >= 4.6 causal-backward regression in #3907 / Dao-AILab/flash-attention#3187.
Scope: this is the DSL-side counterpart of the constant-cluster-destination half of Dao-AILab/flash-attention#3200 (its other half, masking only the diagonal tiles, is flash-attention-only). It removes work at the source; it does not address the residual lowering change tracked in #3907 (per-use re-materialisation of uniform bases in the compute warps, extra BAR / FENCE in the dQ-drain loop), which stays open there.