salsa20: add SSE2/AVX2 intrinsics backends by paddor · Pull Request #576 · RustCrypto/stream-ciphers

paddor · 2026-07-17T00:15:47Z

SSE2 and AVX2 backends using x86 intrinsics, runtime detection via cpufeatures
Diagonal SIMD layout: each register holds one diagonal of the 4x4 state matrix, shuffles rotate between column and row round alignment
SSE2: 4 parallel blocks, 4 __m128i registers per block
AVX2: 4 parallel blocks via 2 __m256i sets, each packing 2 blocks in its 128-bit lanes
Dispatch: AVX2 > SSE2 > scalar fallback
no_std compatible, no new features required
dudect constant-time timing tests (examples/ctbench.rs)

                         Scalar       AVX2 intrinsics    Speedup
  Salsa20
    16 B                363 MB/s        421 MB/s          1.2x
    256 B               531 MB/s      1,551 MB/s          2.9x
    1 KiB               538 MB/s      1,646 MB/s          3.1x
    16 KiB              543 MB/s      1,673 MB/s          3.1x
    32 KiB              543 MB/s      1,673 MB/s          3.1x
    64 KiB              542 MB/s      1,673 MB/s          3.1x

  Salsa20/12
    16 B                484 MB/s        592 MB/s          1.2x
    256 B               847 MB/s      2,285 MB/s          2.7x
    1 KiB               870 MB/s      2,491 MB/s          2.9x
    16 KiB              874 MB/s      2,560 MB/s          2.9x
    32 KiB              873 MB/s      2,563 MB/s          2.9x
    64 KiB              874 MB/s      2,564 MB/s          2.9x

  Salsa20/8
    16 B                592 MB/s        727 MB/s          1.2x
    256 B             1,201 MB/s      2,976 MB/s          2.5x
    1 KiB             1,245 MB/s      3,346 MB/s          2.7x
    16 KiB            1,259 MB/s      3,464 MB/s          2.8x
    32 KiB            1,256 MB/s      3,479 MB/s          2.8x
    64 KiB            1,257 MB/s      3,478 MB/s          2.8x

paddor · 2026-07-17T00:33:44Z

@tarcieri This should be ready for review. Sorry about the new simd feature. Otherwise I'd have had to bump MSRV to 1.89. You decide.

Closes #50.

newpavlov · 2026-07-17T01:10:23Z

We generally aim to minimize third-party dependencies. I would strongly prefer to have a proper self-contained intrinsics-based implementation instead.

tarcieri · 2026-07-17T01:27:48Z

Yeah, we generally eschew crate dependencies for this sort of thing beyond our own cpufeatures crate for no_std-friendly CPU feature detection, even if it results in duplication of the SIMD backends, though ideally we could use a stable core::simd in the future.

ARX algorithms like Salsa/ChaCha in particular use a very limited number of operations (literally the three add-rotate-xor of ARX).

Before looping in a crate dependency I'd be curious to see what is possible with auto-vectorization alone, just by using #[target_feature] annotated "thunk" functions similar to what we do in argon2.

The multiversion crate provides a similar strategy though before going down that road I'd be curious to see what auto-vectorization alone can accomplish.

paddor · 2026-07-17T02:37:43Z

Naive auto-vectorization performs worse because of register spills. I'll follow the the chacha20 example and do the diagonal layout trick.

Add SIMD-accelerated Salsa20 backends using x86 intrinsics with runtime CPU detection via cpufeatures (no_std compatible, same pattern as chacha20). Both backends use diagonal SIMD layout: each register holds one diagonal of the 4x4 state matrix, with shuffles rotating between column and row round alignment. SSE2: 4 parallel blocks, 4 __m128i registers per block. AVX2: 4 parallel blocks via 2 __m256i sets, each packing 2 blocks in its 128-bit lanes. Dispatch order: AVX2 > SSE2 > scalar fallback. Measured 3.1x throughput on Salsa20/20 (543 -> 1673 MB/s) on x86_64 with AVX2.

paddor · 2026-07-17T10:50:30Z

PR updated. Auto-vectorization (with cpufeatures). dudect-bencher is still a new dep though. Should I remove it?

newpavlov

Could you generate Godbolt links to showcase generated assembly for SSE2 and AVX2 backends?

newpavlov · 2026-07-17T12:54:19Z

+            if #[cfg(any(target_arch = "x86", target_arch = "x86_64"))] {
+                let tokens = (avx2_cpuid::init(), sse2_cpuid::init());
+            } else {
+                let tokens = ();


~~I think you can use let tokens = Default::default() here.~~

UPD: InitToken does not currently implement Default. We probably should amend it in a future release.

Not possible because cpufeatures::InitToken doesn't impl Default. Instead, init_tokens() helper was extracted.

newpavlov · 2026-07-17T12:54:43Z

    state: [u32; STATE_WORDS],
+    /// CPU target feature tokens
+    #[allow(dead_code)]
+    tokens: Tokens,


Add reason to the allow attribute.

Remove `examples/ctbench.rs` and the `dudect-bencher` dev-dependency as requested during review. Criterion-based benchmarks will be added in the repo-level `benches/` crate instead.

- Add `reason` to `#[allow(dead_code)]` on the `tokens` field. - Extract `init_tokens()` helper to deduplicate the `cfg_if!` token initialization in `from_raw_state` and `KeyIvInit::new`. (`Default::default()` as suggested doesn't compile because `cpufeatures::InitToken` doesn't impl `Default`.) - Add PR reference `([RustCrypto#576])` to CHANGELOG entry.

Add `salsa20` throughput benchmark to the repo-level `benches/` crate, matching the existing `chacha20` benchmark structure.

tarcieri · 2026-07-17T14:53:47Z

FWIW we've used dudect in the past in crypto-bigint and ultimately removed it because it was unhelpful.

Scrutinizing the generated assembly will suffice for now. I've looked at other tooling for constant-time verification (e.g. cargo-checkct) but if we roll something like that out it will be across the entire repo.

paddor · 2026-07-17T14:57:05Z

Godbolt SSE2: https://rust.godbolt.org/z/Kao683b6v
Godbolt AVX2: https://rust.godbolt.org/z/eWszo8Ee8

tarcieri · 2026-07-17T15:41:54Z

On quick glance (just the Godbolt links) the AVX2 version looks good. In the SSE2 version I'm seeing a massive block of movaps calls which might be stack spilling:

        movaps  xmmword ptr [rsp - 48], xmm15
        movaps  xmmword ptr [rsp + 112], xmm12
        movaps  xmmword ptr [rsp - 80], xmm12
        movaps  xmmword ptr [rsp + 64], xmm8
        movaps  xmmword ptr [rsp - 64], xmm8
        movaps  xmm9, xmm7
        movaps  xmmword ptr [rsp], xmm2
        movaps  xmmword ptr [rsp - 128], xmm2
        movaps  xmmword ptr [rsp + 48], xmm10
        movaps  xmmword ptr [rsp - 96], xmm10
        movaps  xmm13, xmm7
        movaps  xmm12, xmm15
        movaps  xmmword ptr [rsp - 16], xmm14
        movaps  xmmword ptr [rsp + 32], xmm11
        movaps  xmmword ptr [rsp - 112], xmm11
        movaps  xmm2, xmm7
        movaps  xmmword ptr [rsp + 96], xmm15
        movaps  xmmword ptr [rsp - 32], xmm6
        movaps  xmm8, xmm6
        movaps  xmm6, xmm15
        movaps  xmmword ptr [rsp + 16], xmm3
        movaps  xmm5, xmm3
        movaps  xmmword ptr [rsp + 80], xmm7
        movaps  xmm4, xmm7

(Sidebar: curious if chacha20 has a similar problem)

Maybe we should just start with the AVX2 version since that seems more straightforward.

Edit: actually the generated SSE assembly doesn't look that bad with PAR_BLOCKS changed from 4 to 2:

const PAR_BLOCKS: usize = 2;

Trying to do 4 blocks in parallel with SSE appears to be exceeding the available registers.

paddor · 2026-07-17T16:05:26Z

So apparently PAR_BLOCKS=4 is still faster, despite the spills?

● PAR_BLOCKS=2 is slower, not faster. Here's the comparison:

  ┌────────┬────────────────────┬────────────────────┬─────────────┐
  │  Size  │ PAR_BLOCKS=4 (cpb) │ PAR_BLOCKS=2 (cpb) │    Delta    │
  ├────────┼────────────────────┼────────────────────┼─────────────┤
  │ 1 KiB  │ 3.46               │ 3.89               │ +12% slower │
  ├────────┼────────────────────┼────────────────────┼─────────────┤
  │ 2 KiB  │ 3.48               │ 3.88               │ +11% slower │
  ├────────┼────────────────────┼────────────────────┼─────────────┤
  │ 4 KiB  │ 3.43               │ 3.85               │ +12% slower │
  ├────────┼────────────────────┼────────────────────┼─────────────┤
  │ 8 KiB  │ 3.42               │ 3.85               │ +13% slower │
  ├────────┼────────────────────┼────────────────────┼─────────────┤
  │ 16 KiB │ 3.41               │ 3.86               │ +13% slower │
  └────────┴────────────────────┴────────────────────┴─────────────┘

It does seem smelly though. What should we do? It's a small perf gain, so I'd probably prefer fewer spills. That would also allow the compiler to use more registers for other stuff/avoid more reloads? I'm not too familiar with this stuff. I only learned about register spill like a month ago.

tarcieri · 2026-07-17T16:33:24Z

Interesting. I dug up when we tuned chacha20 to use 4-block parallelism, which was also a benchmark-driven result that showed similar improvements: #379

So it seems to be faster in spite of the (egregious looking) stack spilling.

paddor · 2026-07-17T16:44:38Z

What does the compiler do with unused registers before/after the call to salsa20? If salsa20 doesn't need them, then some of the application's registers can survive the call, right? So PAR_BLOCKS=2 => cleaner, slightly slower, but possibly faster for surrounding code (hot loops), but PAR_BLOCKS=4 => messy, faster salsa20, but maybe worse for hot loops?

tarcieri · 2026-07-17T16:57:37Z

That's really going to depend on the code in question. I guess we could look specifically at crates like scrypt and yescrypt

paddor · 2026-07-17T17:18:15Z

So I benched the SSE2 backend for scrypt and yescript.

  ┌─────────────────────────┬──────────────┬──────────────┬───────────────┐
  │        Benchmark        │ PAR_BLOCKS=4 │ PAR_BLOCKS=2 │     Delta     │
  ├─────────────────────────┼──────────────┼──────────────┼───────────────┤
  │ scrypt(n=16384,r=8,p=1) │ 26.77 ms     │ 26.49 ms     │ -1.0% (noise) │
  ├─────────────────────────┼──────────────┼──────────────┼───────────────┤
  │ yescrypt(default)       │ 48.82 ms     │ 48.33 ms     │ -1.0% (noise) │
  └─────────────────────────┴──────────────┴──────────────┴───────────────┘

Is it correct that the SSE2 backend would only be used on CPUs made in 2013 and older?

tarcieri · 2026-07-17T17:21:50Z

For the most part, yes. On #379 I questioned whether it's worth supporting, but we ultimately continued to support it.

paddor · 2026-07-17T17:30:40Z

Support sure, but what about the perf and register spill? PAR_BLOCKS=2 or 4? My digital helper just pointed out: On any machine old enough to lack AVX2, 12% better perf is probably worth having. Even if the assembly is ugly.

tarcieri · 2026-07-17T17:38:55Z

Yes, that was the same argument in #379, which we ultimately went with.

I can't speak to how it would operate on older hardware nor do I have access to such hardware to test with. It would probably be good for someone to test on actual hardware with only SSE2 that lacks AVX2.

paddor · 2026-07-17T17:48:36Z

If you know someone with that kinda hardware? If not, I think we might be engaging in hypothetical/premature optimization... maybe the KISS route is: PAR_BLOCKS=2, clean assembly, and leave it be until there's someone complaining about salsa20 perf on ancient hardware.

tarcieri · 2026-07-17T17:55:04Z

I mean, that's how I feel about the SSE2 backend as a whole, as I expressed in #379. It's hard to gauge the relative merits of a SIMD backend for hardware so old nobody has access to it to test on.

paddor · 2026-07-17T18:09:40Z

So, kick it completely? I would not care. Slightly anachronistic because salsa20 is a bit older than chacha20.

So keeping SSE2 at all is a hypothetical optimization. Nobody out there is doing perf-sensitive work on 13+ year old hardware. The crate would still work, just without the speedup.

tarcieri · 2026-07-18T01:22:50Z

Yeah, I think we can start with AVX2 and then follow up on SSE2 and decide if the maintenance burden is worth it and if there are actually any users who can tell us how it should be tuned

Per review feedback, start with AVX2 and follow up on SSE2 later once there is user demand to guide tuning.

paddor force-pushed the salsa20-simd branch from 3dc0d61 to 16cc781 Compare July 17, 2026 00:28

paddor force-pushed the salsa20-simd branch from 16cc781 to 619a39e Compare July 17, 2026 10:44

paddor changed the title ~~salsa20: SIMD-accelerated backends via fearless_simd~~ salsa20: add SSE2/AVX2 intrinsics backends Jul 17, 2026

newpavlov reviewed Jul 17, 2026

View reviewed changes

paddor added 3 commits July 17, 2026 16:03

salsa20: remove dudect constant-time bench example

c14c65e

Remove `examples/ctbench.rs` and the `dudect-bencher` dev-dependency as requested during review. Criterion-based benchmarks will be added in the repo-level `benches/` crate instead.

benches: add salsa20 criterion benchmark

55b939a

Add `salsa20` throughput benchmark to the repo-level `benches/` crate, matching the existing `chacha20` benchmark structure.

salsa20: drop SSE2 backend, keep AVX2 only

f91ec0a

Per review feedback, start with AVX2 and follow up on SSE2 later once there is user demand to guide tuning.

Uh oh!

Conversation

paddor commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddor commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

newpavlov commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

newpavlov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

newpavlov Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paddor Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

newpavlov Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

paddor Jul 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tarcieri commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tarcieri commented Jul 17, 2026

Uh oh!

paddor commented Jul 17, 2026

Uh oh!

tarcieri commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

paddor commented Jul 17, 2026 •

edited

Loading

paddor commented Jul 17, 2026 •

edited

Loading

tarcieri commented Jul 17, 2026 •

edited

Loading

newpavlov Jul 17, 2026 •

edited

Loading

tarcieri commented Jul 17, 2026 •

edited

Loading

tarcieri commented Jul 17, 2026 •

edited

Loading

paddor commented Jul 17, 2026 •

edited

Loading