Skip to content

perf(demux): strided plain CRF decode; fix CNN GPU ordinal and lock policy - #482

Merged
jayhesselberth merged 1 commit into
mainfrom
perf/475-gpu-crf-scratch
Oct 11, 2026
Merged

jayhesselberth merged 1 commit into
mainfrom
perf/475-gpu-crf-scratch

Conversation

@jayhesselberth

Copy link
Copy Markdown
Member

Closes #475

Shipped

  • Item 2 (strided plain decode, decode_with_strided): GPU encoder + CPU decode, --no-ref-scores, 20k dual-axis, A30 node, 4 interleaved reps (rep 1 warm-up): wall 15.2-15.7 s -> 14.0-14.1 s, CPU 102.5 s -> 84.7 s, trace encode+decode 12.8 s -> 11.7 s. evaluate_demux_crf.py: 0 of 20,000 differ on both axes. Run-to-run noise ~0.2 s.
  • Item 3: cuda_total_mem_bytes takes the configured ordinal. Test batch_budget_queries_the_configured_ordinal (fails pre-fix).
  • Item 4: policy = recover with into_inner (as encoder_gpu.rs). Test poisoned_session_lock_is_recovered (panics pre-fix).

Dropped

  • Item 1 (persistent lattice scratch): default fused GPU path, same harness, 4 reps: wall 7.7 s vs 7.6-7.9 s, trace encode+decode 5.2 s vs 5.1-5.2 s. No measurable change (20k reads is ~40 batches), and it would pin ~314 MB of device memory. Reverted.

Item 2 only applies when the GPU lattice decode is off (ESCAPEPOD_CRF_GPU_DECODE=0 or kernel load failure); the default GPU path is unchanged. Parity guard (#454) untouched.

@jayhesselberth
jayhesselberth merged commit f4576f1 into main Oct 11, 2026
9 checks passed
@jayhesselberth
jayhesselberth deleted the perf/475-gpu-crf-scratch branch October 11, 2026 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(gpu): persistent lattice scratch, strided plain decode; fix cuda_total_mem ordinal and mutex-poison policy

1 participant