Skip to content

Speed up StretchedExponential evaluation - #268

Merged
henrikjacobsenfys merged 4 commits into
developfrom
perf/stretched-exponential
Oct 20, 2026
Merged

henrikjacobsenfys merged 4 commits into
developfrom
perf/stretched-exponential

Conversation

@henrikjacobsenfys

Copy link
Copy Markdown
Member

A user reported that StretchedExponential is very slow. Profiling showed that ~95% of fit time went into _kww_shape, the quadrature that evaluates the transform at every energy point. With a resolution convolution, a single-Q fit took 1.37 s, compared with 0.03 s for a Lorentzian.

This PR makes it about 14× faster, in two commits.

1. Faster quadrature (results unchanged)

  • Cache-sized blocks: the quadrature was memory bound. It now runs in blocks of 2^15 elements (256 kB temporaries, small enough to stay in cache) instead of 2M, with in-place operations.
  • Dropped nodes: nodes where exp(-P) provably underflows to exactly 0 are removed, since P ≥ min(s, s^β) on the rescaled grid. That is about a quarter of the grid for β ≳ 0.5.
  • Results match the old code to rounding, and are still bitwise independent of block boundaries (reduction via einsum, not BLAS).
  • Speedup: 3–4×.

2. interpolate flag, default True

  • What it does: builds a cubic-spline table of log(G/G(0)) against y = asinh(w / w_ref) and interpolates from it. w_ref sits where the profile has dropped by 1e-6, which happens many decades below the half width at small β.
  • Caching: tables are cached per β. Steps in area, center and τ, and all Q values that share β, reuse them.
  • Smooth for fitting: nodes sit on a fixed lattice and the table grows in fixed steps, so the model stays smooth under the small parameter steps lmfit takes for its Jacobian. Finite differences match the quadrature to 1e-5.
  • Accuracy:
    • Within 1.2e-7 of the peak for all β in [0.05, 2].
    • For β ≤ 1, within ~1e-6 at every point, including the far tail, which holds most of the area at small β.
    • For β > 1, the profile is cut to 0 below 1e-14 of the peak, where the quadrature is already rounding noise.
  • interpolate=False keeps the ~1e-11 quadrature.

Timings

before after (quadrature) after (interpolate=True)
1505 points ~17 ms ~4.5 ms ~1.3 ms (0.03 ms cached)
Single-Q fit with resolution 1.37 s 0.41 s 0.10 s

Fitted values and errors are identical to 5 digits with and without interpolation.

Notes for review

  • interpolate is placed right after beta in the constructor, so code passing x_unit or later arguments by position would break.
  • Tests that check the quadrature against the exact Lorentzian/Gaussian to 1e-10 now pin interpolate=False. The interpolated path has its own tests: agreement relative to the peak across 40 β values, pointwise agreement for β ≤ 1, the β>1 cutoff, independence from table length, and smoothness under finite differences.

🤖 Generated with Claude Code

henrikjacobsenfys and others added 2 commits October 18, 2026 13:11
Evaluate the exp-sinh quadrature in cache-sized blocks with in-place
temporaries, and drop nodes whose decaying exponent provably underflows
exp to exactly zero. Results are unchanged to rounding and stay bitwise
independent of block boundaries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add an interpolate flag (default True) that evaluates G_beta from a
cached cubic-spline table of log(G / G(0)) on y = asinh(w / w_ref)
instead of running the quadrature at every energy. Tables are cached
per beta and grow on a fixed lattice, so fits stay smooth under
finite-difference Jacobian steps. Accuracy is ~1e-7 of the peak for all
beta; interpolate=False keeps the ~1e-11 quadrature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@henrikjacobsenfys henrikjacobsenfys added [scope] enhancement Adds/improves features (major.MINOR.patch) [priority] medium Normal/default priority labels Oct 18, 2026
@codecov

codecov Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.17%. Comparing base (762b081) to head (c605276).
⚠️ Report is 6 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #268      +/-   ##
===========================================
+ Coverage    98.15%   98.17%   +0.01%     
===========================================
  Files           60       60              
  Lines         6066     6123      +57     
  Branches      1022     1027       +5     
===========================================
+ Hits          5954     6011      +57     
  Misses          58       58              
  Partials        54       54              
Flag Coverage Δ
unittests 98.17% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...s/sample_model/components/stretched_exponential.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrikjacobsenfys
henrikjacobsenfys merged commit 22c623c into develop Oct 20, 2026
40 of 41 checks passed
henrikjacobsenfys added a commit that referenced this pull request Oct 20, 2026
* update pixi (#266)

* Adding Chudley-Elliott Jump Diffusion Model (#246)

* Add ChudleyElliotJumpDiffusion model and corresponding tests

* Enhance ChudleyElliotJumpDiffusion model documentation and tests; all checks passed

* Fix: replace Chudley-Elliot with Chudley-Elliott (propegated from spelling mistake in Mantid documentation)

* Fix: Add missing ChudleyElliottJumpDiffusion to public API and module exports

* Fix: Improve test clarity and update unit handling in ChudleyElliottJumpDiffusion tests

* Fix: Implement minimum width for residence time in ChudleyElliottJumpDiffusion model and update tests

* Fix: Update error handling in ChudleyElliottJumpDiffusion model and improve test assertions

* chore: pixi update and linting

* Add jump diffusion tutorials and Chudley-Elliott citation

- Add Jump Translational and Chudley-Elliott sections to
  diffusion_model.ipynb, with subheaders for each model
- Name the models in the Diffusion model entry of the tutorials index
- Add the Chudley and Elliott (1961) reference and "See also the
  tutorials" to the ChudleyElliottJumpDiffusion docstring
- Remove the tutorials TODO (#273)

* Speed up StretchedExponential evaluation (#268)

* Speed up StretchedExponential quadrature 3-4x

Evaluate the exp-sinh quadrature in cache-sized blocks with in-place
temporaries, and drop nodes whose decaying exponent provably underflows
exp to exactly zero. Results are unchanged to rounding and stay bitwise
independent of block boundaries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Interpolate the StretchedExponential profile by default

Add an interpolate flag (default True) that evaluates G_beta from a
cached cubic-spline table of log(G / G(0)) on y = asinh(w / w_ref)
instead of running the quadrature at every energy. Tables are cached
per beta and grow on a fixed lattice, so fits stay smooth under
finite-difference Jacobian steps. Accuracy is ~1e-7 of the peak for all
beta; interpolate=False keeps the ~1e-11 quadrature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add example, run pixi update

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Allow plotting parameters against Q^2 (#271)

* Allow plotting parameters against Q^2

Analysis.plot_parameters and ParameterAnalysis.plot take an x_axis
argument ('Q' or 'Q^2'). The values at each Q are unchanged; only the
plotted coordinate is transformed, via the shared add_x_axis_coord
helper and its Q_AXIS_TRANSFORMS table, so further axes are one entry.

Closes #205

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show the Lorentzian width against Q^2 in tutorial 1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Align the data and residuals axes in plots with residuals (#270)

slicerplot_with_residuals drew the residuals in a second Matplotlib
figure below the main one. Each figure was tight_layout'ed on its own,
so the axes' left edges depended on each panel's y tick label and y
label widths and did not line up.

Draw both panels as two axes of one figure with a shared x axis, so
they are laid out together and share x limits, scale and zoom. figsize
now sizes the whole figure, and the residuals panel hides its log
buttons, since logX is shared and logY does not suit residuals.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: easyscience[bot] <2476259+easyscience[bot]@users.noreply.github.com>
Co-authored-by: Henrik Jacobsen <henrik.jacobsen.fys@gmail.com>
Co-authored-by: mark <191927387+markbujehein@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[priority] medium Normal/default priority [scope] enhancement Adds/improves features (major.MINOR.patch)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant