Skip to content

fix: use Loader's saddle-point form for the gamma/beta prefixes and pmfs - #418

Draft
agene0001 wants to merge 3 commits into
statrs-dev:mainfrom
agene0001:feat/loader-saddle-point
Draft

fix: use Loader's saddle-point form for the gamma/beta prefixes and pmfs#418
agene0001 wants to merge 3 commits into
statrs-dev:mainfrom
agene0001:feat/loader-saddle-point

Conversation

@agene0001

Copy link
Copy Markdown
Contributor

Stacked on #417 (beta_reg robustness); review only the last commit until that merges.

Densities and incomplete-function prefixes in the gamma/beta family are written as exp(a·ln x − x − ln_gamma(a)) and friends. Every term grows like a·ln a while the sum stays O(ln a), so at a = 1e4 each term is ~1e5 and the ~1e-11 absolute error of the sum becomes the accuracy ceiling of everything downstream — this is what limited beta_reg to ~6e-11 even where its recurrence converged, and put Binomial::pmf at ~7e-11 relative.

Adds the two kernels from Loader, Fast and Accurate Computation of Binomial Probabilities (2000):

  • stirling_delta(z) — the Stirling-series remainder of ln_gamma, O(1/12z). Table-free: the recurrence δ(z) = δ(z+1) + (z+½)·ln1p(1/z) − 1 lifts any argument to where a 6-term series is good to 1.4e-18, so there is no interpolation table and no recursive dependency on ln_gamma. ≤1.2e-15 absolute, which is what matters since every caller adds it to an exponent.
  • bd0(x, np) = x·ln(x/np) − (x − np), with a double root at x = np. Evaluated by a series in (x−np)/(x+np) so the cancellation happens analytically: ~1e-16 relative uniformly through the root.

Rewriting the prefixes (gamma_lr/gamma_ur, beta_reg, Binomial::pmf, Poisson::pmf) in these terms keeps every piece O(1):

before after
beta_reg, a = b = 1e4 6e-11 8.7e-15
beta_reg, a = b = 1e6 3e-9 2.6e-13
gamma_ur, a ≥ 16 5.3 median ulp, p99 970k 0.38 median, p99 3071
Binomial::pmf 685k median ulp 2.45
Poisson::pmf 649 median ulp 3.16

Two details reviewers should look at:

  1. Both prefixes fall back to the direct form for small parameters (max param < 16): there's no cancellation to remove there, and without the gate FisherSnedecor(1,1).cdf got worse (the recurrence costs 16 steps at a = 0.5).
  2. bd0 is sensitive enough that the rounding of n·p itself matters (∂bd0/∂np = 1 − x/np: half an ulp of np at 6e5 moves the result 2.5e-13 ≈ 1000 ulp of the pmf), so n, 1−p, and the products are carried as double-doubles via bd0_dd. R's dbinom has this same limitation.

Also guards bd0's direct branch where (x/np).ln() under/overflows — bd0(1e-300, 5e99) gave −inf where the answer is +5e99, which surfaced as NaN (or a silent 1.0 where the truth is ~1e-30118) in beta_reg at extreme parameter ratios.

This is the same log-space reasoning as #381, applied to the rest of the family.

🤖 Generated with Claude Code

agene0001 and others added 3 commits August 26, 2026 13:20
`ln_gamma`/`gamma` evaluate Pugh's 10-term Lanczos approximation as the
partial-fraction sum `d_0 + sum_k d_k / (z + k - 1)`. The residues alternate in
sign, so that sum cancels badly - condition number ~3600 around z = 50, i.e.
~440 eps of relative error in the sum alone.

The same quantity as a single fraction `N(z) / D(z)`, derived from those residues
in exact rational arithmetic, has *all-positive* coefficients in both numerator
and denominator (`D(z) = z (z+1) ... (z+9)`, whose expanded coefficients are
unsigned Stirling numbers of the first kind and exact in f64). For z > 0 both
Horner evaluations then have condition number 1. This needs no new external
constants - they are derived from the residues already in the file - and agrees
with the partial-fraction form to 3.2e-17 over [0.5, 3000]. A reversed form in
1/z covers z >= 1e29, where z^10 would overflow.

Two further fixes in the same evaluation:

  * `lanczos_power` compensates the base of `((p + g) / e)^p`. `powf` amplifies a
    relative error in its base by the exponent, so the two roundings in
    `(p + g) / e` were the dominant error (~190 ulp by x = 122). The residuals of
    the addition and the division - the latter against a double-double `e` - are
    recovered exactly and applied as a first-order correction, which is only
    valid while it stays small, so it is gated on that.
  * `gamma`/`ln_gamma` at the positive integers up to 171 come from the exact
    factorial table, which also makes `Gamma(2) == 1` rather than
    1.0000000000000002.
  * `gamma` for x above ~169.7 halves the exponent and squares, because the power
    alone overflows there while the full product stays representable to
    x ~ 171.61. `gamma(171.6)` was `inf`; it is now 1.5858969096673e308.

`sin_pi`/`tan_pi` reduce by the period before calling `sin`/`tan`. `x - round(x)`
is exact, so the error stays relative to the fractional part instead of being
`ulp(PI) / dist_to_pole`, which near the poles cost several decimal digits.

Measured against mpmath at 45 dps:

                          before                after
    ln_gamma p99          45.9 ulp              8.2 ulp
    gamma (positive)      285 median            19 median
    gamma (negative)      109 median, 1173 max  3.4 median, 22.8 max
    digamma (negative)    19.5 median, 1.5e10   1.6 median, 538 max

`gamma`'s remaining ~19-35 ulp is the approximation floor of the f64-rounded Pugh
coefficient set itself (~3.7e-15, confirmed by evaluating the formula in exact
arithmetic), so the evaluation is now compensated to well below the fit.

Four test expectations move as a consequence, each verified against mpmath: one
`Binomial::sf` and two `FisherSnedecor` pdf/ln_pdf literals were fitted to the
old output, and the `Dirichlet`/`MultivariateStudent` doctest values were 34 and
22 ulp from truth (the new outputs are within 2).
…-space pdf

statrs-dev#448 already changed FisherSnedecor's pdf/ln_pdf to a log-space
computation on main, independent of this rebase. Its output for
(10, 1) differs from the direct-formula path this commit's literals
were tuned against by 3.6e-16 (pdf) and 1.55e-15 (ln_pdf), just past
the prior tolerances. Both are last-few-ulp differences between two
correct formulations, not a regression, so the tolerances are widened
to cover it.
Densities and incomplete-function prefixes in this family are written as
`exp(a ln x - x - ln_gamma(a))` and friends. Every term there grows like
`a ln a` while the sum stays `O(ln a)`, so at `a = 1e4` each term is ~1e5 and the
~1e-11 absolute error of the sum becomes the accuracy ceiling of everything
downstream. That is what limited `beta_reg` to ~6e-11 even where its recurrence
converged.

Adds the two building blocks from Loader, "Fast and Accurate Computation of
Binomial Probabilities" (2000):

  * `stirling_delta(z)`, the Stirling-series remainder of `ln_gamma`, which is
    `O(1/(12z))`. It is table-free: the recurrence
    `d(z) = d(z+1) + (z + 1/2) ln(1 + 1/z) - 1` lifts any argument into the range
    where a 6-term series is good to 1.4e-18, so there is no interpolation table
    and no recursive dependency on `ln_gamma`. Accurate to ~1e-15 *absolute*,
    which is what matters since every caller adds it to an exponent.
  * `bd0(x, np) = x ln(x/np) - (x - np)`, which has a double root at `x == np`.
    Evaluated by a series in `(x - np) / (x + np)` so the cancellation is done
    analytically, giving ~1e-16 relative accuracy uniformly through the root.

Rewriting the prefixes in terms of these keeps every piece `O(1)`. Measured
against mpmath, and against `I_{1/2}(a, a) == 1/2` which is exact:

    beta_reg, a = b = 1e4        6e-11  ->  8.7e-15
    beta_reg, a = b = 1e6         3e-9  ->  2.6e-13
    gamma_ur, a >= 16      5.3 median ulp -> 0.38, p99 970k -> 3071
    Binomial::pmf          685k median ulp -> 2.45
    Poisson::pmf            649 median ulp -> 3.16

Both prefixes fall back to the direct form for small parameters, where there is
no cancellation left to remove and `stirling_delta`'s recurrence would cost more
roundings than it saves - without that gate, `FisherSnedecor(1,1).cdf` got
*worse*, since a = b = 0.5 needs 16 recurrence steps.

`bd0` is sensitive enough to its mean argument that the rounding of `n * p` alone
matters: `d bd0 / d np = 1 - x / np`, so half an ulp of `np` at 6e5 moves the
result 2.5e-13, which was a thousand ulps of the resulting pmf. `n`, `1 - p` and
the products are therefore carried as double-doubles via `bd0_dd`. R's `dbinom`
has the same limitation.

Also guards `bd0`'s direct branch, where `(x / np).ln()` silently loses the whole
term when the ratio leaves the normal range - `bd0(1e-300, 5e99)` gave `-inf`
where the answer is `+5e99`, which propagated into `beta_reg` as NaN for extreme
parameter ratios, and as a silent `1.0` where the true value was ~1e-30118.

Left function::beta untouched here: the beta_reg prefix half of this
change is deferred to a separate PR replacing that implementation, so
only the gamma_lr/gamma_ur/Binomial/Poisson side lands. The two
prec::ulps_eq! snaps this originally reintroduced into checked_gamma_lr
around a and x near zero are dropped too - statrs-dev#450 already removed them
from main and the saddle-point path underflows through the existing
ax < -709.78 branch on its own, so they're redundant either way.
@YeungOnion
YeungOnion force-pushed the feat/loader-saddle-point branch from 8833a6f to 600a382 Compare August 26, 2026 18:28
@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants