Repository navigation
Conversation
`column_fractional_z_index` chose which column to bisect with `Base.unsafe_trunc`,
which floors. Its `ii`/`jj` are 1-based node indices, so flooring takes the column
west and south of the query whenever the fractional part exceeds a half — half of
all queries per axis, and 74.5% for at least one axis, measured on a real parent.
That is a half-cell offset, so the sampled height is displaced by roughly half the
column-to-column height difference. Measured against ERA5 0.25 degree over the
Southern Great Plains (45x38x37, terrain 32-1518 m, 906612 queries at 12 heights),
binned by the adjacent-column height difference:
|dZ| (m/cell) median |dh| p95 ratio
0-1 0.31 m 1.19 0.53
2-4 1.68 4.35 0.60
8-16 4.75 13.48 0.44
Aloft the effect is small but biased: floor gives a mean height error of -1.53 m
where nearest gives +0.02 m, and the rms against an exactly inverted reference
halves, 2.63 -> 1.41 m. Temperature and wind move by 0.02 K and 0.05 m/s at p95,
which is not forecast-relevant.
Below 250 m AGL it is not small. There the index is clamped to the column's first
above-ground level, and in ~9% of columns — concentrated over steeper terrain —
the two choices clamp one ERA5 level apart. The sampled height then jumps a whole
layer, 200-240 m, carrying temperature up to 2-3 K and meridional wind up to
7.9 m/s. Domain-wide near-surface temperature rms difference is 0.38 K at 10 m AGL.
A nested child's initial and boundary state over terrain is sensitive to this line,
so runs predating it will not reproduce bit-for-bit near the surface.
Flooring is also inconsistent with what follows: `interpolator` turns ii = 2.7 into
the stencil (2, 3, 0.7), i.e. mostly column 3, while the vertical came from column
2. `column_fractional_z_index` picks a representative column rather than a stencil
corner, so nearest is the right choice.
A second, rarer path reaches the same defect. Where the grid spacing is not a
binary fraction, a query placed exactly on a node round-trips a few ulp below the
integer and floors a whole column down — 0/45 nodes on ERA5's 0.25 degree grid,
but 44/110 on a 0.1 degree grid. That is what the regression test exercises.
Not addressed here, and larger than either version's error: near the surface both
choices sample ~90 m above the query, because the first above-ground ERA5 level
sits ~100 m above terrain on average. No column choice removes that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RpzYqdp9fuGrbRvfXNvJ52
Closes #508. `PressureLevelVerticalDiscretization` specializes `_fractional_indices` on the vertical; `LambertConformalConicGrid` specializes it on the horizontal, because its projected axes are not separable and so cannot route through `fractional_x_index`/`fractional_y_index` the way a `LatitudeLongitudeGrid` does. On a grid that is both, the Lambert method is the more specific and wins outright — no ambiguity, no error — and the vertical falls back to the generic `fractional_z_index`, which reads the column-*mean* height profile. Forcing the two paths apart with `invoke` on an 8x6x5 grid whose column `i` spans 100i to 100i + 1600 m: i z before after exact 1 300.0 0.625 1.5 1.5 3 1000.0 2.375 2.75 2.75 8 1000.0 2.375 1.5 1.5 Before, every column returns the same answer, because every column is reading the domain mean. The index is also unclamped in both directions — 0.625 is below the surface and a query above the column top gives k > Nz — which is what `column_fractional_z_index` clamps against, and what the comment there records as having NaN'd a terrain-following initial state on the GPU. This is not specific to Lambert conformal. Any grid that specializes `_fractional_indices` jointly on the horizontal defeats a vertical specialization the same way. A `horizontal_fractional_indices` hook upstream would let the two compose rather than compete; the joint method here is the local fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RpzYqdp9fuGrbRvfXNvJ52
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
column_fractional_z_index
`column_fractional_z_index` was rewritten on main by #580, which dropped `clip_subsurface!`, `first_above_surface_level` and the hand-rolled `index_binary_search` blend in favor of Oceananigans' `fractional_index`: sub-surface pressure levels now interpolate across the surface rather than clamping above the first above-ground level. Take main's body whole. This branch's only edit to that function is the two `nearest_column_index` calls, which choose the column, and are orthogonal to how the column is bisected. Verified against Oceananigans 0.114.0: test/test_pressure_level_grid.jl passes 1466/1466 on CPU. Reverting the two calls to `Base.unsafe_trunc` fails 6 of them; removing the joint `_fractional_indices` method fails 6. Both halves of the branch still bite. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rule 7 of the restraint rules: a comment describes the code that is there. The import TODO was a design memo about Oceananigans' export list; the method comment restated Oceananigans' `_fractional_indices` and argued for this one; the `Face`-location note described `column_fractional_z_index`. All three move to the pull request description. What is left names the dispatch position and the rounding idiom, which the code does not say on its own. Also fold the test file's second `Oceananigans.Fields` import into the first and separate the final testset from the one above it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kaiyuan-cheng
approved these changes
Oct 23, 2026
glwagner
reviewed
Oct 25, 2026
Comment on lines
+293
to
+294
| const LCCPressureLevelGrid = | ||
| LambertConformalConicGrid{<:Any, <:Any, <:Any, <:Any, <:PressureLevelVerticalDiscretization} |
Member
There was a problem hiding this comment.
Suggested change
| const LCCPressureLevelGrid = | |
| LambertConformalConicGrid{<:Any, <:Any, <:Any, <:Any, <:PressureLevelVerticalDiscretization} | |
| const LCCPressureLevelGrid = LambertConformalConicGrid{<:Any, <:Any, <:Any, <:Any, <:PressureLevelVerticalDiscretization} |
glwagner
approved these changes
Oct 25, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes under
column_fractional_z_index: which column it bisects, and the dispatch that reaches it. The second closes #508 and is what a conformal-conic parent needs in order to get its own column's vertical; the first is a pre-existing bug found while writing the second.1. Pick the column by nearest node, not by truncation
column_fractional_z_indexchose the column to bisect withBase.unsafe_trunc, which floors. Itsii/jjare 1-based node indices, so flooring takes the column west and/or south of the query whenever the fractional part is at least a half.The sampled height is then off by however much that level's height surface tilts across one parent cell. Measured on an ERA5 0.25° parent over the Southern Great Plains (45×39×37, terrain 32–1518 m, 2016-09-10 12Z), queried from a 1/24° child at 12 heights above ground — 592,860 points, each interpolating the parent's own height field through the full stencil, so (sampled − requested) is the error this function contributes:
74.5 % of queries change column. What the fix buys is removing the bias: flooring always picks west and south, so its error carries a sign, and over a nested run that is a systematic displacement of the parent's height field rather than noise that averages out.
Across all 592,860 queries the largest temperature change is 0.151 K and the largest wind change 0.402 m/s, and nothing below 250 m AGL moves more than 7 m in height.
The residual rms does not go to zero and cannot. Its reference is the height solved on the bilinearly blended column, so what remains is the cost of picking one column at all:
_interpolateblends four columns for the field value whilecolumn_fractional_z_indexbisects a single one, which makes the height surface zeroth-order where the field on it is first-order. Going first-order would mean bisecting a blended profile, and that sits on the hot path for every boundarygetbc, relaxation target and child initial condition — worth its own issue rather than this one.A second path reaches the same defect, and is what the lat-lon regression test exercises: where the grid spacing is not a binary fraction, a query placed exactly on a node round-trips a few ulp below the integer and floors a whole column down.
2. Compose the conformal-conic horizontal with the per-column vertical
Closes #508.
PressureLevelVerticalDiscretizationspecializes_fractional_indiceson the vertical;LambertConformalConicGridspecializes it on the horizontal, because its projected axes are not separable and cannot route throughfractional_x_index/fractional_y_index. On a grid that is both, the Lambert method is more specific and wins outright — no ambiguity, no error — and the vertical falls back to the genericfractional_z_index, which reads the column-mean height profile.Forcing the two apart on an 8×6×5 grid whose column
ispans 100i to 100i + 1600 m, so that the domain-mean profile (450–2050 m) matches no column.kis the fractional level index_fractional_indicesreturns, on 5 levels spaced 400 m apart, sok = 1.5sits halfway between a column's first and second level rather than a fraction of its depth:The two 1000 m rows are the tell: before, columns 3 and 8 return the same index although their heights differ by 500 m, because both read the domain mean rather than their own column. Row 1 shows the second hazard —
k = 0.625is below level 1, underground, and nothing clamps it; a query above a column's top runs pastk > Nzthe same way.column_fractional_z_indexclamps both ends.This is not specific to conformal conic grids: any grid that specializes
_fractional_indicesjointly on the horizontal defeats a vertical specialization the same way. Ahorizontal_fractional_indiceshook upstream would let the two compose rather than compete, and the joint method here is the local fix. It importslcc_fractional_indices, which Oceananigans does not export; reimplementing it from the exportedlcc_forwardwould duplicate the Center/Face node offsets.Verification
On Oceananigans 0.114.0 and Julia 1.12.7:
test/test_pressure_level_grid.jlpasses 1466/1466 on CPU under--check-bounds=yes, including two new testsets. The lat-lon one fails without fix 1; the Lambert one fails without fix 2.Base.isambiguousover all 25_fractional_indicesmethods finds 8, every one of them pre-existing and none involving the new method. TheUnion{Center, Face}restriction is load-bearing: dropping it adds eight more.@inferredconcrete,@allocated0 — and it runs inside a KernelAbstractions kernel on the CPU backend.LatitudeLongitudeGridand plainLambertConformalConicGridreach Oceananigans' own methods, andLatitudeLongitudeGrid+PressureLevelVerticalDiscretizationreaches the existing method in this file.Compatibility
No API change: no signatures or exports move, and the new
_fractional_indicesmethod is additive.Results do change for every
PressureLevelVerticalDiscretizationparent, by the amounts in §1 — at most 0.151 K and 0.402 m/s on the grid measured there. Nested runs will not reproduce bit-for-bit across this commit. Flagging rather than deciding: whether that warrants a minor bump or ships as a fix is a maintainer call.