Conversation
…decode HealDA's gpsro_v3 training loader gives every bending-angle level a finite pressure coordinate built from the occultation's own refractivity levels (blended standard-atmosphere/dry pressure below 5 km, dry above) and uses the refraction-corrected tangent height as the Height feature. The e2studio decoder emitted a null pressure and the geometric impact height, so NNJAObsConv/NomadsGDASObsConv GPS-RO rows could not feed the model the way training did. Add earth2studio.data.utils_gpsro with the pure-numpy derivation and the same fallback order as the loader, read HEIT/ARFR from the BUFR message, and expose a gpsro_reject_qfro_bits option on NNJAObsConv for the QFRO-based rejection training applied.
Data sources should expose fields as the archive carries them so the model wrapper, not the source, decides what a field means. The NCEP gpsro decode now leaves `gps` rows raw (null `pres`, geometric impact height in `elev`) and exposes the message's refractivity levels as a new `gps_refractivity` variable (`ARFR`, N-units, `elev` = `HEIT`). The refraction-corrected height / blended pressure derivation and the QFRO bit-5 reject move into the HealDA wrapper (`earth2studio.models.da.utils_gpsro`), which derives them per occultation from those raw rows.
6 tasks
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Earth2Studio Pull Request
Description
Roadmap task: NVIDIA/physicsnemo-roadmap#2985 (feature NVIDIA/physicsnemo-roadmap#2669, Releasing HealDA V2 in E2S).
HealDA needs a finite pressure coordinate and a refraction-corrected height for every GPS-RO bending-angle level, built from the occultation's own refractivity levels. The NCEP gpsro decode only exposed the bending angle (
gps) with a nullpresand the geometric impact height inelev, and the refractivity block was discarded, so the wrapper had nothing to derive from.Following review, the data source stays raw and model-agnostic; the derivation lives with the model:
Data source (
NNJAObsConv,NomadsGDASObsConv)gps_refractivity(gpsro::15036) in both lexicons: the message's refractivity levels (ARFR, N-units, first slot perHEIT, mirroring theBNDAvalue/error slot handling) withelev= the level's geometric height (HEIT) and nullpres. Rows carry the occultation'stime,type(SAID),stationandquality(QFRO) so consumers can regroup them per occultation.gpsrows are unchanged frommain: nullpres,elev= impact parameter minus Earth radius of curvature. No QC and no derived vertical coordinate in the source.HealDA wrapper
earth2studio.models.da.utils_gpsro:refraction_corrected_height(Bouguer fixed point onn(h)(R_c + h) = a), hydrostaticdry_pressure_hpafrom refractivity, USSA-1976height_to_pressure_hpa,blended_pressure_hpa(0.8 standard + 0.2 drybelow 5 km, dry above),gpsro_level_coordinateswith the fallback order blended -> dry -> standard at corrected height -> standard at geometric height,qfro_bit_set, andassign_gpsro_coordinates(frame)which groupsgps/gps_refractivityrows by occultation (time,type,station) and fillspres/elevon thegpsrows, dropping the refractivity rows. A nominal 6371 km radius of curvature is used (metre-level height error versus the message'sELRC).HealDA.input_coordsacceptsgps_refractivity;HealDA.prep_convrejects gps-family rows withQFRObit 5 set when aqualitycolumn is present, then callsassign_gpsro_coordinates.Usage: request
["u", "v", "q", "t", "pres", "gps", "gps_refractivity"]from the source and pass the frame toHealDAas before; withoutgps_refractivityrows the wrapper falls back to the standard atmosphere at the geometric height.Out of scope: SAID allowlist and height / bending-angle QC beyond the QFRO reject, NNJA SATWND source (#1151), NNJA-vocabulary HealDA-v2 wrapper (#975).
Tests:
test/models/da/test_utils_gpsro.py(analytic exponential column, fixed-point convergence, blend weights, fallback chain, QFRO bit numbering,assign_gpsro_coordinateswith and without a profile and across occultations),test/models/da/test_da_healda.py(prep_convderivation matches a direct call, bit-5 reject, other bits kept),test/data/test_utils_ncep.py(raw refractivity rows exposed with header lat/lon and metadata; none emitted when onlygpsis requested).Checklist
Dependencies
None. Touches
HealDA.prep_convalongside #1151; the two merge cleanly (verified locally), only the CHANGELOG bullets are adjacent.