Skip to content

Quiet MAPL logging in the GOCART2G regression cases; add regression README - #501

Merged
pchakraborty merged 2 commits into
release/MAPL-v3from
feature/pchakrab/regression-logging
Nov 10, 2026
Merged

pchakraborty merged 2 commits into
release/MAPL-v3from
feature/pchakrab/regression-logging

Conversation

@pchakraborty

@pchakraborty pchakraborty commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Adds a logging.yaml to each run1/run2 directory of the GOCART2G regression cases (dust-sa, seasalt-sa, sulfate-sa, nitrate-sa, gocart+seasalt+dust+sulfate, gocart+seasalt+dust+sulfate+nitrate) and loads it from mapl.yaml with a top-level pflogger: logging.yaml. It sends logs to the console only, sets the MAPL logger to ERROR, and keeps MAPL.profiler at INFO on the root rank, so the timing report is still printed.

Why: the cases run without a field dictionary, so MAPL warns once per variable spec per rank (History SetServices, apply_field_dictionary_defaults_ in VariableSpec.F90). MAPL's default logging writes each warning to warnings_and_errors.log under an MPI lock, which serializes the ranks. In the GEOS Moist cases this was about 6 s of a 17 s test.

Also:

  • ESMF/GOCART2G_GridComp/regression/README.md (new): how the regression cases run, their common setup (including the logging above), each case (nitrate-sa with its DuSsStub, and gocart+seasalt+dust+sulfate+nitrate), how to build a case from MAPL2 captures, and cmpchk.py.
  • SU2G_GridComp/mapl3-porting-notes.md: maps each MAPL2 friendly target to its MAPL3 service (DYNAMICS -> TRADV done, MOIST -> MTR to do, TURBULENCE -> TRDIFF blocked), lists the tracers each child would subscribe, and notes how Moist (Port GEOS_MoistGridComp to MAPL3 (GFDL_1M, BACM_1M, GF, UW) GEOSgcm_GridComp#1566) provides MTR.

Adds the MAPL3 error-macro rule to AGENTS.md (_RC, _STAT, _ASSERT with a message, _RETURN(_SUCCESS); _RC cannot end a one-line if; no !---- separators).

The same logging.yaml is being added to the regression cases in GEOSgcm_GridComp (GEOS-ESM/GEOSgcm_GridComp#1574), GEOSradiation_GridComp (GEOS-ESM/GEOSradiation_GridComp#116) and FVdycoreCubed_GridComp (GEOS-ESM/FVdycoreCubed_GridComp#445). The PRs are independent of each other.

Testing

ctest -L REGRESSION (ifx Debug): 32/32 pass with these files in every GEOS case directory. The GEOS cases' summed test time went from 189 s to 132 s.

Each run1/run2 case directory gets a logging.yaml (console output only, MAPL logger at ERROR) that mapl.yaml loads with `pflogger: logging.yaml`. Without a field dictionary MAPL warns once per variable spec per rank, and its default logging writes each warning to warnings_and_errors.log under an MPI lock, which serializes the ranks. AGENTS.md: add the MAPL3 error-macro rule.
regression/README.md describes how the cases run, their common setup (including logging.yaml), each case (with nitrate-sa, its DuSsStub, and gocart+seasalt+dust+sulfate+nitrate), how to build a case from MAPL2 captures, and cmpchk.py. SU2G porting notes: map each MAPL2 friendly target to its MAPL3 service (TRADV done, MTR to do, TRDIFF blocked) and note how Moist (GEOSgcm_GridComp#1566) provides MTR.
@pchakraborty pchakraborty changed the title Quiet MAPL logging in the GOCART2G regression cases Quiet MAPL logging in the GOCART2G regression cases; add regression README Nov 10, 2026
@pchakraborty
pchakraborty merged commit 40316d5 into release/MAPL-v3 Nov 10, 2026
14 checks passed
@pchakraborty
pchakraborty deleted the feature/pchakrab/regression-logging branch November 10, 2026 14:36
aoloso pushed a commit that referenced this pull request Nov 10, 2026
The merge of release/MAPL-v3 (c6b2e73) picked up #501's refactor of the
SU2G connection block, which split a single call into three
(child_items / _AERO / _AERO_DP). That hunk was also applied to the
carbonaceous block, where it clobbered the CA-specific code:

  - the "! AERO_DP" call lost its src_names/dst_names and became a
    duplicate of the child_items connection, so CA_<inst>_AERO_DP was
    never connected even though SetServices declares the import spec;
  - the child_items and _AERO connections were each registered twice.

The result was a SIGSEGV in ESMC_BaseGetInfo under
initialize_accept_transfer for
gocart+seasalt+dust+sulfate+nitrate+carbon-bc/run{1,2}. carbon-bc-sa was
unaffected because it runs against FakeGOCART rather than the real
GOCART2G parent, so this code path is not exercised there.

Restores the CA block to its pre-merge form; it is now byte-identical to
bcf089a. Unrelated to the CA.xx -> CA_xx rename.

Verified: ctest -L REGRESSION 31/31 pass.
aoloso pushed a commit that referenced this pull request Nov 10, 2026
#501 added logging.yaml and `pflogger: logging.yaml` to every GOCART2G
regression case, but the two carbon cases only exist on this branch, so
the merge left them out: they still ran with MAPL's default logging,
which writes each rank's warnings to warnings_and_errors.log under an MPI
lock and serializes the ranks.

Copies the shared logging.yaml (byte-identical to the other 12) into
carbon-bc-sa/run{1,2} and
gocart+seasalt+dust+sulfate+nitrate+carbon-bc/run{1,2}, and adds the
matching `pflogger: logging.yaml` line to each mapl.yaml. All 16 run
directories now have the same logging setup.

Also documents the carbon cases in regression/README.md, which #501 added
without them: a carbon-bc-sa row in the standalone table, a note that it
is the one case whose DataFor component is not named after its child
(DataForCA_internal.nc vs CA_bc_internal.nc), and a section for
gocart+seasalt+dust+sulfate+nitrate+carbon-bc noting that it is the only
case exercising the parent's carbonaceous connection block.

Effect: the "No configure file specified for logging layer" notice and
the per-rank field-dictionary warnings are gone, and the cases are ~30%
faster (run1 11.6s -> 7.8s, run2 11.5s -> 8.4s, carbon-bc-sa 3.0s ->
2.3s).

Verified: ctest -L REGRESSION 31/31 pass.
pchakraborty pushed a commit that referenced this pull request Nov 11, 2026
* Port of NI2G to MAPL3

* Added tests for  NI2G

* Added standa alone nitrate test

* Port of CA2G to MAPL3

* Added standalone carbon test

* Added combined carbon regression test

* Make nitrate-sa standalone test actually run: add DuSsStub_GridComp

NI2G's Initialize unconditionally reads custom 'radius'/'fnum' ESMF Info
attributes off its DU/SS import fields, which only the real DU2G/SS2G
components' Initialize routines attach. The generic configurable_gridcomp
'DataForNI' stand-in used by this harness can supply DU/SS field values
but has no mechanism to attach those attributes.

Add a small dedicated DuSsStub_GridComp that exports placeholder DU/SS
fields carrying the same radius/fnum Info DU2G/SS2G attach (values taken
from DU2G_instance_DU.yaml/SS2G_instance_SS.yaml), and wire it into
nitrate-sa/run1 and run2 (declared before FakeGOCART so its Initialize
runs first). Finishes bringing run2 to parity with run1's already-wired
DuSsStub connections, and removes DU/SS from run2/data-for-ni.yaml now
that DuSsStub supplies them.

With this in place NI2G's Initialize gets past the radius/fnum read and
only fails later at missing ExtData (opticsBands_NI.v2.1.0.RRTMGP.nc4),
a data-availability issue, not a code issue. Notes updated accordingly.

* Moved the DU & SS stub to regression. Read SS and DU from import nc4 file for NI standalone test

* Change to get run2 of Nitrate to pass

* added check whether to de-activate diurnal biomass burning

* Changes to get nitrate and carbon-bc tests to run

* Rename CA instances from CA.xx to CA_xx

Replace the MAPL2-era dotted separator in the carbonaceous aerosol
instance names with an underscore: CA.oc/CA.bc/CA.br -> CA_oc/CA_bc/CA_br.

Rationale:

- MAPL3 History uses '.' as the <gridcomp>.<variable> dive-down delimiter
  (HistoryUtilities.F90 replace_delimiter, which substitutes only the
  first '.'). HistoryGridComp_private.F90 states the contract directly:
  "Collection names are permitted to include period ('.') characters, but
  gridcomps are not." A dotted child name contradicts that, and composed
  names such as CA.oc.data are unparseable under a first-dot-wins grammar.
- The dot carried no information: nothing in GOCART ever split on it.
- CA_bcEXTTAU is a legal Fortran identifier, so the dot no longer forces
  the GOCART2G_StateSpecs.rc ALIAS column (kept for now only so existing
  pointer names are undisturbed).
- Removes GOCART's need for the '.' relaxation in MAPL's is_valid_name.

Covers child/instance names, CA2G_instance_* filenames and their symlinks
(AMIP/, AMIP.20C/, regression run1/, run2/), ESMF field, state and
info-key names, GOCART2G_StateSpecs.rc CA import rows, the carbon-bc-sa
and gocart+...+carbon-bc harnesses, and g2g_rst_generator.py. Also
includes the uppercase comparisons in GOCART2G_AerosolActivationMod.F90
(CA.BC/CA.OC/CA.BR), which match against an upper-cased aeroName.

Chem_AeroGeneric.F90's varname(6:) offset is unaffected: CA_bc is the same
5 characters wide as CA.bc was.

Not zero-diff for I/O: checkpoint, restart and History variable names
change, so regression baselines under REGRESSION_DATA_DIR and existing
GEOS restarts must be renamed or regenerated.

The commented-out .data instance variants were left untouched.

Verified: MAPL_GridCompSpecs_ACG.py regenerates all 39 CA import specs
with short_name 'CA_bcEXTTAU' and pointer 'bcexttau' unchanged;
Chem_Shared2G, CA2G_GridComp and GOCART2G_GridComp build and link cleanly
with ifx-stack, no new warnings.

* Rename data-driven instance variants from .data to _data

Follow-on to the CA.xx -> CA_xx rename, completing the removal of the
period from GOCART2G instance names:

  DU.data    -> DU_data
  SS.data    -> SS_data
  SU.data    -> SU_data
  NI.data    -> NI_data
  CA.oc.data -> CA_oc_data
  CA.bc.data -> CA_bc_data
  DU.test    -> DU_test

All of these are currently commented out in GOCART2G_GridComp.yaml /
GOCART2G_GridComp.rc and the regression GOCART2G_GridComp.yaml files, so
this is a naming change only. The `index(name, "data")` dispatch in
GOCART2G_GridCompMod.F90 and determine_data_driven() in
Chem_AeroGeneric.F90 match the bare substring "data" and are unaffected.

Composed names like CA.oc.data were the case MAPL3 History could not
parse at all: replace_delimiter() substitutes only the first '.', so a
two-dot child name has no valid <gridcomp>.<variable> reading.

Also fixes a typo in GOCART2G_GridComp.rc ("CA.bc.datac" -> CA_bc_data)
and updates the CA2G/SU2G porting notes.

Deliberately left alone: the unported legacy GOCART_GridComp (1G), which
is commented out of ESMF/CMakeLists.txt and keys off its own
`modifier = '.data'` against component names supplied by legacy configs
outside this repository. Also left alone: the AMIP.20C config
directories, which are config-set labels rather than gridcomp or
instance names and so do not collide with History's delimiter.

Verified: Chem_Shared2G, CA2G_GridComp and GOCART2G_GridComp build and
link cleanly with ifx-stack; all YAML parses; repo-wide sweep finds no
remaining dotted instance tokens outside historical prose.

* Fix bad merge resolution in the CA connection block

The merge of release/MAPL-v3 (c6b2e73) picked up #501's refactor of the
SU2G connection block, which split a single call into three
(child_items / _AERO / _AERO_DP). That hunk was also applied to the
carbonaceous block, where it clobbered the CA-specific code:

  - the "! AERO_DP" call lost its src_names/dst_names and became a
    duplicate of the child_items connection, so CA_<inst>_AERO_DP was
    never connected even though SetServices declares the import spec;
  - the child_items and _AERO connections were each registered twice.

The result was a SIGSEGV in ESMC_BaseGetInfo under
initialize_accept_transfer for
gocart+seasalt+dust+sulfate+nitrate+carbon-bc/run{1,2}. carbon-bc-sa was
unaffected because it runs against FakeGOCART rather than the real
GOCART2G parent, so this code path is not exercised there.

Restores the CA block to its pre-merge form; it is now byte-identical to
bcf089a. Unrelated to the CA.xx -> CA_xx rename.

Verified: ctest -L REGRESSION 31/31 pass.

* Bring the carbon regression cases in line with #501

#501 added logging.yaml and `pflogger: logging.yaml` to every GOCART2G
regression case, but the two carbon cases only exist on this branch, so
the merge left them out: they still ran with MAPL's default logging,
which writes each rank's warnings to warnings_and_errors.log under an MPI
lock and serializes the ranks.

Copies the shared logging.yaml (byte-identical to the other 12) into
carbon-bc-sa/run{1,2} and
gocart+seasalt+dust+sulfate+nitrate+carbon-bc/run{1,2}, and adds the
matching `pflogger: logging.yaml` line to each mapl.yaml. All 16 run
directories now have the same logging setup.

Also documents the carbon cases in regression/README.md, which #501 added
without them: a carbon-bc-sa row in the standalone table, a note that it
is the one case whose DataFor component is not named after its child
(DataForCA_internal.nc vs CA_bc_internal.nc), and a section for
gocart+seasalt+dust+sulfate+nitrate+carbon-bc noting that it is the only
case exercising the parent's carbonaceous connection block.

Effect: the "No configure file specified for logging layer" notice and
the per-rank field-dictionary warnings are gone, and the cases are ~30%
faster (run1 11.6s -> 7.8s, run2 11.5s -> 8.4s, carbon-bc-sa 3.0s ->
2.3s).

Verified: ctest -L REGRESSION 31/31 pass.

---------

Co-authored-by: Amidu O.  Oloso <aoloso@gs6101-bucy.gsfc.nasa.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

0 diff The changes in this pull request have verified to be zero-diff with the target branch. MAPL3 Skip Changelog Skip changelog enforcer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants