Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
521c175
Fix a one-cell region offset when reading onto fine Float32 grids
xkykai Jul 25, 2026
fb1deaa
Add C3S 300 m leaf area index ingestion
xkykai Jul 25, 2026
3f0db4e
Note the record's tropical low bias in the leaf-area-index example
xkykai Jul 25, 2026
5327f75
Add MODIS land-cover ingestion and a class-aware leaf-area gap fill
xkykai Jul 28, 2026
bdad4fc
Place a composited dataset's samples at their window centres
xkykai Jul 28, 2026
6147185
Write a leaf area index of zero where there is no canopy
xkykai Jul 28, 2026
b7152cd
Drop the C3S 300 m leaf-area adapter
xkykai Jul 28, 2026
c650058
Merge branch 'main' into xk/lai-canopy-roughness
xkykai Jul 28, 2026
810368f
Drop references to the removed leaf-area adapter
xkykai Jul 28, 2026
080b564
Use American spellings throughout
xkykai Jul 28, 2026
4927283
Move product-specific composite date rules into their adapters
xkykai Jul 29, 2026
27484bf
Split gap filling along time out of inpainting.jl
xkykai Jul 29, 2026
bd7b475
Refactor time averaging functions and improve metadata handling for w…
xkykai Jul 29, 2026
dbb2808
Enhance Copernicus land albedo handling with dekad window functions a…
xkykai Aug 6, 2026
8f62032
Refactor gap filling parameters and improve documentation
xkykai Aug 6, 2026
720c620
Refactor variable names for clarity and consistency across tests and …
xkykai Aug 7, 2026
18ec447
Merge branch 'main' into xk/lai-canopy-roughness
xkykai Aug 17, 2026
c524911
Merge branch 'main' into xk/lai-canopy-roughness
xkykai Aug 17, 2026
5938ea1
Handle seasonal climatologies as whole cycles and tighten MODIS/nativ…
xkykai Aug 17, 2026
f9a2c51
Merge branch 'xk/lai-canopy-roughness' of https://github.com/Numerica…
xkykai Aug 17, 2026
82dea95
Refactor MODIS land cover tests for clarity and accuracy
xkykai Aug 17, 2026
05ddf80
Refactor and improve documentation in DataWrangling modules
xkykai Aug 17, 2026
e86154d
Add MODIS land product processing functions and utilities
xkykai Aug 17, 2026
1df83e0
Remove unnecessary newline in lai_screened function definition
xkykai Aug 17, 2026
b80a1bf
Refactor MODIS land cover functions: remove unused landcover_change_f…
xkykai Aug 17, 2026
6d2dc0f
Add donor pools for seasonal gap filling and update related functions
xkykai Aug 17, 2026
aa6330b
Refactor documentation in earthdata and gap_filling modules: streamli…
xkykai Aug 17, 2026
db8672a
Update documentation in gap_filling.jl: clarify parameter description…
xkykai Aug 17, 2026
32a23ff
Refactor gap filling and donor pool functions: rename to follow Pasca…
xkykai Aug 17, 2026
8b3959e
Refactor donor pools documentation: enhance clarity and detail for se…
xkykai Aug 17, 2026
30004db
Cache MODIS granules beside the reprojected files (#568)
xkykai Aug 24, 2026
e8e2439
Cache the seasonal gap fill beside its lattice (#569)
xkykai Aug 24, 2026
b93b503
Land MCD12Q1 class fractions and the majority class on a model grid (…
xkykai Sep 3, 2026
00cce77
Merge main into xk/lai-canopy-roughness and address review
xkykai Sep 4, 2026
9929b8e
Carry the reducer on MODISLAIClimatology and in its cache filename (#…
xkykai Sep 4, 2026
a96d403
Resolve PR #424 conflicts with current main
kaiyuan-cheng Sep 22, 2026
886e0e4
Merge main into xk/lai-canopy-roughness
xkykai Oct 6, 2026
8198f75
Average calendar windows with Oceananigans' time_average
xkykai Oct 6, 2026
186fe25
Compare averaged times on the host so the test runs on GPU
xkykai Oct 7, 2026
3e3995f
Merge remote-tracking branch 'origin/main' into xk/lai-canopy-roughness
kaiyuan-cheng Oct 13, 2026
a83ba73
Fix physics issues in the MODIS leaf-area gap fill and land-cover regrid
kaiyuan-cheng Oct 14, 2026
f8ef338
Fix class remapping on model grids and denial-score documentation
kaiyuan-cheng Oct 16, 2026
d437162
Merge origin/main into xk/lai-canopy-roughness
kaiyuan-cheng Oct 16, 2026
4c6c29b
Simplify the MODIS land, gap-fill and class-fraction code
kaiyuan-cheng Oct 16, 2026
962ba3b
Regrid class fractions onto the columns of three-dimensional grids
kaiyuan-cheng Oct 17, 2026
644e2e5
Leave the seasonal gap fill's time axis open unless asked to wrap
kaiyuan-cheng Oct 17, 2026
838cd2c
Key the leaf-area climatology files on the quality screen
kaiyuan-cheng Oct 17, 2026
76adaa7
Mark the class-fraction area test broken until Oceananigans regrids b…
kaiyuan-cheng Oct 17, 2026
542dae9
Land MODIS leaf area on model grids as a mean over valid land area
kaiyuan-cheng Oct 17, 2026
8b818b0
Fix Copernicus albedo composite averaging windows
kaiyuan-cheng Oct 19, 2026
14c6c1f
Preserve missing MODIS retrievals and clarify seasonal scaling
kaiyuan-cheng Oct 19, 2026
fbb1ae5
Merge origin/main into xk/lai-canopy-roughness
kaiyuan-cheng Oct 19, 2026
231c149
Fix MODIS discovery, climatology caching, and dataset field handling
kaiyuan-cheng Oct 19, 2026
38f5368
Merge branch 'main' into xk/lai-canopy-roughness
kaiyuan-cheng Oct 20, 2026
b2d22a8
Address review of the MODIS gap fill, time averaging and class fractions
kaiyuan-cheng Oct 20, 2026
ce4778f
Key MODIS inpainting caches by variable and screen, and fix 2-D locat…
kaiyuan-cheng Oct 21, 2026
757d844
Merge branch 'main' into xk/lai-canopy-roughness
kaiyuan-cheng Oct 21, 2026
b3738f5
Copy onto the native grid in one place, and share one class_fractions
kaiyuan-cheng Oct 21, 2026
eec3cfc
Fix seasonal coverage and MODIS dates, and simplify tests
kaiyuan-cheng Oct 21, 2026
9440387
clean up
kaiyuan-cheng Oct 21, 2026
e7fe06b
Restore MODIS constructor signatures and document each gap-fill method
kaiyuan-cheng Oct 22, 2026
938e99b
Merge origin/main into xk/lai-canopy-roughness
kaiyuan-cheng Oct 22, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ env:
COPERNICUSMARINE_SERVICE_PASSWORD: ${{ secrets.COPERNICUSMARINE_SERVICE_PASSWORD }}
ECCO_USERNAME: ${{ secrets.ECCO_USERNAME }}
ECCO_WEBDAV_PASSWORD: ${{ secrets.ECCO_WEBDAV_PASSWORD }}
EARTHDATA_USERNAME: ${{ secrets.EARTHDATA_USERNAME }}
EARTHDATA_PASSWORD: ${{ secrets.EARTHDATA_PASSWORD }}
JULIA_CUDA_USE_COMPAT: false

jobs:
Expand Down
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ NCDatasets = "0.12, 0.13, 0.14"
NaturalEarth = "0.1"
NetworkOptions = "1"
OceanBioME = "0.19.1, 0.20"
Oceananigans = "0.111, 0.112, 0.113, 0.114"
Oceananigans = "0.112, 0.113, 0.114"
OffsetArrays = "1.14"
PrecompileTools = "1"
PythonCall = "0.9.28"
Expand Down
2 changes: 2 additions & 0 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ examples = [
# Example("Differentiable ERA5-forced slab land", "era5_forced_slab_land"; build_always=false, gpu=false),
Example("ERA5-forced Terrarium land", "era5_forced_terrarium_land"; build_always=false, gpu=false),
Example("ERA5 downscaling with Breeze", "breeze_downscaling_era5"; build_always=true, gpu=true),
Example("MODIS leaf area index", "modis_leaf_area_index"; build_always=false, gpu=false),
Example("MODIS land cover and gap filling", "modis_landcover_gap_fill"; build_always=false, gpu=false),
]

# Developer examples from docs/src/developers/ directory
Expand Down
4 changes: 3 additions & 1 deletion docs/src/Metadata/metadata_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,4 +217,6 @@ NumericalEarth currently ships connectors for the following data products:
| `GHSBuiltS` | `:built_up_fraction` — plan-area built-up fraction (10 m or 100 m) over a regional window; reprojected from GHSL World-Mollweide (needs `ArchGDAL`) | [GHSL GHS-BUILT-S](https://human-settlement.emergency.copernicus.eu) |
| `GHSBuilt` | `:built_up_fraction` and `:building_volume` (m³ per m²): the GHS-BUILT-S fraction and its product with the epoch-2018 ANBH height, paired on the built-up surface pixels over a regional window; reprojected from GHSL World-Mollweide (needs `ArchGDAL`) | [GHSL GHS-BUILT-S and GHS-BUILT-H](https://human-settlement.emergency.copernicus.eu) |
| `ESAWorldCover` | `:landcover_class`, `:vegetation_fraction`, and a per-class `:<class>_fraction` — 10 m land-cover classification aggregated block-wise over a regional window (needs `ArchGDAL`) | [ESA WorldCover](https://esa-worldcover.org/en/data-access) |

| `MCD15A2H` | `:leaf_area_index`, `:fpar`, `:leaf_area_index_uncertainty`, `:landcover_code` — 8-day 500 m MODIS composites (needs `ArchGDAL`) | [MCD15A2H V061](https://www.earthdata.nasa.gov/data/catalog/lpcloud-mcd15a2h-061) |
| `MODISLAIClimatology` | 46-period seasonal climatology of an [`MCD15A2H`](@ref) variable | [MCD15A2H V061](https://www.earthdata.nasa.gov/data/catalog/lpcloud-mcd15a2h-061) |
| `MCD12Q1` | `:landcover_class`, `:quality_flag`, `:land_water_mask` — annual 500 m land cover on [`MCD15A2H`](@ref)'s lattice | [MCD12Q1 V061](https://www.earthdata.nasa.gov/data/catalog/lpcloud-mcd12q1-061) |
7 changes: 7 additions & 0 deletions docs/src/appendix/notation.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,12 @@ Symbols introduced by [`VariablySaturatedHydrology`](@ref),
| ``\alpha`` | `α` | albedo | Surface reflectivity (–) |
| ``\epsilon`` | `ϵ` | emissivity | Surface emissivity (–) |

## Vegetation properties

| Math | Code | Property | Description |
|:----:|:----:|:---------|:------------|
| ``𝒜`` | `𝒜` | leaf area index | One-sided leaf area per unit ground area (m² m⁻²) |

## Similarity theory / surface layer

| Math | Code | Property | Description |
Expand Down Expand Up @@ -366,6 +372,7 @@ Most symbols can be entered in the Julia REPL and in editors with Julia support
| `𝒬` | `\scrQ` | Script Q (heat flux) |
| `ℐ` | `\scrI` | Script I (radiative intensity) |
| `ℒ` | `\scrL` | Script L (latent heat) |
| `𝒜` | `\scrA` | Script A (area index) |
| `𝓃` | `\scrn` | Script n (pore-size uniformity) |
| `𝓂` | `\scrm` | Script m (van Genuchten `1 - 1/𝓃`) |
| `τ` | `\tau` | Tau (kinematic stress) |
Expand Down
636 changes: 636 additions & 0 deletions examples/modis_landcover_gap_fill.jl

Large diffs are not rendered by default.

355 changes: 355 additions & 0 deletions examples/modis_leaf_area_index.jl

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions ext/NumericalEarthArchGDALExt/NumericalEarthArchGDALExt.jl
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ using NumericalEarth.DataWrangling.GHSL: GHSBuilt, GHSBuiltH, GHSBuiltS, GHSLMet
ghsl_regional_raster, built_surface_to_fraction,
mask_building_height, ghsl_pixel_window, download_GHSL_cache,
longitude_latitude_to_mollweide, MOLLWEIDE_RADIUS
using NumericalEarth.DataWrangling.MODISLand: MODISLand, granule_urls, regional_lattice,
stored_granule_layers, write_lattice_file
using NumericalEarth.DataWrangling.OpenLandMap: assemble_raster_window, raster_window_indices,
validate_wgs84_longitude_latitude, validate_geographic_northup
using NumericalEarth.DataWrangling.WorldCover: ESAWorldCoverMetadatum, version_year, version_string,
Expand All @@ -48,6 +50,7 @@ include("ibcao.jl")
include("asterged.jl")
include("globfp3d.jl")
include("ghsl.jl")
include("modis_land.jl")
include("openlandmap.jl")
include("worldcover.jl")

Expand Down
73 changes: 73 additions & 0 deletions ext/NumericalEarthArchGDALExt/modis_land.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#####
##### MODIS land products — Earthdata granule fetch + sinusoidal HDF-EOS reprojection
#####
##### A region spans several sinusoidal tiles, so every granule covering it is fetched and
##### the layers are mosaicked and warped to the region's latitude-longitude window in one
##### `gdalwarp` call each. The digital numbers are written through unchanged; masking,
##### quality screening, and scaling happen on read, where the product's decode rules live.
#####

# Locate the `HDF4_EOS:EOS_GRID:"…":<grid>:<layer>` subdataset without hardcoding the
# product's grid-group name.
function modis_subdataset(granule_path, layer)
ArchGDAL.read(granule_path) do ds
for entry in ArchGDAL.metadata(ds; domain = "SUBDATASETS")
occursin("_NAME=", entry) || continue
name = split(entry, "_NAME="; limit = 2)[2]
endswith(name, ":" * layer) && return name
end
error("Layer $(layer) not found among the HDF-EOS subdatasets of $(granule_path).")
end
end

# Mosaic one layer of `granule_paths` onto the regional lattice. Nearest-neighbor
# resampling throughout: every layer is either a bit-packed quality byte or a digital number
# whose out-of-range codes carry meaning, and averaging either would invent values. A lattice
# cell is 463 m north-south but 463 cos φ m east-west, so away from the equator each 463 m
# sinusoidal pixel fills about 1 / cos φ neighboring cells. Uncovered cells take the fill
# code 255, which the read path rejects on both the digital numbers and the quality bytes.
function warp_modis_layer(granule_paths, layer, lattice)
sources = ArchGDAL.IDataset[]

warped = try
for path in granule_paths
push!(sources, ArchGDAL.read(modis_subdataset(path, layer)))
end

ArchGDAL.gdalwarp(sources,
["-t_srs", "EPSG:4326",
"-te", string(lattice.west), string(lattice.south),
string(lattice.east), string(lattice.north),
"-ts", string(lattice.Nx), string(lattice.Ny),
"-r", "near",
"-ot", "Byte",
"-dstnodata", "255"]) do destination
ArchGDAL.read(destination, 1)
end
finally
foreach(ArchGDAL.destroy, sources)
end

# GDAL writes rows north→south; flip to the ascending latitude axis of the stored file.
return reverse(warped, dims = 2)
end

function MODISLand.modis_granules_to_netcdf(metadatum::MODISLand.MODISLandMetadatum, nc_path)
lattice = regional_lattice(metadatum)
urls = granule_urls(metadatum)

# A sinusoidal tile spans 10° and carries every layer of the product, so the same
# granules serve neighboring regions, sibling variables, and every year a climatology
# composites. Keeping them beside the warped files makes those warps a local operation.
granule_cache = mkpath(joinpath(dirname(nc_path), "granules"))
fetched = count(url -> !isfile(joinpath(granule_cache, basename(url))), urls)
fetched > 0 && @info string("Fetching ", fetched, " of ", length(urls),
" MODIS granules for ", metadatum.dates, "...")

granule_paths = [earthdata_download_cached(url, granule_cache) for url in urls]
layers = Dict(layer => warp_modis_layer(granule_paths, layer, lattice)
for layer in stored_granule_layers(metadatum.dataset))
write_lattice_file(nc_path, lattice, layers)

return nothing
end
119 changes: 57 additions & 62 deletions src/DataWrangling/CopernicusLandAlbedo/CopernicusLandAlbedo.jl
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@ module CopernicusLandAlbedo

export CopernicusAlbedo, CopernicusAlbedoClimatology, build_monthly_climatology!

using Dates: Dates, DateTime, Month, year, month, daysinmonth
using Dates: Dates, DateTime, Day, Month, year, month, daysinmonth
using DocStringExtensions: TYPEDSIGNATURES
using Downloads: Downloads
using NCDatasets: NCDataset, defVar, nomissing
using Oceananigans: Center
using Oceananigans.DistributedComputations: @root

using ..DataWrangling: DataWrangling, Metadata, Metadatum, BoundingBox,
metadata_path, default_download_directory, native_convention_longitude, unzip
metadata_path, default_download_directory,
native_convention_longitude, native_cell_range, write_atomically, unzip

# TODO: move `group_by_calendar_month` and `is_zip` (product-agnostic download helpers) from
# ERA5 into DataWrangling proper, so dataset modules don't reach across siblings.
Expand Down Expand Up @@ -59,6 +60,9 @@ series derived from SPOT/VGT and PROBA-V observations. Provides `:albedo`, the b
blue-sky albedo blended from the black-sky (`AL_DH_BB`) and white-sky (`AL_BH_BB`)
broadband albedos with diffuse fraction `diffuse_fraction`.

Each sample uses a trailing 20-day composite ending on its nominal date. Successive
composites overlap, and the retrieval also uses prior BRDF estimates.

Files are global on a regular 1/112° latitude-longitude grid; build the `Metadata`
with a lon/lat [`BoundingBox`](@ref) to window a region at read time. Downloads come
from the C3S `satellite-albedo` catalog entry and require the CDSAPI backend:
Expand Down Expand Up @@ -171,6 +175,14 @@ DataWrangling.all_dates(::CopernicusAlbedo, variable) = copernicus_albedo_ten_da
# 12 climatological months; the year is arbitrary, only the month matters.
DataWrangling.all_dates(::CopernicusAlbedoClimatology, variable) = [DateTime(2018, m, 1) for m in 1:12]

DataWrangling.is_seasonal_climatology(::CopernicusAlbedoClimatology) = true

# C3S v2 PUGS §2.5: the stamp labels the last observation of a 20-day composite.
DataWrangling.averaging_window(md::Metadatum{<:CopernicusAlbedo}) =
(DateTime(md.dates) - Day(19), DateTime(md.dates) + Day(1))

DataWrangling.window_span(::Metadata{<:CopernicusAlbedo}) = nothing

# Each climatological month averages every ten days falling in it, so it spans the calendar month.
DataWrangling.averaging_window(md::Metadatum{<:CopernicusAlbedoClimatology}) =
DataWrangling.calendar_month_window(md)
Expand Down Expand Up @@ -254,25 +266,24 @@ source are preserved.
function repack_albedo_pair(blacksky, whitesky, destination_names, filepath, expected_size)
blacksky_path, blacksky_name = blacksky
whitesky_path, whitesky_name = whitesky
staging_path = filepath * ".tmp"

NCDataset(staging_path, "c") do destination
NCDataset(blacksky_path) do source
λ = nomissing(source[coordinate_name(source, ("lon", "longitude"))][:])
φ = nomissing(source[coordinate_name(source, ("lat", "latitude"))][:])
(length(λ), length(φ)) == expected_size ||
error("CGLS albedo file grid $((length(λ), length(φ))) does not match ",
"the expected $expected_size; the dataset traits need updating.")
defVar(destination, "lon", λ, ("lon",))
defVar(destination, "lat", φ, ("lat",))
copy_packed_variable!(destination, source, blacksky_name, destination_names[1])
end
NCDataset(whitesky_path) do source
copy_packed_variable!(destination, source, whitesky_name, destination_names[2])

write_atomically(filepath) do staging_path
NCDataset(staging_path, "c") do destination
NCDataset(blacksky_path) do source
λ = nomissing(source[coordinate_name(source, ("lon", "longitude"))][:])
φ = nomissing(source[coordinate_name(source, ("lat", "latitude"))][:])
(length(λ), length(φ)) == expected_size ||
error("CGLS albedo file grid $((length(λ), length(φ))) does not match ",
"the expected $expected_size; the dataset traits need updating.")
defVar(destination, "lon", λ, ("lon",))
defVar(destination, "lat", φ, ("lat",))
copy_packed_variable!(destination, source, blacksky_name, destination_names[1])
end
NCDataset(whitesky_path) do source
copy_packed_variable!(destination, source, whitesky_name, destination_names[2])
end
end
end

mv(staging_path, filepath; force = true)
return nothing
end

Expand All @@ -286,25 +297,10 @@ end

#####
##### Reading — blend the black-sky/white-sky pair into the blue-sky array. Regional
##### reads hyperslab exactly the native-grid cell window off disk with no global
##### reads take exactly the native-grid cell window off disk with no global
##### materialization, so `retrieve_data` and `read_file_coords` must window identically.
#####

# 1-based native cell range covered by `bbox` on the axis `(left, right)` split into `N`
# cells — must match `restrict()` in metadata_field.jl (returned count = `i⁺ - i⁻`, so the
# cell range `(i⁻+1):i⁺` has that same length, pinning the region offset to di = dj = 0).
function albedo_cell_range(bbox, interfaces, N)
left, right = interfaces
Δ = (right - left) / N
i⁻ = clamp(floor(Int, (bbox[1] - left) / Δ - 1/2), 0, N)
i⁺ = clamp(ceil( Int, (bbox[2] - left) / Δ + 1/2), 0, N)
if i⁺ ≤ i⁻
i⁺ = min(i⁻ + 1, N)
i⁻ = max(i⁺ - 1, 0)
end
return (i⁻ + 1):i⁺
end

"""
albedo_read_window(metadatum)

Expand All @@ -326,8 +322,8 @@ function albedo_read_window(metadatum)
left, right = native_longitude
span = bbox_longitude[2] - bbox_longitude[1]
(span == 360 || (bbox_longitude[1] ≥ left && bbox_longitude[2] > right)) && return nothing
icols = albedo_cell_range(bbox_longitude, native_longitude, Nx)
jrows = albedo_cell_range(region.latitude, DataWrangling.latitude_interfaces(metadatum), Ny)
icols = native_cell_range(bbox_longitude, native_longitude, Nx)
jrows = native_cell_range(region.latitude, DataWrangling.latitude_interfaces(metadatum), Ny)

return icols, jrows
end
Expand Down Expand Up @@ -357,7 +353,7 @@ function DataWrangling.retrieve_data(metadatum::CopernicusAlbedoMetadatum)
icols, jrows = window
_, Ny, _ = size(metadatum.dataset, metadatum.name)
# The file stores latitude north→south, so the ascending cells `jrows` sit at file
# rows (Ny − last + 1):(Ny − first + 1). Only this hyperslab leaves disk.
# rows (Ny − last + 1):(Ny − first + 1). Only this window leaves disk.
file_rows = (Ny - last(jrows) + 1):(Ny - first(jrows) + 1)
NCDataset(path) do ds
α_bs = copernicus_albedo_decode.(ds[blacksky_name][icols, file_rows])
Expand Down Expand Up @@ -435,35 +431,34 @@ function write_monthly_mean(filepath, source_paths, variable_names, latitude_chu
nomissing(ds["lon"][:]), nomissing(ds["lat"][:])
end
Nx, Ny = length(λ), length(φ)
staging_path = filepath * ".tmp"

NCDataset(staging_path, "c") do destination
defVar(destination, "lon", λ, ("lon",))
defVar(destination, "lat", φ, ("lat",))

for variable_name in variable_names
monthly_mean = defVar(destination, variable_name, Float32, ("lon", "lat");
deflatelevel = 2, shuffle = true)

for j in 1:latitude_chunk:Ny
rows = j:min(j + latitude_chunk - 1, Ny)
Σα = zeros(Float32, Nx, length(rows))
n = zeros(Int32, Nx, length(rows))

for path in source_paths
NCDataset(path) do ds
α = copernicus_albedo_decode.(ds[variable_name][:, rows])
@. Σα += ifelse(isnan(α), 0f0, α)
@. n += !isnan(α)

write_atomically(filepath) do staging_path
NCDataset(staging_path, "c") do destination
defVar(destination, "lon", λ, ("lon",))
defVar(destination, "lat", φ, ("lat",))

for variable_name in variable_names
monthly_mean = defVar(destination, variable_name, Float32, ("lon", "lat");
deflatelevel = 2, shuffle = true)

for j in 1:latitude_chunk:Ny
rows = j:min(j + latitude_chunk - 1, Ny)
Σα = zeros(Float32, Nx, length(rows))
n = zeros(Int32, Nx, length(rows))

for path in source_paths
NCDataset(path) do ds
α = copernicus_albedo_decode.(ds[variable_name][:, rows])
@. Σα += ifelse(isnan(α), 0f0, α)
@. n += !isnan(α)
end
end
end

monthly_mean[:, rows] = @. ifelse(n == 0, NaN32, Σα / n)
monthly_mean[:, rows] = @. ifelse(n == 0, NaN32, Σα / n)
end
end
end
end

mv(staging_path, filepath; force = true)
return nothing
end

Expand Down
5 changes: 5 additions & 0 deletions src/DataWrangling/CopernicusLandAlbedo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,11 @@ on rebuild.

## Notes

- This adapter requests C3S version 2. Its [Product User Guide, section 2.5](https://dast.copernicus-climate.eu/documents/satellite-albedo/D3.3.4-v2.0_PUGS_CDR_SA_MULTI_SENSOR_v2.0_PRODUCTS_v1.2.pdf)
describes a trailing 20-day composite ending on the file's nominal date, with prior
BRDF estimates also contributing. Consecutive composites overlap, so `window_bounds`
rejects them. Monthly climatologies average the composite estimates assigned to each
nominal month; they are not exact calendar-month means of the underlying observations.
- Ocean and unretrieved pixels are `NaN` (no inpainting by default — this is a land dataset).
- Downloads are global and shared across regions; regional windowing happens at read
time via the `BoundingBox` machinery.
Loading
Loading