Skip to content

BEER: bank views fold 12 million pixels per update, independent of event rate #1272

Description

@SimonHeybrock

BEER's two detector views fold and merge the full 12-million-pixel bank on every update, at a cost independent of how many events arrived. This is the same pathology that PR #1270 removed for TBL's and ODIN's Timepix3, where it was what OOM-killed the service.

Measured

Real preprocessor plus registered workflow, one bank, on a devcontainer:

view source pixels output per update
bank_view 12,000,000 250x250 751 ms
panel_view 12,000,000 12x125x125 617 ms

The cost is entirely pixel-driven: bank_view takes 777 ms with 1,000 events per update and 790 ms with 500,000.

For scale, the pre-remap TBL path measures 861 ms on the same machine, and was observed at ~11 s on the beamline. Applying that ratio puts a single BEER bank view near 9 s. There are two banks and two views, so with all four started the service is well past the adaptive batcher's 8 s ceiling, which is the unbounded-backlog path of #378.

What is needed

Remapping event ids onto the coarse grid in the preprocessor, before pixel grouping, as DownsamplePixelIds already does. BEER cannot use it as it stands: resolve_downsampling requires a square grid with a power-of-two side and id = x * side + y, and BEER's bank is (panel=12, y=1000, x=1000).

Generalizing it to a contiguous mixed-radix grid with per-axis integer division would cover BEER, and NMX too. Worth noting that the power-of-two constraint exists only to make the source resolution safe to infer from observed event ids, because the geometry file is static and may be stale. BEER declares its ids in code (detector_pixel_ranges), so there is nothing to infer and no bound to enforce -- the inference machinery is not needed here, only the remap.

The ids already have the property the remap depends on: contiguous and 1-based, 1..12,000,000 for the south bank and 12,000,001..24,000,000 for the north.

Blocked on confirming the axis order

The remap inverts the file's own enumeration, so a wrong axis order silently transposes or scrambles the image. For TBL and ODIN this is checked against detector_number in the geometry artifact. BEER has no artifact to check: the banks carry depends_on = '.' in the NeXus baseline, and the pixel ranges are hardcoded. beer/views.py already records that the file does not reveal which of the two equally sized spatial axes x_pixel_offset indexes.

So this needs the instrument team to confirm the (panel, y, x) decomposition and the axis order before the remap can be trusted, in the same way ODIN's file needed checking before #1270 covered it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:workflowsInstrument configs, geometry, reduction and science logicperformanceLatency, throughput, CPU or memory cost

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions