Context
Child of #2118. #2077 moved Ray Data actor stages to ActorPoolStrategy and added min/max/initial worker sizing. #2086 made ray_stage_spec, xenna_stage_spec, and num_workers configurable through ProcessingStage.with_(). #2110 changed CPU video stage reservations to enable Ray Data fusion.
Required coverage
- Fixed actor pools via
num_workers()
- Autoscaling actor pools via
MIN_WORKERS, MAX_WORKERS, and INITIAL_WORKERS
- Ray Data task-stage
TaskPoolStrategy
- Precedence/warnings when fixed and autoscaling settings conflict
- Xenna cluster-wide worker counts vs
num_workers_per_node and invalid combinations
- Shallow-merge and explicit-
None semantics for with_() stage-spec overrides
- Reserved
num_workers hook and ASR dataloader_num_workers rename
- Resource matching and Ray Data stage fusion implications
Target docs
api-reference/processing-stage.mdx
- Ray Data/Xenna executor and infrastructure references
- Per-stage runtime/configuration cross-links
- 26.06 migration/release notes
Acceptance criteria
Related PRs: #2077, #2086, #2110
Context
Child of #2118. #2077 moved Ray Data actor stages to
ActorPoolStrategyand added min/max/initial worker sizing. #2086 maderay_stage_spec,xenna_stage_spec, andnum_workersconfigurable throughProcessingStage.with_(). #2110 changed CPU video stage reservations to enable Ray Data fusion.Required coverage
num_workers()MIN_WORKERS,MAX_WORKERS, andINITIAL_WORKERSTaskPoolStrategynum_workers_per_nodeand invalid combinationsNonesemantics forwith_()stage-spec overridesnum_workershook and ASRdataloader_num_workersrenameTarget docs
api-reference/processing-stage.mdxAcceptance criteria
with_()examples cover resources, runtime env, worker counts, and stage specsRelated PRs: #2077, #2086, #2110