Repository navigation
Conversation
…ct-ai#1993) * chore: move env packages to dedicated `envs` extras group Move math-env and reverse-text from main dependencies into a new `envs` optional-dependencies group alongside all other environment packages used in configs (aime2024, aime2025, alphabet-sort, code-env, color-codeword, logic-env, math-python, math500, science-env, wiki-search). All sourced from the primeintellect index. Install with: uv sync --extra envs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove primeintellect/ prefix from env IDs in configs Envs are now installed directly via the envs extras group, so the primeintellect/ prefix is no longer needed for runtime installation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: mark primeintellect index as explicit Prevents uv from querying the primeintellect index for every package during resolution, drastically speeding up uv sync/lock. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Adds automated PR review using anthropics/claude-code-action@v1. Reviews run on PR open/sync/reopen and responds to @claude mentions. Co-authored-by: sami jaghouar <sami@primeintellect.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…rimeIntellect-ai#1997) Point pydantic-config to main which now includes proper string-to-type coercion for dict fields in BaseConfig (samsja/pydantic_config#23). Co-authored-by: sami jaghouar <sami@primeintellect.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…#2003) * fix: filter empty trajectory rollouts in scheduler Empty trajectories consume batch slots without producing training signal. Filter them after sampling so they don't count toward batch progress, and track the count as a metric for observability. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * rename to empty_rollouts for consistency Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove empty batch retry logic from orchestrator The scheduler already filters empty trajectories, so the orchestrator should not silently retry on empty batches — better to fail fast. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…2006) * feat: disk-backed image offloading for VLM orchestrator memory Replace base64 image data in rollout trajectories with file paths immediately after receiving from ZMQ. This eliminates the triangle duplication problem where 15-turn rollouts store 120 copies of screenshot base64 strings in memory. Images are written to {output_dir}/assets/images/{sha256}.png and deduped by content hash. The VLM image cache reads from disk instead of decoding from base64 strings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: add VLM memory benchmark for disk offloading Benchmark generates synthetic rollouts with triangle-duplicated screenshots, simulates ZMQ msgpack roundtrip, and measures RSS with and without disk-backed image offloading. Results with 32 rollouts, 15 turns, 512x384 screenshots (~770KB each): - Without offload: +2.9 GB (91 MB/rollout) - With offload: +7.8 MB (0.24 MB/rollout) - 99.7% memory reduction Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: ruff lint fixes in vlm_memory_bench Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: ruff format Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove benchmarks folder Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Revert "chore: remove benchmarks folder" This reverts commit 7991813. * revert: restore benchmarks folder, only remove vlm_memory_bench.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle file:// URLs in _extract_images_from_messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…#1987) * feature: switch fused lm head to sequence chunking * fix: deprecate old fused lm head chunk config * fix: update benchmark token chunk flag * fix: stream vocab shards inside sequence chunked lm head * fix: retune benchmark token chunk to 1024 * fix: treat benchmark memory as an upper bound * fix: remove fused lm head chunk alias * fix: remove token chunk size validation
…i#2000)" (PrimeIntellect-ai#2015) This reverts commit bcb504b.
…completion (PrimeIntellect-ai#2014) Empty trajectories were filtered after sampling from the buffer, which could yield incomplete groups and break the advantage computation's assumption that each group has exactly `rollouts_per_example` rollouts. Now empty trajectories are detected per-rollout as they complete. When one is found, `rollouts_to_schedule` is incremented so the group naturally re-fills via `_fill_inflight_requests`, and the group is only yielded once it has the full count of non-empty rollouts. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: per-env metrics logging with consistent naming convention
Adopt {metric}/{scope}/{stat} naming where scope is "all" (global) or
an env name. All metrics that were previously global-only now get
per-env breakdowns when multiple envs are present.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: always show per-env breakdown and drop reward std/median
Remove the nunique() > 1 gate so per-env metrics are always logged,
even with a single env. Remove std and median from reward stats.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: metrics/{env}/{metric} format, remove error type breakdown
- Change verifier/filter metrics to metrics/{env}/{metric} (no global, no stat suffix)
- Remove per-error-type logging (error/{error_type})
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove redundant batch/{env}/{env} keys
Env ratios only make sense at the global level (batch/all/{env}).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: separate solve/batch/off_policy metrics, flatten async_level
- Extract solve_none, solve_all, effective_batch_size as top-level
metrics with /all and /{env} breakdowns
- Move off_policy_level to off_policy_level/all/{stat}
- Make async_level a flat key
- batch/ now only contains env ratios and inflight/cancelled counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: per-env off_policy_level tracking
Add task field to InflightRolloutInfo so off_policy_level can be
reported per env as off_policy_level/{env}/{stat}.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: rename batch/ inflight/cancelled to progress/, simplify batch/{env}
- batch/inflight_rollouts, batch/inflight_samples, batch/cancelled_rollouts
moved to progress/ prefix
- batch/all/{env} simplified to batch/{env}
- async_level moved to scheduler/async_level
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: inline grouped_stats, separate global and per-env sections
Remove grouped_stats helper in favor of explicit inlined metrics.
Global metrics come first, per-env metrics in a dedicated section after.
All metrics consistently group by example_id before aggregating.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: validate that env name is not "all" (reserved for global metrics)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* update scheduler config
* fix: use example_id grouping for solve stats instead of index arithmetic
The previous index-based approach (index // rollouts_per_example)
assumed rollouts were contiguous by example, which is not guaranteed
when the scheduler interleaves groups from different envs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update scheduler test for new task field on InflightRolloutInfo
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add changelog entry for metrics logging overhaul
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: validate unique env names, use notna(), simplify metrics_df access
- Add unique env name validation on OrchestratorConfig and EvalConfig
- Use notna() instead of ~isna() for error rate
- Use index-based loc instead of copying metrics_df with task/example_id
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract compute_solve_rates, inline _off_policy_steps_by_task
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: inline solve_stats, use compute_solve_rates directly
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* also log per_env stop conditions
* minor
* style
* address bugbot
* min diff in changelog
* update changelog
* revert metrics_df
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…rimeIntellect-ai#2012) * feat: allow num_infer_nodes=0 with fake data in multi-node SLURM RL Skip launching inference servers and orchestrator when num_infer_nodes=0, running only the trainer with fake data. Useful for benchmarking multi-node training without needing inference resources. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: skip weight broadcast when using fake data No inference server exists in fake data mode, so broadcasting weights would either hang (NCCL) or write unnecessary files (filesystem). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* add zero grad tracking * address comments
* fix: re-schedule errored rollouts instead of filtering after group completion
Rollouts with a populated "error" field are now re-scheduled (like empty
trajectories) instead of being accepted into the group. Both conditions
are checked independently so a rollout that is both empty and errored
increments both counters. Adds scheduler/errored_rollouts metric.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove now-dead error/mean metrics from orchestrator
Errored rollouts are filtered in the scheduler before reaching the
batch, so error is always None in results_df. Remove the error column
and the error/all/mean and error/{env}/mean metrics. Error tracking
is now handled by scheduler/errored_rollouts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: per-env metrics for empty and errored rollouts
Track empty_rollouts and errored_rollouts counts per task. Report both
global (scheduler/empty_rollouts/all, scheduler/errored_rollouts/all)
and per-env (scheduler/empty_rollouts/{task}, scheduler/errored_rollouts/{task})
metrics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: move empty/errored rollout metrics to their own section
Use empty_rollouts/ and errored_rollouts/ prefixes instead of nesting
under scheduler/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* make httpx connect timout configurable * Set default connect_timeout to 30s and add CHANGELOG entry Co-authored-by: rasdani <rasdani@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: rasdani <rasdani@users.noreply.github.com>
…rimeIntellect-ai#2020) * fix: use repr for rollout error log to show full error chain The rollout error warning was using str formatting which only showed the short error_chain_str. Using repr shows the full error_chain_repr with the actual error message, making debugging much easier. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: print error_chain_repr instead of full error dict Show only the error_chain_repr from the rollout error dict in the warning log, which contains the actual useful error message rather than the entire dict. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * simplify: index directly into error dict Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: make errored_rollouts and empty_rollouts metrics rates per step Instead of logging raw counts, divide by total rollouts in the step to get the fraction of errored/empty rollouts. This makes the metrics comparable across steps with different numbers of rollouts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…imeIntellect-ai#2004) * Fix duplicate policy updates during cancellation * fix: cancel inflight scheduler policy update on stop --------- Co-authored-by: Sebastian <sebastian@primeintellect.ai>
…eIntellect-ai#2026) * feat: add Qwen 3.5 MoE model support with EP and CP integration Adds a custom Qwen 3.5 MoE (GatedDeltaNet + MoE) implementation with: - HF <-> PrimeRL weight conversion (fused/unfused expert formats) - Expert Parallelism support (MoE layers auto-detected by apply_ep) - Context Parallelism support (ring attention patching for flash attention layers) - Router replay via routed_experts argument - Unit tests for forward pass, weight roundtrip, router replay, and CP patching Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * remove vflm warning * feat: add custom VLM support for Qwen 3.5 MoE Extend Qwen3_5MoeForCausalLM to handle both text-only and VLM configs. When the config has a vision_config, the model creates a composite body (HF frozen vision encoder + custom PrimeRL text model). Weight conversion auto-detects VLM keys and remaps accordingly. - Unified model class (no separate VLM file) driven by config - Config-based VLM detection fallback for local model paths - VLM dispatch in get_model() via _CUSTOM_VLM_MAPPING - mini_moe.py preset for qwen3_5_moe_vlm testing - 6 new GPU tests covering forward/backward/weights/roundtrip/router/meta Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: add debug SFT config for real Qwen3.5-35B-A3B VLM Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: support VLM layer key format in weight broadcast VLM models use `model.language_model.layers.*` instead of `model.layers.*`, which crashed get_max_layer_num and caused filter_state_dict_by_layers to silently drop layer weights. Also fixes off-by-one in filter_state_dict_by_layers that skipped layer 0. * use registry-based approach * run ruff * chore: remove debug SFT configs for Qwen3.5 MoE Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: sami jaghouar <sami@primeintellect.ai> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: sami jaghouar <sami.jaghouar@gmail.com>
…ai#2025) Move max_retries from EvalEnvConfig to EnvConfig so it applies to both training and eval environments. The scheduler now looks up max_retries per task from the env config instead of hardcoding 0. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ai#2027) MergedColumnParallelLinearWithShardedLoRA.slice_lora_a was hardcoded for exactly 2 subloras, causing IndexError with Qwen3.5's 4-way qkvz fusion when fully_sharded_loras=True. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* make the loss equal weight per unmaked token * minor fixes * run ruff * prevent nan-poisoning * update --------- Co-authored-by: hallerite <git@hallerite.com>
…llect-ai#2034) Previously, custom losses were normalized by batch_size (number of sequences) while the default loss was normalized by the number of unmasked tokens. This made the effective learning rate depend on sequence length for custom losses. Now all loss types consistently normalize by the number of unmasked tokens. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* bump verifiers bumps to verifiers `de2ed50` PrimeIntellect-ai/verifiers@de2ed50 Signed-off-by: rasdani <73563550+rasdani@users.noreply.github.com> * update lock file --------- Signed-off-by: rasdani <73563550+rasdani@users.noreply.github.com>
…ntellect-ai#2035) * fix: handle DTensor weights in MultiLoRAGroupedExperts for EP compatibility When Expert Parallelism is enabled, distribute_module() converts all parameters to DTensors sharded across the EP mesh. The base GroupedExperts path handles this via the @expert_parallel decorator, but MultiLoRAGroupedExperts.forward() bypassed the decorator entirely, passing DTensors directly to torch._grouped_mm alongside regular tensors. Mirror the @expert_parallel decorator logic: convert DTensors to local shards, apply generate_permute_indices for token reordering and alignment padding, and unpermute the output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gather EP-sharded LoRA weights in state_dict and exclude base_layer from modules_to_save Two fixes for EP + LoRA MoE weight broadcast: 1. state_dict_for_adapter: call full_tensor() on DTensor LoRA weights before per-expert indexing. With EP, these weights are sharded across ranks and direct indexing triggers a DTensor redistribute assertion. 2. save_lora_config: exclude "base_layer" params from modules_to_save detection. After EP/FSDP wrapping, base layer params can have requires_grad=True, causing them to be incorrectly added to modules_to_save which vLLM rejects. Tested end-to-end: Qwen3-30B-A3B EP=4, LoRA rank 16, 30 steps at batch sizes 128 and 256 on H200. * fix: re-freeze base params after EP replaces them with DTensors ExpertParallel() replaces expert params with DTensors that default to requires_grad=True, overriding the freeze applied by LoRA setup. Re-run freeze_all_except_lora_and_specified after apply_ep to restore correct grad state. This also reverts the save_lora_config workaround (base_layer filter) since the root cause is now fixed. * fix: use new_zeros instead of new_empty for unpermuted output buffer new_empty leaves uninitialized memory in positions not covered by permuted_indices, which could leak garbage values into the output. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…i#2028) atexit handler ensures spawned env server processes are terminated when the orchestrator crashes, preventing orphaned subprocesses that keep the container alive and leave the run stuck in RUNNING.
…rimeIntellect-ai#2417) 66b277f hardcoded is_causal + window_size_left/right assuming flash_attn_3 3.0.0 API, but forward uses causal while backward uses is_causal in the installed package. Add _set_fa3_signature_params() that inspects the get_default_args() dict to set whichever key is actually present, covering both param name variants across FA3 versions.
deadsnakes has been pretty flakey during image builds on CI, and afaict i think uv already installs python so don't think it's actually required
) The bench reported `num_tokens = batch * seq // cp`, which under-reports tps and MFU by ~cp× when CP > 1. CP is sequence-sharded data parallelism (cp ranks split the seq dim of the same `batch × seq` tokens), so the unique training tokens per step is `batch × seq` regardless of cp — identical to the convention RL already uses (which excludes cp from the `dp` mesh used for token counting). Effect: cp=2 runs no longer look 2× slower than they really are in SFT bench output; numbers become directly comparable across cp values. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump vllm 0.19 → 0.20
vLLM 0.20.0 lands MoE LoRA refactor (#40338) and CPU offloading fix
(#39617) upstream — drop now-redundant prime-rl monkey patches:
monkey_patch_fused_moe_lora_dp and monkey_patch_offloading_connector_
cpu_block_count.
PyPI vllm-0.20.0 is built against CUDA 13 and links libcudart.so.13;
our env is pytorch-cu128 (CUDA 12). Pin vllm to the cu129 wheel from
wheels.vllm.ai (libcudart.so.12) for binary compat. Promote
torchvision/torchaudio to direct deps so [tool.uv.sources] pins them
to pytorch-cu128 (cu13 transitive resolution would break torch ops).
Add vllm = false exemption to the 7-day exclude-newer cooldown since
0.20.0 was published 2026-04-27, inside the window.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: migrate cu128 → cu130
Now that vllm 0.20.0 is the floor and PyPI ships it for CUDA 13
natively, drop our cu129-wheel detour and move the rest of the
GPU stack to cu130:
- pytorch-cu128{,-test} indices → pytorch-cu130{,-test}
torch/torchvision/torchaudio/flash_attn_3 follow.
- flash-attn extra: bump to v0.9.4 cu130torch2.11 prebuilt wheel.
- Drop the wheels.vllm.ai cu129 URL pin — PyPI vllm 0.20.0 (cu13)
is now the default.
- override-dependencies: nvidia-cudnn-cu12 → nvidia-cudnn-cu13.
Lockfile pulls torch 2.11.0+cu130, torchvision/torchaudio 0.26.0/2.11.0
+cu130, flash-attn 2.8.3+cu130torch2.11, vllm 0.20.0, and swaps every
nvidia-*-cu12 transitive for its cu13 counterpart.
Not addressed in this change (draft until ready):
- Host driver upgrade: 575.64.03 → ≥580.65.06 required for CUDA 13.
- samsja prebuilt wheels (deep-ep, deep-gemm, nixl-cu12) still
point at cu12 builds — runtime-incompatible with cu13. Need
rebuilds before merging.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(deps): bump vllm 0.20.0 → 0.20.1
Picks up the DeepSeek V4 patch release and a handful of bug fixes
(CUDA-graph max_num_batched_token, num_gpu_blocks_override, cumem
expandable_segments, structured-output kwargs). Strict patch — none
of the kept/dropped monkey-patches are touched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(deps): ship cu13 disagg wheels, switch nixl to PyPI
deep-ep / deep-gemm: rebuilt against CUDA 13.1, stamped with `.cu13`
local version, uploaded alongside the existing cu12 wheels at v0.5.0.
nixl: dropped the URL pin entirely. PyPI `nixl-cu13` 1.0.1 ships UCX
auditwheel'd with mlx5_ib / rdmacm / cuda transports — verified
end-to-end with a 2-node basic_two_peers.py run that picked rc_mlx5
for inter-node RMA. The custom install_nixl_from_source.sh path is
kept (now generates `nixl-cuMAJOR` per torch's CUDA major) for
clusters that need a vendored UCX, but it's no longer the default.
install_ep_kernels.sh:
- Falls back to /usr/local/cuda-${MAJOR} when torch reports 13.0
but only cuda-13 (-> 13.1) is installed; rejects toolkit/torch
major mismatches.
- Exports CPATH=$CUDA_HOME/include/cccl for CUDA 13, where CCCL
headers (e.g. <cuda/std/tuple>) moved into the cccl/ subdir and
torch's CUDAExtension doesn't add it on its own.
- Post-build --set-runpath /tmp/deepep_build/nvshmem/lib rewrites
the DT_RUNPATH on deep_ep_cpp.so to match the cu12 convention the
existing inference/RL templates already hook (they symlink the
user's NVSHMEM at that path on each compute node).
install_deep_gemm.sh: post-build cu-suffix stamp.
scripts/wheel_add_cu_suffix.py (new): repacks a wheel with
+...cuXX appended to its local version, optionally stripping or
rewriting DT_RUNPATH on every .so. Keeps METADATA / dist-info /
RECORD / filename consistent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(templates): fall back to ep_kernels_workspace nvshmem in disagg launchers
inference.sbatch.j2 and multi_node_rl.sbatch.j2 still set up the
/tmp/deepep_build/nvshmem/lib symlink the deep_ep wheel needs at
runtime, but now also try $PROJECT_DIR/ep_kernels_workspace/nvshmem/lib
(where install_ep_kernels.sh actually places NVSHMEM) if
$OUTPUT_DIR/nvshmem isn't staged.
Also adds the same hook to multi_node_rl.sbatch.j2's disagg block
(was previously only in inference.sbatch.j2, despite multi_node_rl
running deep_ep in-process when disagg is enabled).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: drop unused shutil import, apply ruff format
Caught by the Style workflow — `shutil` was speculatively imported
when wheel_add_cu_suffix.py was first sketched but never used, and
ruff's formatter wanted a couple of cosmetic tweaks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(deps): exempt torchvision/torchaudio from exclude-newer cooldown
Both are sourced from the `pytorch-cu130` index alongside `torch` and
`flash_attn_3`. That index doesn't publish upload dates, so any package
pulled from it must be exempt from the 7-day `exclude-newer` cooldown
or `uv lock` may fail to resolve when a new release lands inside the
window.
Caught by bugbot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(deps): refresh uv.lock metadata under uv 0.11.9
Previous lockfile was generated with uv 0.11.7. After the bugbot
fix added `torchvision`/`torchaudio` to `exclude-newer-package`,
CI (which runs uv 0.11.9) detected a metadata drift and failed
`uv sync --locked`, even though the resolved packages were
identical.
Re-locking with uv 0.11.9 reorders the `exclude-newer-package`
section and normalizes the `exclude-newer` value; no resolution
changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* add cu12 cudart to venv dep
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Jackmin801 <ongjackm@gmail.com>
…tellect-ai#2416) * feat: split prime-rl-configs into separate workspace package Goal: make `prime_rl.configs.*` pip-installable on its own (~13 deps) without pulling the full GPU/ML stack (~329 deps). Layout — uv workspace with two members: - root `prime-rl` (heavy stack, unchanged location at src/prime_rl/) - `packages/prime-rl-configs/` (slim wheel, owns prime_rl.configs.* and the small slim utils: utils/config.py, utils/logger.py, utils/validation.py) Mechanism — both wheels contribute to a PEP 420 namespace package `prime_rl/` (no __init__.py at the top level, nor under utils/). Each wheel ships disjoint files; pip merges them at install time. - `pip install prime-rl-configs` → schema only - `pip install prime-rl` → full stack (depends on prime-rl-configs) Code changes: - Move src/prime_rl/configs/ → packages/prime-rl-configs/src/prime_rl/configs/ - Move utils/{config,logger,validation}.py → packages/prime-rl-configs/src/prime_rl/utils/ - Move rgetattr/rsetattr from utils/utils.py → utils/config.py (was the only heavy import the configs needed; this breaks the dep) - Replace `Path(prime_rl.__file__).parent / "templates"` (None on a namespace package) with `find_package_resource("templates")` which iterates `prime_rl.__path__` to locate the heavy wheel's data dir - Delete src/prime_rl/__init__.py; move its `import _compat` side-effect into the actual entrypoint scripts (orchestrator.py, entrypoints/rl.py, entrypoints/sft.py — trainer/{rl,sft}/train.py already had it) - Dockerfile.cuda: also COPY packages/ so the workspace member is available during `uv sync` Verified: - `uv sync --all-extras` resolves and builds both wheels - 333 unit tests pass (none touched) - all 6 entrypoints (rl/sft/trainer/orchestrator/env-server/inference) load and print --help - fresh `pip install ./packages/prime-rl-configs` (no extras) installs 13 packages, imports all configs, parses examples/.../rl.toml with a [slurm] section, and sys.modules contains zero of: torch / transformers / vllm / wandb / ring_flash_attn / verifiers / prime / datasets / liger_kernel Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: add slim-configs-install job to cpu_tests Fresh venv → `uv pip install ./packages/prime-rl-configs` → assert that: - all 6 config modules import - sys.modules contains zero of: torch, transformers, vllm, wandb, ring_flash_attn, verifiers, prime, datasets, liger_kernel - find_package_resource("templates") returns None (the heavy wheel's data dir isn't shipped on slim) - a real example config with [slurm] parses; template_path stays None Guards the slim-import contract against future regressions where something under prime_rl.configs.* might re-introduce a heavy import. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(ruff): mark prime_rl as first-party for isort Without this, ruff's import sorter classifies prime_rl.configs.* as third-party (because those files now live under packages/, not src/), and groups them apart from prime_rl.utils.* / prime_rl.trainer.* etc. Telling ruff that prime_rl is a single first-party namespace keeps all prime_rl.* imports in one block, regardless of which workspace wheel ships the submodule. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(configs): drop logger dep, use stdlib warnings Configs were doing get_logger().warning(...) at validation time, which forced loguru into the slim wheel and a logger.py copy alongside it. None of these warnings need a configured logger — they're config-time deprecation notices and misconfiguration hints. Stdlib `warnings.warn` covers the use case without any runtime infrastructure. - Replace 7 `get_logger().warning(...)` call sites in configs/rl.py and configs/orchestrator.py with `warnings.warn(...)`. Deprecation messages use `DeprecationWarning`; the rest use the default `UserWarning`. - Move `prime_rl/utils/logger.py` back to the heavy `prime-rl` wheel (`src/prime_rl/utils/`). - Drop `loguru` from `prime-rl-configs` deps; add it to `prime-rl`. - Slim CI: stop importing `prime_rl.utils.logger` (it's no longer in the slim wheel) and add `loguru` to the forbidden-modules list. Slim install drops from 13 to 12 packages. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: import _compat in remaining ring_flash_attn entry points Integration tests like tests/integration/test_sft_lora.py import prime_rl.trainer.weights, which transitively pulled ring_flash_attn (via models layer / utils.cp) without prime_rl._compat ever running, producing the ImportError that failed CI. Add `import prime_rl._compat` to the two remaining top-level ring_flash_attn import sites: utils/cp.py and trainer/models/layers/ ring_attn.py. The trainer entrypoints (rl/sft/train.py) already had this import. `# ruff: noqa: I001` on both files because ruff's isort, with known-first-party=prime_rl, sorts the prime_rl._compat first-party import after the third-party ring_flash_attn import — which defeats the patch ordering. The shim must run before ring_flash_attn is imported, so the file order is intentional. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: pin pydantic-config to git fork in slim package The PyPI package named `pydantic-config` is a different project that doesn't expose BaseConfig / cli (those are in samsja's fork). Without this pin, plain `pip install prime-rl-configs` resolves the wrong package and configs.* imports crash with: ImportError: cannot import name 'BaseConfig' from 'pydantic_config' uv pip install accidentally worked because it reads the root workspace pyproject.toml's [tool.uv.sources] when installing from a git subdirectory, which redirected pydantic-config to the fork. Plain pip doesn't do that — it only sees the leaf pyproject.toml. PEP 508 direct reference works for both uv and pip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: install slim wheel from /tmp to bypass workspace source discovery The previous step did `uv pip install ./packages/prime-rl-configs` from inside the cloned repo. uv silently reads the root pyproject.toml's [tool.uv.sources] in that case and applies the pydantic-config git override — masking the issue an external pip-only user would hit. Build the wheel into /tmp/slim-wheel and install it from /tmp so uv can't discover the workspace. This catches dep-resolution bugs that only show up in a real external install. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: explain dist-name vs import-path in slim package README Distribution name (`prime-rl-configs`) and import path (`prime_rl.configs.*`) are different — easy to trip on. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rimeIntellect-ai#2429) DeprecationWarning is silenced by default outside __main__, so users never saw notices for max_tokens / [[orchestrator.env]] / [orchestrator.sampling]. FutureWarning is shown under default filters and is the right category for "will be removed in a future release" notices. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* bump vllm version * fix cuda version pin * correct flash attn * chore: drop monkey patches now upstream in vLLM 0.20 vLLM 0.20.1 ships fixes for the LoRA+MoE+DP>1 corruption (#40338) and the OffloadingConnector CPU block-count miscalculation (#39617), so remove the local monkey patches and the corresponding call site in transformers_v5_compat. Refresh stale "vLLM 0.16/0.19" comments. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com> Co-authored-by: hallerite <git@hallerite.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…lect-ai#2431) * install: check platform and exit early on unsupported platforms * handle stripped ansi escape prefix for color output * Revert "handle stripped ansi escape prefix for color output" This reverts commit b8850f7.
* Feat: deep-gemm-fp8 * Feat: cleanup * nit * Style * Feat: works now * Feat: fp8 grouped gemm works * Fix: skip unaligned + qwen3.5 * chore(deep_gemm): rebuild d30fc36 wheel against torch 2.11 / cu128 The wheel was built against torch 2.10 + cu128; the merge from main bumped torch to 2.11.0+cu128 / vllm to 0.20.1, breaking ABI compat with the old build. Rebuilt via scripts/install_deep_gemm.sh --wheel-dir tools/wheels. Also updates install_deep_gemm.sh: - default ref pinned to d30fc36 (was 477618c) - script header documents the wheel-dir flow used to refresh tools/wheels/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(deep_gemm): pin to v0.5.0 release URL (uploaded d30fc36 wheel) The d30fc36 wheel is now hosted on the v0.5.0 GitHub release alongside the existing 477618c build, so we can drop the local path-based pin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(fp8): make deep_gemm import optional for CPU-only envs deep_gemm only ships CUDA wheels; importing prime_rl on a CPU box currently fails with ImportError before any FP8 path is even touched. Wrap both call sites in try/except so the import is graceful — actual deep_gemm calls live inside autograd Function bodies that only run on GPU anyway. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(fp8): wire FP8 grouped GEMM into NemotronH; assert disabled for GPT-OSS Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…esponse tokens (PrimeIntellect-ai#2377) * feat: weighted length shaping over completion + tool-response tokens Generalize the length_shaping brevity bonus to a weighted combination of model-completion tokens and tool-response tokens, so the brevity signal can penalize tool-response token consumption (read from a harness metric like rlm_total_tool_response_tokens) independently from model output. - Add `tool_response_lengths` to AdvantageInputs (optional) - Add `length_shaping_completion_weight` (default 1.0) and `length_shaping_tool_response_weight` (default 0.0) to DefaultAdvantageConfig — defaults preserve existing behavior. - effective_lengths = a * completion + b * tool_response, fed to the existing correctness-gated brevity bonus. - Add get_tool_response_len() that pulls *_total_tool_response_tokens from rollout metrics and returns 0 if absent (no-op for non-tool envs). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(advantage): guard zero-cost division in efficiency shaping When `mean_correct_cost == 0` (e.g. tool-only shaping without a harness metric, or all-zero turn counts), `costs / mean_correct_cost` was 0/0 → NaN advantages. `clamp(0, 1)` doesn't recover from NaN. Detect zero mean and zero out the bonus, falling back to plain GRPO for that group. Also a latent bug for `length_penalty="turns"` if a group ever had all empty trajectories — same fix covers it. * fix(advantage): constrain weights to finite non-negative; cast rewards to float - Config: add `ge=0, allow_inf_nan=False` on both weight fields. Negative weights would invert the shaping (reward more tokens); inf/nan poison advantages downstream. - `default_advantage_fn`: cast rewards to float32. The previous `dtype=rewards.dtype` for costs truncated fractional weighted sums when rollouts emitted integer rewards. Casting also fixes a latent crash on int rewards in `_efficiency_shaping`'s `rewards.mean()` (Long unsupported). * refactor(advantage): discriminated length_penalty sub-config Replace the scalar `length_penalty: Literal["tokens","turns"] | None` plus two flat weight fields with a discriminated union on `type`: TokensLengthPenaltyConfig(completion_weight, tool_response_weight) TurnsLengthPenaltyConfig Default for the tokens config is (1.0, 1.0), making total-context shaping the default when token-shaping is enabled. The discriminator pattern matches the existing AdvantageConfig (default | custom) shape and keeps weight semantics scoped to the variant they apply to — no cross-field validator needed. CHANGELOG entry added. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ct-ai#2418) * feat(cp): add ulysses all-to-all path alongside ring Adds a `model.cp_style` arg ("ring" default | "ulysses") that selects between the existing ring CP and a new Ulysses all-to-all variant. Ulysses redistributes Q/K/V via two seq <-> head all-to-alls and runs vanilla flash attention on the full sequence with H/cp heads, so the attention kernel does not need to be CP-aware. Works OOB with FA2/3/4, linear attention, mamba, etc. Constraint: cp_size must divide both num_attention_heads and num_kv_heads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: ruff format ulysses_attn.py Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cp): refuse cp_style='ring' on linear-attn / Mamba models Ring CP is a softmax-attention algorithm (sequence ring all-gather of K/V). It cannot correctly drive non-softmax kernels (Qwen3.5 DeltaNet linear-attn, NemotronH Mamba). Previously the linear-attn CP setup hooks were called unconditionally under ring, which silently mixed two different CP layouts in the same model. Now: - Refuse cp_style='ring' at startup if the model has any linear/SSM attention layer, with a clear error pointing at cp_style='ulysses'. - Only call setup_hybrid_cp / setup_nemotron_h_cp under cp_style='ulysses'. - setup_sparse_mla_cp (still softmax) keeps working under both. Ulysses' all-to-all is purely on Q/K/V tensors, so the linear/SSM kernel runs unchanged on a sequence shard — this is the correct CP path for hybrid models. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…d` (PrimeIntellect-ai#2439) `temperature` is part of the forward signature but had no docstring entry, which trips the auto_docstring check. Add a description matching the convention used in the other model files. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ntellect-ai#2441) Alternative implementation of fp32 lm_head proposed by @Jackmin801 on PR PrimeIntellect-ai#2438: use torch.mm(..., out_dtype=torch.float32) (PyTorch >= 2.10) so the matmul accumulates and emits fp32 directly without zero-padding the bf16 operands or maintaining a separate fp32 weight copy. The win is purely in the epilogue: F.linear(bf16, bf16) accumulates in fp32 internally on tensor cores but truncates the output to bf16 before returning. Native out_dtype=fp32 keeps the full mantissa for the downstream softmax/sample. Removes the cached fp32 weight + tensor-version invalidation logic from PR PrimeIntellect-ai#2438 — not needed when there's no separate fp32 copy. Wired identically to PR PrimeIntellect-ai#2438: inference.enable_fp32_lm_head config flag -> additional_config["fp32_lm_head"] -> captured on LogitsProcessor.__init__ -> per-instance flag read in _get_logits. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
) `replace_linear_with_fp8_blockwise_linear` uses `re.search(pattern, name)` to decide which linears to keep in BF16. The default ignore list had "mlp.gate." which is a regex where `.` matches any character. As a result, dense MLPs' `mlp.gate_proj` (the trailing `.` matched `_`) were being skipped from FP8 conversion alongside the MoE router gate. For FP8 inference checkpoints (e.g. `zai-org/GLM-5-FP8`) where the dense MLP gate_proj IS quantized to FP8, this leaves the trainer's copy in BF16 — causing hidden-state divergence in early dense layers that propagates through the MoE router and inflates trainer↔inference KL mismatch. Escape the dots (`mlp\.gate\.`) so the pattern matches only the actual MoE router gate (`mlp.gate.weight`, `mlp.gate.e_score_correction_bias`) and not `mlp.gate_proj` / `mlp.gate_up_proj`. Verified on a 12-node GLM-5.1-FP8 RL run: trainer log now reports `Replaced 477` (was 474) with `mlp.gate_proj` no longer in the `first skipped(name)` list. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…PrimeIntellect-ai#2449) * make liveness timeout configurable; set default to 30s * server config already sets up the default 30s value
…#2453) * Fix: upcast to i64 * Fix: 1 more upcast
* Feat: update vllm * Fix: silu mul patch address overflow (DPEP16) * Feat: fix weight transfer * Chore: pin DeepEP DeepGEMM release wheels * nit: style
…ntellect-ai#2454) * Feat: support EPLB with quantize_in_weight_transfer * Feat: EPLB+quantize_in_weight_transfer * Nit: style
* Fix: add layerwise as default * Fix; cleanup * fix(filesystem): apply layerwise reload to filesystem transport Mirror the same fix as the NCCL path: replace `model.load_weights + process_weights_after_loading` with `load_weights_checkpoint_layerwise`. Without it, MoE + filesystem transport hits the same `Current vLLM config is not set` AssertionError at `worker/filesystem.py` on the first weight update. Verified end-to-end on Qwen3-30B-A3B-Thinking-2507 + reverse-text: filesystem broadcast returns 200 OK, orchestrator step 2 consumed new weights cleanly (off-policy level 1, reward 0.3583), trainer steady-state at ~29K tok/s, MFU 21%, no NaN, no errors. --------- Co-authored-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pattern as previous sync (a08acfe): strip CUDA-only deps (tilelang, flash-linear-attention, disagg, flash-attn CUDA wheels, nvidia-* overrides), replace pytorch-cu128 index with pytorch-rocm, point vllm/flash-attn at local ROCm wheels, remove aarch64 env and build isolation config. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.