Skip to content

[PAN2] Add layerwise NVFP4 inference for Blackwell - #54

Open
Uxito-Ada wants to merge 11 commits into
Tele-AI:mainfrom
Uxito-Ada:pan2-nvfp4
Open

Uxito-Ada wants to merge 11 commits into
Tele-AI:mainfrom
Uxito-Ada:pan2-nvfp4

Conversation

@Uxito-Ada

@Uxito-Ada Uxito-Ada commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add PAN2 layerwise NVFP4 conversion and loading for Blackwell GPUs.
  • Stream one quantized transformer block at a time while keeping the VAE on CUDA; activation quantization uses the TorchAO NVFP4 Triton path (MSLK Python-only package is supported).
  • Add official PAN2 prompt/config options to the benchmark and document conversion/inference examples.

Validation

The official PAN2 T2V example was used: 704x1248, 241 frames, 24 FPS, seed 123, swimmer prompt from inference/prompts/examples.json, and the PAN2 scheduler. BF16 could not fit on the 24 GiB RTX 5090. A real one-step NVFP4 run completed with 241 frames, 229.54 s end-to-end latency, 14.36 GiB peak allocated / 20.71 GiB reserved, and 1.05 generated FPS (0.044x realtime). The one-step output is attached below; it is a capacity/path validation, not a quality claim. A 50-step run was started and measured at roughly 83-92 s per denoise step, so the single-card block-offload profile is not realtime.

Official PAN2 NVFP4 RTX 5090 metrics

pan2-official-nvfp4-one-step.mp4

Model feasibility

MiniMax-H3-Image and LightX2V H3 were evaluated before adding a misleading integration: both use the same 33.1B H3-Image DiT. Official H3-Image lists 66.28 GB BF16 DiT weights, a 66.7 GB BF16 Qwen3-VL text encoder, and a 2.34 GB VAE. NVFP4 can reduce the DiT to roughly 17-19 GB, but the complete pipeline still cannot reside in a 24 GiB RTX 5090. LightX2V's official single-GPU profile therefore enables model-level CPU offload; its four-GPU profiles disable offload.

Generated reports and media are PR attachments only; no model output is committed to the repository.

5090 Feasibility Report

The detailed Chinese report is available locally at reports/pan2_5090_nvfp4_feasibility.md and summarized here:

  • PAN2 official workload: 704x1248, 241 frames, seed 123. One-step NVFP4 measured 229.54 s end-to-end, 14.36 GiB allocated, 20.71 GiB reserved, 1.05 generated FPS, 0.0437x realtime.
  • The 50-step run reached step 5 at 82.4--91.8 s/step before being stopped; projected denoise time is about 70--77 minutes.
  • MiniMax-H3-Image and LightX2V H3 use the same 33.1B DiT. Official component sizes are 66.28 GB BF16 DiT, 66.7 GB BF16 Qwen3-VL text encoder, and 2.34 GB VAE. NVFP4 reduces only the DiT to an estimated 17--19 GB; the complete pipeline still cannot reside in 24 GiB.
  • LightX2V's official one-GPU config enables model/text-encoder/VAE CPU offload; it is a runtime optimization, not a smaller model. Neither H3 option is a full-resident realtime candidate on this card.

This distinguishes measured PAN2 data from H3 estimates; no H3 estimate is presented as a local benchmark.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant