Skip to content

[PAN2] Add layerwise NVFP4 inference for Blackwell - #53

Closed
Uxito-Ada wants to merge 10 commits into
Tele-AI:mainfrom
Uxito-Ada:feat/minimax-h3-image-nvfp4
Closed

Uxito-Ada wants to merge 10 commits into
Tele-AI:mainfrom
Uxito-Ada:feat/minimax-h3-image-nvfp4

Conversation

@Uxito-Ada

@Uxito-Ada Uxito-Ada commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add PAN2 layerwise NVFP4 conversion and loading for Blackwell GPUs.
  • Stream one quantized transformer block at a time while keeping the VAE on CUDA; activation quantization uses the TorchAO NVFP4 Triton path (MSLK Python-only package is supported).
  • Add official PAN2 prompt/config options to the benchmark and document conversion/inference examples.

Validation

The official PAN2 T2V example was used: 704x1248, 241 frames, 24 FPS, seed 123, swimmer prompt from inference/prompts/examples.json, and the PAN2 scheduler. BF16 could not fit on the 24 GiB RTX 5090. A real one-step NVFP4 run completed with 241 frames, 229.54 s end-to-end latency, 14.36 GiB peak allocated / 20.71 GiB reserved, and 1.05 generated FPS (0.044x realtime). The one-step output is attached below; it is a capacity/path validation, not a quality claim. A 50-step run was started and measured at roughly 83-92 s per denoise step, so the single-card block-offload profile is not realtime.

Official PAN2 NVFP4 RTX 5090 metrics

Download the official-config one-step NVFP4 MP4

Model feasibility

MiniMax-H3-Image and LightX2V H3 were evaluated before adding a misleading integration: both use the same 33.1B H3-Image DiT. Official H3-Image lists 66.28 GB BF16 DiT weights, a 66.7 GB BF16 Qwen3-VL text encoder, and a 2.34 GB VAE. NVFP4 can reduce the DiT to roughly 17-19 GB, but the complete pipeline still cannot reside in a 24 GiB RTX 5090. LightX2V's official single-GPU profile therefore enables model-level CPU offload; its four-GPU profiles disable offload.

Generated reports and media are PR attachments only; no model output is committed to the repository.

@Uxito-Ada
Uxito-Ada marked this pull request as draft November 18, 2026 06:08
@Uxito-Ada Uxito-Ada closed this Nov 18, 2026
@Uxito-Ada
Uxito-Ada deleted the feat/minimax-h3-image-nvfp4 branch November 18, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant