How to use from the
Use from the
Diffusers library
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import export_to_video

# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda")
pipe.load_lora_weights("ModelsLab/MiniMax-H3-Turbo-RL")

prompt = "A man with short gray hair plays a red electric guitar."

output = pipe(prompt=prompt).frames[0]
export_to_video(output, "output.mp4")

MiniMax-H3 Turbo V4 + RL (rl_s3)

One LoRA file for MiniMax-H3 Ref2VA (MiniMaxAI/MiniMax-H3, transformer_ref) that combines:

  1. Turbo LoRA V4 by larryvrh (larryvrh/MiniMax-H3-Turbo-Lora, minimax_h3_turbo_v4_step600_ema.safetensors, Apache-2.0): the 8-step distillation adapter.
  2. rl_s3: a rank-64 LoRA trained by ModelsLab on top of Turbo V4 with closed-loop, pure-ODE online Flow-DPO (best-of-N rollouts scored by in-RAM video/audio critics, paired against frozen V4 on the same seed, high-frequency KL lock to V4).

The two are stored as exact branches side by side, not merged: for every attention/MLP module ฮ”W = [B_v4 | B_rl] ยท [A_v4 ; A_rl] (rank 128), and V4's rank-16 adaln_proj.linear branches are kept as they are. Merging into BF16 base weights would round away most of these small deltas, so load the file as LoRA branches.

File Size Layout
minimax_h3_turbo_v4_rl_s3.safetensors 1.40 GB (BF16) ComfyUI / ai-toolkit keys (blocks.N.attn.qkv_proj.lora_A.weight, ...), same layout as Turbo V4. 208 attention/MLP modules at rank 128 (200 DiT-block modules = V4 โŠ• RL; 8 token-refiner modules = V4 zero-padded) + 51 AdaLN modules at rank 16 (V4 only). 518 BF16 tensors.

How to run it

The recommended sampler is E4a: Turbo V4's 9-point shifted grid (u = 1, .875, โ€ฆ, .125, 0, shift 12 video / 3 audio) with one extra node at u = 0.04 (ฯƒ_video = 0.333) and a 2nd-order Heun corrector on [ฯƒ 0.632 โ†’ 0.333]: 10 sigma points, 10 transformer forwards. Reference implementation for the NVIDIA Sol-H3 engine: heun_tail.py in the ModelsLab H3 server repo; the plain 8-forward Euler schedule also works.

Load it with any loader that accepts Turbo V4's file (same keys, larger rank). Sol-H3 / PEFT loaders read the rank from the tensors.

Evaluation (16 public clips, seed 11, ratios ร— the real clip)

Arm Forwards Detail Flow Motion Made-up speech
Turbo V4, Euler 8 1.43 0.87 0.94 1/7
Turbo V4, E4a 10 1.59 0.92 1.02 1/7
Turbo V4 + rl_s3, E4a 10 1.59 0.92 1.01 1/7

Honest attribution: on this public set almost all of the gain over Turbo V4 comes from the E4a sampler; rl_s3 matches V4 + E4a within noise on these metrics (closer to the real clip than V4 + E4a on Lab b for 11/16 clips and on saturation for 8/16). Measured with a BF16 reference pipeline, not with the MXFP8 production engine.

Samples: before / after

Same prompt, references and seed (11) in each row. Before: Turbo V4, 8-step Euler (the previous production setup). After: this file (Turbo V4 + rl_s3) with the E4a 10-NFE sampler. Rendered at 1344ร—768 with the BF16 reference pipeline (not the MXFP8 production engine). Prompts and reference images come from the public clips credited below; the videos and sound are generated.

Before: Turbo V4, 8 steps After: Turbo V4 + RL, E4a
Flower market (multi-subject, 5 s)
Danza de los Voladores (portrait action, 10 s)
View from Fisherman's Bastion (ambience, 10 s)
Office dialogue (speech, 5 s)

Source clips (used only to write the prompt and pick reference frames):

  • Flower market: Netflix Open Content, source file 96x2160 60fps 10bit 420, CC BY 4.0
  • Danza de los Voladores: Wikimedia Commons, Danza de los Voladores (Dance of the Flyers) by Emma Matthies, CC BY 4.0
  • View from Fisherman's Bastion: Wikimedia Commons, View from Fisherman's Bastion (2) by Elekes Andor, CC BY 4.0
  • Office dialogue: Netflix Open Content, source file 3840x2160 59.94fps SDR, CC BY 4.0

Training data and licences

The RL stages (rl_s0 โ†’ rl_s3) used short clips (only their cached latents and the prompts written for them) from:

Source Licence
DVIDS (dvidshub.net) Public domain (US federal government work); DVIDS API terms
Wikimedia Commons Per-file free licences
Netflix Open Content CC BY 4.0
UltraVideo (APRIL-AIGC/UltraVideo) UltraVideo License (CC BY 4.0 with restrictions, non-commercial research)
Replay (facebookresearch/replay_dataset) CC BY-NC 4.0
Xiph test media: SVT Only for developing, testing and presenting technology standards
Xiph test media: Netflix CC BY-NC-ND 4.0

No media from these datasets is redistributed here. No customer or user data was used for training.

Licence and use

  • The weights are a derivative of MiniMax-H3 and are released under the MiniMax-H3 Community License Agreement, including its territory and use restrictions. Read it before use.
  • The Turbo V4 component keeps its Apache-2.0 licence and credit: larryvrh, larryvrh/MiniMax-H3-Turbo-Lora (revision 43a74557).
  • Several training sources are licensed for non-commercial research only (see the table). Check that your use is compatible with them.
Downloads last month
-
Inference Providers NEW

This task can take several minutes

Model tree for ModelsLab/MiniMax-H3-Turbo-RL

Adapter
(205)
this model