# PAN 2

PAN2 is a foundation world model that simulates how a scene evolves under a time-grounded plan of actions and camera movements. Its 29B generative simulator is a dual-stream multimodal diffusion transformer, integrated here; the [reasoning backbone](https://huggingface.co/IFM/PAN2#reasoning-backbone) that writes its prompts is not.

PAN 2 is integrated as [Modular Diffusers](../../modular_diffusers/overview) blocks only. [PAN2Blocks](/docs/diffusers/main/en/api/pipelines/pan2#diffusers.PAN2Blocks) supports two workflows:

| Workflow | Inputs |
|---|---|
| `t2v` | `prompt` |
| `i2v` | `prompt`, `image` |

```py
import torch
from diffusers import ModularPipeline
from diffusers.utils import export_to_video, load_image

pipe = ModularPipeline.from_pretrained("IFM/PAN2")
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")

prompt = "A red fox trots through fresh snow in a pine forest at dawn."

video = pipe(prompt=prompt, generator=torch.Generator().manual_seed(0), output="videos")[0]
export_to_video(video, "pan2_t2v.mp4", fps=24)

image = load_image("path/to/first_frame.png")
video = pipe(prompt=prompt, image=image, generator=torch.Generator().manual_seed(0), output="videos")[0]
export_to_video(video, "pan2_i2v.mp4", fps=24)
```

Classifier-free guidance is configured on the guider (`guidance_scale=2.5` by default):

```py
pipe.update_components(guider=pipe.guider.new(guidance_scale=5.0))
```

Every call takes a single prompt, which is encoded without padding. The transformer therefore never needs an attention mask, and every [attention backend](../../optimization/attention_backends) is available:

```py
pipe.transformer.set_attention_backend("flash_hub")  # FlashAttention-2
```

## Safety checker

PAN 2 runs guardrails on the prompt before generation and on every generated video after it, and a blocked prompt or video raises a `ValueError`. The checks are on by default, and `load_components()` loads the guardrails, which need the [PAN 2 guardrail](https://github.com/ifm-ai/pan2-guardrail) package:

```bash
pip install git+https://github.com/ifm-ai/pan2-guardrail
```

## Context parallelism

[PAN2Transformer3DModel](/docs/diffusers/main/en/api/models/pan2_transformer3d#diffusers.PAN2Transformer3DModel) supports [context parallelism](../../training/distributed_inference#context-parallelism). The text and video token counts are usually not divisible by the number of devices (the number of text tokens depends on the prompt), so use `ulysses_anything` or `ring_anything` for inference. Plain `ulysses_degree` / `ring_degree`, for example for training, requires both token counts to be divisible by the context parallel size.

```py
# torchrun --nproc_per_node=2 pan2_context_parallel.py
import torch
import torch.distributed as dist
from diffusers import ContextParallelConfig, ModularPipeline

dist.init_process_group("nccl")
rank = dist.get_rank()
torch.cuda.set_device(rank)

pipe = ModularPipeline.from_pretrained("IFM/PAN2")
pipe.load_components(dtype=torch.bfloat16)
pipe.to(f"cuda:{rank}")
pipe.transformer.enable_parallelism(config=ContextParallelConfig(ulysses_degree=2, ulysses_anything=True))

video = pipe(prompt="A red fox trots through fresh snow.", generator=torch.Generator().manual_seed(0), output="videos")[0]
dist.destroy_process_group()
```

## PAN2ModularPipeline[[diffusers.PAN2ModularPipeline]]

#### diffusers.PAN2ModularPipeline[[diffusers.PAN2ModularPipeline]]

```python
diffusers.PAN2ModularPipeline(blocks: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = None, pretrained_model_name_or_path: str | os.PathLike | None = None, components_manager: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None, collection: str | None = None, workflow: str | None = None, modular_config_dict: dict[str, typing.Any] | None = None, config_dict: dict[str, typing.Any] | None = None, **kwargs)
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/pan2/modular_pipeline.py#L42)

A ModularPipeline for PAN 2: text-to-video, and image-to-video from a first frame.

```py
import torch
from diffusers import ModularPipeline

pipe = ModularPipeline.from_pretrained("IFM/PAN2")
pipe.load_components(dtype=torch.bfloat16)
```

#### check_safety_checker_loaded[[diffusers.PAN2ModularPipeline.check_safety_checker_loaded]]

```python
check_safety_checker_loaded()
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/pan2/modular_pipeline.py#L103)

Raise if the checks are on but the PAN 2 guardrail is not loaded.

#### disable_safety_checker[[diffusers.PAN2ModularPipeline.disable_safety_checker]]

```python
disable_safety_checker()
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/pan2/modular_pipeline.py#L118)

Turn off the prompt and video checks, at your own risk.

#### enable_safety_checker[[diffusers.PAN2ModularPipeline.enable_safety_checker]]

```python
enable_safety_checker()
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/pan2/modular_pipeline.py#L97)

Check the prompt and the generated videos with the PAN 2 guardrail.

## PAN2Blocks[[diffusers.PAN2Blocks]]

#### diffusers.PAN2Blocks[[diffusers.PAN2Blocks]]

```python
diffusers.PAN2Blocks()
```

[Source](https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/pan2/modular_blocks_pan2.py#L251)

Modular pipeline blocks for text-to-video and image-to-video generation with PAN 2.

Supported workflows:
- `t2v`: requires `prompt`
- `i2v`: requires `prompt`, `image`

Components:
text_encoder (`Qwen3_5TextModel`) tokenizer (`Qwen2Tokenizer`) safety_checker (`PAN2SafetyChecker`) vae
(`AutoencoderKLWan`) video_processor (`VideoProcessor`) scheduler (`FlowMatchEulerDiscreteScheduler`)
transformer (`PAN2Transformer3DModel`) guider (`ClassifierFreeGuidance`)

Configs:
enable_safety_checker (default: True)

Inputs:
prompt (`str`):
The prompt to guide generation, a single string.
negative_prompt (`str`, *optional*):
The prompt not to guide generation, a single string. When guidance needs one and none (or an empty one)
is passed, PAN 2's default negative prompt is used, its time ranges set to the clip's duration.
max_sequence_length (`int`, *optional*, defaults to 1536):
Maximum sequence length for prompt encoding.
num_frames (`int`, *optional*, defaults to 313):
Number of video frames to generate, of the form `4 * n + 1`.
fps (`float`, *optional*, defaults to 24.0):
The frame rate of the video, which sets its duration in the default negative prompt.
image (`Image`, *optional*):
The first frame, a single image.
height (`int`, *optional*):
The height in pixels of the generated video, 704 by default.
width (`int`, *optional*):
The width in pixels of the generated video, 1248 by default.
num_videos_per_prompt (`int`, *optional*, defaults to 1):
The number of videos to generate per prompt.
image_latents (`Tensor`, *optional*):
The normalized first-frame latents. Can be generated from the vae_encoder step.
num_inference_steps (`int`):
The number of denoising steps.
sigmas (`list`, *optional*):
Custom sigmas for the denoising process.
latents (`Tensor`):
Pre-generated noisy latents to use as inputs for video generation.
generator (`Generator`, *optional*):
Torch generator for deterministic generation.
attention_kwargs (`dict`, *optional*):
Additional kwargs for attention processors.
output_type (`str`, *optional*, defaults to np):
Output format: 'pil', 'np', 'pt'.

Outputs:
videos (`list`):
The generated videos.