VAST-AI/EdenLive-5B-Diffusers

EdenLive 5B — Diffusers

EdenLive generates video from an RGB proxy video, a reference image, and a text prompt. The reference image provides appearance, while the proxy video provides motion and scene structure.

This repository contains the 5B bidirectional checkpoint in Diffusers format.

Project · Code · Original checkpoints

Installation

Install a CUDA-compatible version of PyTorch and torchvision, then install Diffusers from the official repository and the required dependencies:

pip install --upgrade "git+https://github.com/huggingface/diffusers.git"
pip install "transformers>=4.56.0" accelerate sentencepiece ftfy \
  av imageio imageio-ffmpeg

Quick start

Prepare a reference image and an RGB proxy video sampled at 24 fps. The example uses the first 49 frames of the proxy video.

Replace the input paths and prompt with your own. Resample your video before loading it if its frame rate differs from 24 fps.

import torch
from diffusers import EdenLivePipeline
from diffusers.utils import export_to_video, load_image, load_video

pipe = EdenLivePipeline.from_pretrained(
    "VAST-AI/EdenLive-5B-Diffusers",
    torch_dtype={
        "default": torch.bfloat16,
        "proxy_encoder": torch.float32,
    },
)
pipe.enable_model_cpu_offload()

frames = pipe(
    image=load_image("reference.png"),
    video=load_video("proxy-24fps.mp4"),
    prompt="A character explores a snowy landscape.",
    height=704,
    width=1280,
    num_frames=49,
    num_inference_steps=48,
    guidance_scale=3.0,
    generator=torch.Generator(device="cpu").manual_seed(42),
).frames[0]

export_to_video(frames, "edenlive-5b.mp4", fps=24)

Usage notes

• The pipeline requires a reference image and a proxy video in addition to the text prompt.
• The output frame count must satisfy $N = 4k + 1$ and must not exceed the number of input video frames. Here, $k$ is a nonnegative integer.
• Custom output height and width must be multiples of 32.
• Keep the proxy encoder in FP32 when loading the other components in BF16. The transformer preserves its internal proxy projection in FP32.
• The example uses CPU offload to reduce GPU memory usage. If the model fits in GPU memory, replace pipe.enable_model_cpu_offload() with pipe.to("cuda").
• The standard Wan VAE is used by default.

License

EdenLive components use Apache 2.0. DINOv3 components are covered by the DINOv3 License.

Citation

@misc{edenlive2026,
  title  = {EdenLive: Real-Time Generative Rendering from Any Visual Reference},
  author = {Zehuan Huang and Tianxing Xu and Chunli Peng and Cheng Zhang and Zeyi Zheng and Yifei Song and Zekai Gu and Zhipeng Yu and Peng Wang and Yanpei Cao},
  year   = {2026},
  url    = {https://vast-eden.ai/?p=edenlive}
}