github.com/PKU-YuanGroup/MagicTime @main

MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

If you like our project, please give us a star ⭐ on GitHub for the latest update.

[](https://huggingface.co/spaces/BestWishYsh/MagicTime?logs=build) [](https://replicate.com/camenduru/magictime) [](https://colab.research.google.com/github/camenduru/MagicTime-jupyter/blob/main/MagicTime_jupyter.ipynb) [](https://huggingface.co/papers/2404.05014) [](https://arxiv.org/abs/2404.05014) [](https://pku-yuangroup.github.io/MagicTime/) [](https://huggingface.co/datasets/BestWishYsh/ChronoMagic) [](https://twitter.com/_akhaliq/status/1777538468043792473) [](https://twitter.com/vhjf36495872/status/1777525817087553827?s=61&t=r2HzCsU2AnJKbR8yKSprKw) [](https://zenodo.org/doi/10.5281/zenodo.10960665) [](https://github.com/PKU-YuanGroup/MagicTime/blob/main/LICENSE) [](https://github.com/PKU-YuanGroup/MagicTime)

This repository is the official implementation of MagicTime, a metamorphic video generation pipeline based on the given prompts. The main idea is to enhance the capacity of video generation models to accurately depict the real world through our proposed methods and dataset. 💡 We also have other video generation projects that may interest you ✨. > [**Open-Sora Plan: Open-Source Large Video Generation Model**](https://arxiv.org/abs/2412.00131) > Bin Lin, Yunyang Ge and Xinhua Cheng etc. [](https://github.com/PKU-YuanGroup/Open-Sora-Plan) [](https://github.com/PKU-YuanGroup/Open-Sora-Plan) [](https://arxiv.org/abs/2412.00131) > > [**Helios: Real Real-Time Long Video Generation Model**](https://arxiv.org/abs/2603.04379) > Shenghai Yuan, Jinfa Huang and Xianyi He etc. > [](https://github.com/PKU-YuanGroup/Helios) [](https://github.com/PKU-YuanGroup/Helios) [](https://arxiv.org/abs/2603.04379) > > [**OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generation**](https://arxiv.org/abs/2505.20292) > Shenghai Yuan, Xianyi He and Yufan Deng etc. > [](https://github.com/PKU-YuanGroup/OpenS2V-Nexus) [](https://github.com/PKU-YuanGroup/OpenS2V-Nexus) [](https://arxiv.org/abs/2505.20292) > > [**ConsisID: Identity-Preserving Text-to-Video Generation by Frequency Decomposition**](https://arxiv.org/abs/2411.17440) > Shenghai Yuan, Jinfa Huang and Xianyi He etc. > [](https://github.com/PKU-YuanGroup/ConsisID/) [](https://github.com/PKU-YuanGroup/ConsisID/) [](https://arxiv.org/abs/2411.17440) > > [**ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation**](https://arxiv.org/abs/2406.18522) > Shenghai Yuan, Jinfa Huang and Yongqi Xu etc. > [](https://github.com/PKU-YuanGroup/ChronoMagic-Bench/) [](https://github.com/PKU-YuanGroup/ChronoMagic-Bench/) [](https://arxiv.org/abs/2406.18522) > ## 📣 News * ⏳⏳⏳ Training a stronger model with the support of [Open-Sora Plan](https://github.com/PKU-YuanGroup/Open-Sora-Plan). * ⏳⏳⏳ Release the training code of MagicTime. * `[2026.03.08]` 👋 We introduce [Helios](https://github.com/PKU-YuanGroup/Helios), a breakthrough video generation model that achieves minute-scale, high-quality video synthesis at **19.5 FPS on a single H100** GPU — without relying on conventional long video anti-drifting strategies or standard video acceleration techniques. Welcome to check [Technical Report](https://huggingface.co/papers/2603.04379)! * `[2025.04.08]` 🔥 We have updated our technical report. Please click [here](https://arxiv.org/abs/2404.05014) to view it. * `[2025.03.28]` 🔥 MagicTime has been accepted by **TPAMI**, and we will update arXiv with more details soon, keep tuned! * `[2024.07.29]` We add *batch inference* to [inference_magictime.py](https://github.com/PKU-YuanGroup/MagicTime/blob/main/inference_magictime.py) for easier usage. * `[2024.06.27]` Excited to share our latest [ChronoMagic-Bench](https://github.com/PKU-YuanGroup/ChronoMagic-Bench), a benchmark for metamorphic evaluation of text-to-time-lapse video generation, and is fully open source! Please check out the [paper](https://arxiv.org/abs/2406.18522). * `[2024.05.27]` Excited to share our latest Open-Sora Plan v1.1.0, which significantly improves video quality and length, and is fully open source! Please check out the [report](https://github.com/PKU-YuanGroup/Open-Sora-Plan/blob/main/docs/Report-v1.1.0.md). * `[2024.04.14]` Thanks [@camenduru](https://twitter.com/camenduru) and [@ModelsLab](https://modelslab.com/) for providing [Jupyter Notebook](https://github.com/camenduru/MagicTime-jupyter) and [Replicate Demo](https://replicate.com/camenduru/magictime). * `[2024.04.13]` 🔥 We have compressed the size of repo with less than 1.0 MB, so that everyone can clone easier and faster. You can click [here](https://github.com/PKU-YuanGroup/MagicTime/archive/refs/heads/main.zip) to download, or use `git clone --depth=1` command to obtain this repo. * `[2024.04.12]` Thanks [@Kijai](https://github.com/kijai) and [@Baobao Wang](https://www.bilibili.com/video/BV1wx421U7Gn/?spm_id_from=333.1007.top_right_bar_window_history.content.click) for providing ComfyUI Extension [ComfyUI-MagicTimeWrapper](https://github.com/kijai/ComfyUI-MagicTimeWrapper). If you find related work, please let us know. * `[2024.04.11]` 🔥 We release the Hugging Face Space of MagicTime, you can click [here](https://huggingface.co/spaces/BestWishYsh/MagicTime?logs=build) to have a try. * `[2024.04.10]` 🔥 We release the inference code and model weight of MagicTime. * `[2024.04.09]` 🔥 We release the arXiv paper for MagicTime, and you can click [here](https://arxiv.org/abs/2404.05014) to see more details. * `[2024.04.08]` 🔥 We release the subset of ChronoMagic dataset used to train MagicTime. The dataset includes 2,265 metamorphic video-text pairs and can be downloaded at [HuggingFace Dataset](https://huggingface.co/datasets/BestWishYsh/ChronoMagic) or [Google Drive](https://drive.google.com/drive/folders/1WsomdkmSp3ql3ImcNsmzFuSQ9Qukuyr8?usp=sharing). * `[2024.04.08]` 🔥 **All codes & datasets** are coming soon! Stay tuned 👀! ## 😮 Highlights MagicTime shows excellent performance in **metamorphic video generation**. ### Related Resources * [ChronoMagic](https://huggingface.co/datasets/BestWishYsh/ChronoMagic): including 2265 time-lapse video-text pairs. (captioned by GPT-4V) * [ChronoMagic-Bench](https://huggingface.co/datasets/BestWishYsh/ChronoMagic-Bench/tree/main): including 1649 time-lapse video-text pairs. (captioned by GPT-4o) * [ChronoMagic-Bench-150](https://huggingface.co/datasets/BestWishYsh/ChronoMagic-Bench/tree/main): including 150 time-lapse video-text pairs. (captioned by GPT-4o) * [ChronoMagic-Pro](https://huggingface.co/datasets/BestWishYsh/ChronoMagic-Pro): including 460K time-lapse video-text pairs. (captioned by ShareGPT4Video) * [ChronoMagic-ProH](https://huggingface.co/datasets/BestWishYsh/ChronoMagic-ProH): including 150K time-lapse video-text pairs. (captioned by ShareGPT4Video) ### Metamorphic Videos vs. General Videos Compared to general videos, metamorphic videos contain physical knowledge, long persistence, and strong variation, making them difficult to generate. We show compressed .gif on github, which loses some quality. The general videos are generated by the [Animatediff](https://github.com/guoyww/AnimateDiff) and **MagicTime**.

Type	"Bean sprouts grow and mature from seeds"	"[...] construction in a Minecraft virtual environment"	"Cupcakes baking in an oven [...]"	"[...] transitioning from a tightly closed bud to a fully bloomed state [...]"
General Videos
Metamorphic Videos

### Gallery We showcase some metamorphic videos generated by **MagicTime**, [MakeLongVideo](https://github.com/xuduo35/MakeLongVideo), [ModelScopeT2V](https://github.com/modelscope), [VideoCrafter](https://github.com/AILab-CVC/VideoCrafter?tab=readme-ov-file), [ZeroScope](https://huggingface.co/cerspense/zeroscope_v2_576w), [LaVie](https://github.com/Vchitect/LaVie), [T2V-Zero](https://github.com/Picsart-AI-Research/Text2Video-Zero), [Latte](https://github.com/Vchitect/Latte) and [Animatediff](https://github.com/guoyww/AnimateDiff) below.

Method	"cherry blossoms transitioning [...]"	"dough balls baking process [...]"	"an ice cube is melting [...]"	"a simple modern house's construction [...]"
MakeLongVideo
ModelScopeT2V
VideoCrafter