lightx2v commited on
Commit
d47609b
·
verified ·
1 Parent(s): 7b3211f

Update modelcard: add I2V 4-step lora V1 & T2V 4-step lora V1.1

Browse files
Files changed (1) hide show
  1. README.md +59 -11
README.md CHANGED
@@ -1,6 +1,10 @@
1
  ---
2
  language: en
3
  license: apache-2.0
 
 
 
 
4
  ---
5
 
6
  # Wan2.2-Lightning
@@ -13,9 +17,54 @@ We are excited to release the distilled version of <a href="https://wan.video"><
13
  - **High-quality**: The distilled model delivers visuals on par with the base model in most scenarios, sometimes even better.
14
  - **Complex Motion Generation**: Despite the reduction to just 4 steps, the model retains excellent motion dynamics in the generated scenes.
15
 
 
 
 
 
 
16
 
17
  ## Video Demos
18
- ### Wan2.2-T2V-A14B-NFE4 Demo
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  The videos below can be reproduced using [examples/prompt_list.txt](examples/prompt_list.txt).
21
 
@@ -74,16 +123,12 @@ In some results, the direction of the vehicles may be reversed.
74
  </tr>
75
  </table>
76
 
77
- ## 🔥 Latest News!!
78
-
79
- * Aug 04, 2025: 👋 We have open the [Wan2.2-T2V-A14B-NFE4](https://hf-mirror.com/lightx2v/Wan2.2-Lightning). Enjoy!
80
- - [Kijai's ComfyUI WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) is an implementation of Wan models for ComfyUI. Thanks to its Wan-only focus, it's on the frontline of getting cutting edge optimizations and hot research features.
81
 
82
 
83
  ## 📑 Todo List
84
  - [x] Wan2.2-T2V-A14B-4steps
 
85
  - [ ] Wan2.2-TI2V-5B-4steps
86
- - [ ] Wan2.2-I2V-A14B-4steps
87
 
88
  ## 🚀 Run Wan2.2-Lightning
89
 
@@ -160,7 +205,7 @@ DASH_API_KEY=your_key torchrun --nproc_per_node=8 generate.py --task t2v-A14B -
160
  torchrun --nproc_per_node=8 generate.py --task t2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-T2V-A14B --lora_dir ./Wan2.2-Lightning/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1 --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage" --use_prompt_extend --prompt_extend_method 'local_qwen' --prompt_extend_target_lang 'zh'
161
  ```
162
 
163
- <!--
164
  #### Run Image-to-Video Generation
165
 
166
  This repository supports the `Wan2.2-I2V-A14B` Image-to-Video model and can simultaneously support video generation at 480P and 720P resolutions.
@@ -168,7 +213,7 @@ This repository supports the `Wan2.2-I2V-A14B` Image-to-Video model and can simu
168
 
169
  - Single-GPU inference
170
  ```sh
171
- python generate.py --task i2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-I2V-A14B --offload_model True --convert_model_dtype --image examples/i2v_input.JPG --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
172
  ```
173
 
174
  > This command can run on a GPU with at least 80GB VRAM.
@@ -179,9 +224,10 @@ python generate.py --task i2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-I2V-A14B
179
  - Multi-GPU inference using FSDP + DeepSpeed Ulysses
180
 
181
  ```sh
182
- torchrun --nproc_per_node=8 generate.py --task i2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-I2V-A14B --image examples/i2v_input.JPG --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
183
  ```
184
 
 
185
  - Image-to-Video Generation without prompt
186
 
187
  ```sh
@@ -225,7 +271,8 @@ python generate.py --task ti2v-5B --size 1280*704 --ckpt_dir ./Wan2.2-TI2V-5B --
225
  torchrun --nproc_per_node=8 generate.py --task ti2v-5B --size 1280*704 --ckpt_dir ./Wan2.2-TI2V-5B --dit_fsdp --t5_fsdp --ulysses_size 8 --image examples/i2v_input.JPG --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
226
  ```
227
 
228
- > The process of prompt extension can be referenced [here](#2-using-prompt-extention). -->
 
229
 
230
 
231
 
@@ -239,4 +286,5 @@ We built upon and reused code from the following projects: [Wan2.1](https://gith
239
 
240
  We also adopt the evaluation text prompts from [Movie Gen Bench](https://github.com/facebookresearch/MovieGenBench), which is licensed under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0) License. The original license can be found [here](https://github.com/facebookresearch/MovieGenBench/blob/main/LICENSE).
241
 
242
- The selected prompts are further enhanced using the `Qwen/Qwen2.5-14B-Instruct`model [Qwen](https://huggingface.co/Qwen).
 
 
1
  ---
2
  language: en
3
  license: apache-2.0
4
+ base_model:
5
+ - Wan-AI/Wan2.2-T2V-A14B
6
+ - Wan-AI/Wan2.2-I2V-A14B
7
+ - Wan-AI/Wan2.2-TI2V-5B
8
  ---
9
 
10
  # Wan2.2-Lightning
 
17
  - **High-quality**: The distilled model delivers visuals on par with the base model in most scenarios, sometimes even better.
18
  - **Complex Motion Generation**: Despite the reduction to just 4 steps, the model retains excellent motion dynamics in the generated scenes.
19
 
20
+ ## 🔥 Latest News!!
21
+ * Aug 07, 2025: 👋 We have open the [Wan2.2-I2V-A14B-NFE4-V1](https://hf-mirror.com/lightx2v/Wan2.2-Lightning/tree/main/Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1). A [workflow](https://hf-mirror.com/lightx2v/Wan2.2-Lightning/blob/main/Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1/Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1-forKJ.json) compatible with [Kijai's ComfyUI WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) is inside this link. Enjoy!
22
+ * Aug 07, 2025: 👋 We have open the [Wan2.2-T2V-A14B-NFE4-V1.1](https://hf-mirror.com/lightx2v/Wan2.2-Lightning/tree/main/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1.1). A [workflow](https://hf-mirror.com/lightx2v/Wan2.2-Lightning/blob/main/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1.1/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1.1-forKJ.json) compatible with [Kijai's ComfyUI WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) is inside this link. The generation quality of V1.1 is slightly better than V1. Enjoy!
23
+ * Aug 04, 2025: 👋 We have open the [Wan2.2-T2V-A14B-NFE4-V1](https://hf-mirror.com/lightx2v/Wan2.2-Lightning/tree/main/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1). Enjoy!
24
+ - [Kijai's ComfyUI WanVideoWrapper](https://github.com/kijai/ComfyUI-WanVideoWrapper) is an implementation of Wan models for ComfyUI. Thanks to its Wan-only focus, it's on the frontline of getting cutting edge optimizations and hot research features.
25
 
26
  ## Video Demos
27
+ ### Wan2.2-I2V-A14B-NFE4-V1 Demo
28
+
29
+ The videos below can be reproduced using [examples/i2v_prompt_list.txt](examples/i2v_prompt_list.txt) and [examples/i2v_image_path_list.txt](examples/i2v_image_path_list.txt).
30
+
31
+ <table border="0" style="width: 100%; text-align: left; margin-top: 20px;">
32
+ <tr>
33
+ <td>
34
+ <video src="https://github.com/user-attachments/assets/4f6bb1e0-9e2b-4eb2-8b9f-0678ccd5b4ec" width="100%" controls loop></video>
35
+ </td>
36
+ <td>
37
+ <video src="https://github.com/user-attachments/assets/bb249553-3f52-40b3-88f9-6e3bca1a8358" width="100%" controls loop></video>
38
+ </td>
39
+ <td>
40
+ <video src="https://github.com/user-attachments/assets/17a6d26a-dd63-47ef-9a98-1502f503dfba" width="100%" controls loop></video>
41
+ </td>
42
+ </tr>
43
+ <tr>
44
+ <td>
45
+ <video src="https://github.com/user-attachments/assets/6ccc69cf-e129-456f-8b93-6dc709cb0ede" width="100%" controls loop></video>
46
+ </td>
47
+ <td>
48
+ <video src="https://github.com/user-attachments/assets/6cf9c586-f37a-47ed-ab5b-e106c3877fa8" width="100%" controls loop></video>
49
+ </td>
50
+ <td>
51
+ <video src="https://github.com/user-attachments/assets/27e82fdf-88af-44ac-b987-b48aa3f9f793" width="100%" controls loop></video>
52
+ </td>
53
+ </tr>
54
+ <tr>
55
+ <td>
56
+ <video src="https://github.com/user-attachments/assets/36a76f1d-2b64-4b16-a862-210d0ffd6d55" width="100%" controls loop></video>
57
+ </td>
58
+ <td>
59
+ <video src="https://github.com/user-attachments/assets/4bc36c70-931e-4539-be8c-432d832819d3" width="100%" controls loop></video>
60
+ </td>
61
+ <td>
62
+ <video src="https://github.com/user-attachments/assets/488b9179-741b-4b9d-8f23-895981f054cb" width="100%" controls loop></video>
63
+ </td>
64
+ </tr>
65
+ </table>
66
+
67
+ ### Wan2.2-T2V-A14B-NFE4-V1 Demo
68
 
69
  The videos below can be reproduced using [examples/prompt_list.txt](examples/prompt_list.txt).
70
 
 
123
  </tr>
124
  </table>
125
 
 
 
 
 
126
 
127
 
128
  ## 📑 Todo List
129
  - [x] Wan2.2-T2V-A14B-4steps
130
+ - [x] Wan2.2-I2V-A14B-4steps
131
  - [ ] Wan2.2-TI2V-5B-4steps
 
132
 
133
  ## 🚀 Run Wan2.2-Lightning
134
 
 
205
  torchrun --nproc_per_node=8 generate.py --task t2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-T2V-A14B --lora_dir ./Wan2.2-Lightning/Wan2.2-T2V-A14B-4steps-lora-rank64-Seko-V1 --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage" --use_prompt_extend --prompt_extend_method 'local_qwen' --prompt_extend_target_lang 'zh'
206
  ```
207
 
208
+
209
  #### Run Image-to-Video Generation
210
 
211
  This repository supports the `Wan2.2-I2V-A14B` Image-to-Video model and can simultaneously support video generation at 480P and 720P resolutions.
 
213
 
214
  - Single-GPU inference
215
  ```sh
216
+ python generate.py --task i2v-A14B --size "1280*720" --ckpt_dir ./Wan2.2-I2V-A14B --lora_dir ./Wan2.2-Lightning/Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1 --offload_model True --base_seed 42 --prompt_file examples/i2v_prompt_list.txt --image_path_file examples/i2v_image_path_list.txt
217
  ```
218
 
219
  > This command can run on a GPU with at least 80GB VRAM.
 
224
  - Multi-GPU inference using FSDP + DeepSpeed Ulysses
225
 
226
  ```sh
227
+ torchrun --nproc_per_node=8 generate.py --task i2v-A14B --size 1280*720 --ckpt_dir ./Wan2.2-I2V-A14B --lora_dir ./Wan2.2-Lightning/Wan2.2-I2V-A14B-4steps-lora-rank64-Seko-V1 --dit_fsdp --t5_fsdp --ulysses_size 8 --base_seed 42 --prompt_file examples/i2v_prompt_list.txt --image_path_file examples/i2v_image_path_list.txt
228
  ```
229
 
230
+ <!--
231
  - Image-to-Video Generation without prompt
232
 
233
  ```sh
 
271
  torchrun --nproc_per_node=8 generate.py --task ti2v-5B --size 1280*704 --ckpt_dir ./Wan2.2-TI2V-5B --dit_fsdp --t5_fsdp --ulysses_size 8 --image examples/i2v_input.JPG --prompt "Summer beach vacation style, a white cat wearing sunglasses sits on a surfboard. The fluffy-furred feline gazes directly at the camera with a relaxed expression. Blurred beach scenery forms the background featuring crystal-clear waters, distant green hills, and a blue sky dotted with white clouds. The cat assumes a naturally relaxed posture, as if savoring the sea breeze and warm sunlight. A close-up shot highlights the feline's intricate details and the refreshing atmosphere of the seaside."
272
  ```
273
 
274
+ > The process of prompt extension can be referenced [here](#2-using-prompt-extension).
275
+ -->
276
 
277
 
278
 
 
286
 
287
  We also adopt the evaluation text prompts from [Movie Gen Bench](https://github.com/facebookresearch/MovieGenBench), which is licensed under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0) License. The original license can be found [here](https://github.com/facebookresearch/MovieGenBench/blob/main/LICENSE).
288
 
289
+ The selected prompts are further enhanced using the `Qwen/Qwen2.5-14B-Instruct`model [Qwen](https://huggingface.co/Qwen).
290
+