VicFonch commited on
Commit
5f16114
·
unverified ·
1 Parent(s): 8a5492d

README.md: Update

Browse files
Files changed (1) hide show
  1. README.md +66 -7
README.md CHANGED
@@ -1,11 +1,70 @@
1
  ---
2
- language: en
3
  tags:
4
- - model_hub_mixin
5
- - pytorch_model_hub_mixin
 
 
6
  ---
7
 
8
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
- - Code: https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI
10
- - Paper: https://arxiv.org/pdf/2504.05402
11
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  tags:
3
+ - video-frame-interpolation
4
+ - diffusion-model
5
+ - animation
6
+ - uncertainty-estimation
7
  ---
8
 
9
+ # 🤖 Multi‑Input ResShift Diffusion VFI
10
+
11
+ <div align="left">
12
+ <a href='https://arxiv.org/pdf/2504.05402'><img src='https://img.shields.io/badge/arXiv-2405.17933-b31b1b.svg'></a>
13
+ <a href='https://doubiiu.github.io/projects/ToonCrafter/'><img src='https://img.shields.io/badge/Repo-Code-blue'></a> &nbsp;
14
+ <a href='https://colab.research.google.com/drive/1MGYycbNMW6Mxu5MUqw_RW_xxiVeHK5Aa#scrollTo=EKaYCioiP3tQ'><img src='https://img.shields.io/badge/Colab-Demo-Green'></a>
15
+ </div>
16
+
17
+ ## ⚙️ Setup
18
+
19
+ Start by downloading the source code directly from GitHub.
20
+
21
+ ```bash
22
+ git clone https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI.git
23
+ ```
24
+
25
+ Create a conda environment and install all the requirements
26
+
27
+ ```bash
28
+ conda create -n multi-input-resshift python=3.10
29
+ conda activate multi-input-resshift
30
+ pip install -r requirements.txt
31
+ ```
32
+
33
+ **Note**: Make sure your system is compatible with **CUDA 12.4**. If not, install [CuPy](https://docs.cupy.dev/en/stable/install.html) according to your current CUDA version.
34
+
35
+ ## 🚀 Inference Example
36
+
37
+ ```python
38
+ import os
39
+ from PIL import Image
40
+ import numpy as np
41
+ import matplotlib.pyplot as plt
42
+
43
+ from torchvision.transforms import Compose, ToTensor, Resize, Normalize
44
+ from utils.utils import denorm
45
+ from model.hub import MultiInputResShiftHub
46
+
47
+ model = MultiInputResShiftHub.from_pretrained("vfontech/Multiple-Input-Resshift-VFI")
48
+ model.requires_grad_(False).cuda().eval()
49
+
50
+ img0_path = r"_data\example_images\frame1.png"
51
+ img2_path = r"_data\example_images\frame3.png"
52
+
53
+ transforms = Compose([
54
+ Resize((256, 448)),
55
+ ToTensor(),
56
+ Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
57
+ ])
58
+
59
+ img0 = transforms(Image.open(img0_path).convert("RGB")).unsqueeze(0).cuda()
60
+ img2 = transforms(Image.open(img2_path).convert("RGB")).unsqueeze(0).cuda()
61
+
62
+ img1 = model.reverse_process([img0, img2], 0.5)
63
+
64
+ plt.figure(figsize=(10, 5))
65
+ plt.subplot(1, 2, 1)
66
+ plt.imshow(denorm(img0, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
67
+ plt.subplot(1, 2, 2)
68
+ plt.imshow(denorm(It, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
69
+ plt.show()
70
+ ```