vfontech commited on
Commit
0d42fed
·
verified ·
1 Parent(s): 20fd48f

Push model using huggingface_hub.

Browse files
Files changed (2) hide show
  1. README.md +7 -73
  2. config.json +1 -1
README.md CHANGED
@@ -1,77 +1,11 @@
1
  ---
 
2
  tags:
3
- - video-frame-interpolation
4
- - diffusion-model
5
- - animation
6
- - uncertainty-estimation
7
- license: mit
8
- language:
9
- - en
10
- pipeline_tag: image-to-image
11
  ---
12
 
13
- # 🤖 Multi‑Input ResShift Diffusion VFI
14
-
15
- <div align="left" style="display: flex; flex-direction: row; gap: 15px">
16
- <a href='https://arxiv.org/pdf/2504.05402'><img src='https://img.shields.io/badge/arXiv-2405.17933-b31b1b.svg'></a>
17
- <a href='https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI'><img src='https://img.shields.io/badge/Repo-Code-blue'></a>
18
- <a href='https://colab.research.google.com/drive/1MGYycbNMW6Mxu5MUqw_RW_xxiVeHK5Aa#scrollTo=EKaYCioiP3tQ'><img src='https://img.shields.io/badge/Colab-Demo-Green'></a>
19
- </div>
20
-
21
- ## ⚙️ Setup
22
-
23
- Start by downloading the source code directly from GitHub.
24
-
25
- ```bash
26
- git clone https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI.git
27
- ```
28
-
29
- Create a conda environment and install all the requirements
30
-
31
- ```bash
32
- conda create -n multi-input-resshift python=3.10
33
- conda activate multi-input-resshift
34
- pip install -r requirements.txt
35
- ```
36
-
37
- **Note**: Make sure your system is compatible with **CUDA 12.4**. If not, install [CuPy](https://docs.cupy.dev/en/stable/install.html) according to your current CUDA version.
38
-
39
- ## 🚀 Inference Example
40
-
41
- ```python
42
- import os
43
- from PIL import Image
44
- import numpy as np
45
- import matplotlib.pyplot as plt
46
-
47
- from torchvision.transforms import Compose, ToTensor, Resize, Normalize
48
- from utils.utils import denorm
49
- from model.hub import MultiInputResShiftHub
50
-
51
- model = MultiInputResShiftHub.from_pretrained("vfontech/Multiple-Input-Resshift-VFI")
52
- model.requires_grad_(False).cuda().eval()
53
-
54
- img0_path = r"_data\example_images\frame1.png"
55
- img2_path = r"_data\example_images\frame3.png"
56
-
57
- transforms = Compose([
58
- Resize((256, 448)),
59
- ToTensor(),
60
- Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
61
- ])
62
-
63
- img0 = transforms(Image.open(img0_path).convert("RGB")).unsqueeze(0).cuda()
64
- img2 = transforms(Image.open(img2_path).convert("RGB")).unsqueeze(0).cuda()
65
- tau = 0.5
66
-
67
- img1 = model.reverse_process([img0, img2], tau)
68
-
69
- plt.figure(figsize=(10, 5))
70
- plt.subplot(1, 3, 1)
71
- plt.imshow(denorm(img0, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
72
- plt.subplot(1, 3, 2)
73
- plt.imshow(denorm(img1, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
74
- plt.subplot(1, 3, 3)
75
- plt.imshow(denorm(img2, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]).squeeze().permute(1, 2, 0).cpu().numpy())
76
- plt.show()
77
- ```
 
1
  ---
2
+ language: en
3
  tags:
4
+ - model_hub_mixin
5
+ - pytorch_model_hub_mixin
 
 
 
 
 
 
6
  ---
7
 
8
+ This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
+ - Code: https://github.com/VicFonch/Multi-Input-Resshift-Diffusion-VFI
10
+ - Paper: https://arxiv.org/pdf/2504.05402
11
+ - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
  "etas_end": 0.99,
3
  "flow_kwargs": {
4
- "pretrained_path": "_pretrain_models/anime_interp_full.ckpt"
5
  },
6
  "flow_model": "raft",
7
  "kappa": 2.0,
 
1
  {
2
  "etas_end": 0.99,
3
  "flow_kwargs": {
4
+ "pretrained_path": null
5
  },
6
  "flow_model": "raft",
7
  "kappa": 2.0,