Update README.md
Browse files
README.md
CHANGED
|
@@ -22,39 +22,6 @@ These are the control weights trained on [black-forest-labs/FLUX.1-dev](htpss://
|
|
| 22 |
and [TIGER-Lab/OmniEdit-Filtered-1.2M](https://huggingface.co/datasets/TIGER-Lab/OmniEdit-Filtered-1.2M) for image editing. We use the
|
| 23 |
[Flux Control framework](https://blackforestlabs.ai/flux-1-tools/) for fine-tuning.
|
| 24 |
|
| 25 |
-
prompt: Give this the look of a traditional Japanese woodblock print.
|
| 26 |
-

|
| 27 |
-
prompt: Give this the look of a traditional Japanese woodblock print.
|
| 28 |
-

|
| 29 |
-
prompt: Give this the look of a traditional Japanese woodblock print.
|
| 30 |
-

|
| 31 |
-
prompt: Give this the look of a traditional Japanese woodblock print.
|
| 32 |
-

|
| 33 |
-
prompt: transform the setting to a winter scene
|
| 34 |
-

|
| 35 |
-
prompt: transform the setting to a winter scene
|
| 36 |
-

|
| 37 |
-
prompt: transform the setting to a winter scene
|
| 38 |
-

|
| 39 |
-
prompt: transform the setting to a winter scene
|
| 40 |
-

|
| 41 |
-
prompt: Change it to look like it's in the style of an impasto painting.
|
| 42 |
-

|
| 43 |
-
prompt: Change it to look like it's in the style of an impasto painting.
|
| 44 |
-

|
| 45 |
-
prompt: Change it to look like it's in the style of an impasto painting.
|
| 46 |
-

|
| 47 |
-
prompt: Change it to look like it's in the style of an impasto painting.
|
| 48 |
-

|
| 49 |
-
prompt: turn the color of mushroom to gray
|
| 50 |
-

|
| 51 |
-
prompt: turn the color of mushroom to gray
|
| 52 |
-

|
| 53 |
-
prompt: turn the color of mushroom to gray
|
| 54 |
-

|
| 55 |
-
prompt: turn the color of mushroom to gray
|
| 56 |
-

|
| 57 |
-
|
| 58 |
|
| 59 |
## License
|
| 60 |
|
|
@@ -110,15 +77,15 @@ import torch
|
|
| 110 |
|
| 111 |
path = "sayakpaul/FLUX.1-dev-edit-v0" # to change
|
| 112 |
edit_transformer = FluxTransformer2DModel.from_pretrained(path, torch_dtype=torch.bfloat16)
|
| 113 |
-
|
| 114 |
"black-forest-labs/FLUX.1-dev", transformer=edit_transformer, torch_dtype=torch.bfloat16
|
| 115 |
).to("cuda")
|
| 116 |
|
| 117 |
# load the turbo LoRA
|
| 118 |
-
|
| 119 |
hf_hub_download("ByteDance/Hyper-SD", "Hyper-FLUX.1-dev-8steps-lora.safetensors"), adapter_name="hyper-sd"
|
| 120 |
)
|
| 121 |
-
|
| 122 |
|
| 123 |
image = load_image("./assets/mushroom.jpg") # resize as needed.
|
| 124 |
print(image.size)
|
|
@@ -185,30 +152,30 @@ You can also choose to perform quantization if the memory requirements cannot be
|
|
| 185 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/car.jpg" alt="Source Image 1" width="150"><br>
|
| 186 |
<em>Give this the look of a traditional Japanese woodblock print.</em>
|
| 187 |
</td>
|
| 188 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_car.jpg" alt="Edited Image gs 10"
|
| 189 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_car.jpg" alt="Edited Image gs 20"
|
| 190 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_car.jpg" alt="Edited Image gs 30"
|
| 191 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_car.jpg" alt="Edited Image gs 40"
|
| 192 |
</tr>
|
| 193 |
<tr>
|
| 194 |
<td align="center">
|
| 195 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/green_creature" alt="Source Image 2" width="150"><br>
|
| 196 |
<em>transform the setting to a winter scene</em>
|
| 197 |
</td>
|
| 198 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_green_creature.jpg" alt="Edited Image gs 10"
|
| 199 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_green_creature.jpg" alt="Edited Image gs 20"
|
| 200 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_green_creature.jpg" alt="Edited Image gs 30"
|
| 201 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_green_creature.jpg" alt="Edited Image gs 40"
|
| 202 |
</tr>
|
| 203 |
<tr>
|
| 204 |
<td align="center">
|
| 205 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/mushroom.jpg" alt="Source Image 3" width="150"><br>
|
| 206 |
<em>turn the color of mushroom to gray</em>
|
| 207 |
</td>
|
| 208 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_mushroom.jpg" alt="Edited Image gs 10"
|
| 209 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_mushroom.jpg" alt="Edited Image gs 20"
|
| 210 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_mushroom.jpg" alt="Edited Image gs 30"
|
| 211 |
-
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_mushroom.jpg" alt="Edited Image gs 40"
|
| 212 |
</tr>
|
| 213 |
</table>
|
| 214 |
|
|
|
|
| 22 |
and [TIGER-Lab/OmniEdit-Filtered-1.2M](https://huggingface.co/datasets/TIGER-Lab/OmniEdit-Filtered-1.2M) for image editing. We use the
|
| 23 |
[Flux Control framework](https://blackforestlabs.ai/flux-1-tools/) for fine-tuning.
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
## License
|
| 27 |
|
|
|
|
| 77 |
|
| 78 |
path = "sayakpaul/FLUX.1-dev-edit-v0" # to change
|
| 79 |
edit_transformer = FluxTransformer2DModel.from_pretrained(path, torch_dtype=torch.bfloat16)
|
| 80 |
+
pipeline = FluxControlPipeline.from_pretrained(
|
| 81 |
"black-forest-labs/FLUX.1-dev", transformer=edit_transformer, torch_dtype=torch.bfloat16
|
| 82 |
).to("cuda")
|
| 83 |
|
| 84 |
# load the turbo LoRA
|
| 85 |
+
pipeline.load_lora_weights(
|
| 86 |
hf_hub_download("ByteDance/Hyper-SD", "Hyper-FLUX.1-dev-8steps-lora.safetensors"), adapter_name="hyper-sd"
|
| 87 |
)
|
| 88 |
+
pipeline.set_adapters(["hyper-sd"], adapter_weights=[0.125])
|
| 89 |
|
| 90 |
image = load_image("./assets/mushroom.jpg") # resize as needed.
|
| 91 |
print(image.size)
|
|
|
|
| 152 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/car.jpg" alt="Source Image 1" width="150"><br>
|
| 153 |
<em>Give this the look of a traditional Japanese woodblock print.</em>
|
| 154 |
</td>
|
| 155 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_car.jpg" alt="Edited Image gs 10"></td>
|
| 156 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_car.jpg" alt="Edited Image gs 20"></td>
|
| 157 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_car.jpg" alt="Edited Image gs 30"></td>
|
| 158 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_car.jpg" alt="Edited Image gs 40"></td>
|
| 159 |
</tr>
|
| 160 |
<tr>
|
| 161 |
<td align="center">
|
| 162 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/green_creature" alt="Source Image 2" width="150"><br>
|
| 163 |
<em>transform the setting to a winter scene</em>
|
| 164 |
</td>
|
| 165 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_green_creature.jpg" alt="Edited Image gs 10"></td>
|
| 166 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_green_creature.jpg" alt="Edited Image gs 20"></td>
|
| 167 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_green_creature.jpg" alt="Edited Image gs 30"></td>
|
| 168 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_green_creature.jpg" alt="Edited Image gs 40"></td>
|
| 169 |
</tr>
|
| 170 |
<tr>
|
| 171 |
<td align="center">
|
| 172 |
<img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/assets/mushroom.jpg" alt="Source Image 3" width="150"><br>
|
| 173 |
<em>turn the color of mushroom to gray</em>
|
| 174 |
</td>
|
| 175 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-10_mushroom.jpg" alt="Edited Image gs 10"></td>
|
| 176 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-20_mushroom.jpg" alt="Edited Image gs 20"></td>
|
| 177 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-30_mushroom.jpg" alt="Edited Image gs 30"></td>
|
| 178 |
+
<td align="center"><img src="https://huggingface.co/datasets/sayakpaul/sample-datasets/resolve/main/flux-edit-artifacts/edited_gs-40_mushroom.jpg" alt="Edited Image gs 40"></td>
|
| 179 |
</tr>
|
| 180 |
</table>
|
| 181 |
|