Added training info
Browse files
README.md
CHANGED
|
@@ -18,9 +18,9 @@ language:
|
|
| 18 |
library_name: diffusers
|
| 19 |
---
|
| 20 |
|
| 21 |
-
#
|
| 22 |
|
| 23 |
-
These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new type of conditioning. You can find some example images in the following.
|
| 24 |
|
| 25 |
**prompt**: contemporary living room of a house
|
| 26 |
|
|
@@ -35,4 +35,34 @@ These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with new
|
|
| 35 |
**prompt**: contemporary living room, high quality, 4k, realistic
|
| 36 |
|
| 37 |
**negative prompt**: low quality, monochrome, low res
|
| 38 |
-

|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
library_name: diffusers
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# ControlNet - mfidabel/controlnet-segment-anything
|
| 22 |
|
| 23 |
+
These are controlnet weights trained on runwayml/stable-diffusion-v1-5 with a new type of conditioning. You can find some example images in the following.
|
| 24 |
|
| 25 |
**prompt**: contemporary living room of a house
|
| 26 |
|
|
|
|
| 35 |
**prompt**: contemporary living room, high quality, 4k, realistic
|
| 36 |
|
| 37 |
**negative prompt**: low quality, monochrome, low res
|
| 38 |
+

|
| 39 |
+
|
| 40 |
+
## Training
|
| 41 |
+
|
| 42 |
+
**Training Data** This model was trained using a Segmented dataset based on the [COYO-700M Dataset](https://huggingface.co/datasets/kakaobrain/coyo-700m).
|
| 43 |
+
[Stable Diffusion v1.5](https://huggingface.co/runwayml/stable-diffusion-v1-5) checkpoint was used as the base model for the controlnet.
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
The model was trained as follows:
|
| 47 |
+
|
| 48 |
+
- 25k steps with the [SAM-COYO-2k](https://huggingface.co/datasets/mfidabel/sam-coyo-2k) dataset
|
| 49 |
+
- 28k steps with the [SAM-COYO-2.5k](https://huggingface.co/datasets/mfidabel/sam-coyo-2.5k) dataset
|
| 50 |
+
- 38k steps with the [SAM-COYO-3k](https://huggingface.co/datasets/mfidabel/sam-coyo-3k) dataset
|
| 51 |
+
|
| 52 |
+
In that particular order.
|
| 53 |
+
|
| 54 |
+
- **Hardware**: Google Cloud TPUv4-8 VM
|
| 55 |
+
|
| 56 |
+
- **Optimizer**: AdamW
|
| 57 |
+
|
| 58 |
+
- **Train Batch Size**: 2 x 4 = 8
|
| 59 |
+
|
| 60 |
+
- **Learning rate**: 0.00001 constant
|
| 61 |
+
|
| 62 |
+
- **Gradient Accumulation Steps**: 1
|
| 63 |
+
|
| 64 |
+
- **Resolution**: 512
|
| 65 |
+
|
| 66 |
+
Learning rate: warmup to 0.0001 for 10,000 steps and then kept constant
|
| 67 |
+
|
| 68 |
+
|