File size: 2,651 Bytes
8e01959 dd52bb4 8e01959 dd52bb4 8e01959 dd52bb4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
language: zh
license: creativeml-openrail-m
tags:
- diffusion
- zh
- Chinese
---
# Midu-Stable-Diffusion-2-Chinese-Style-v0.1
## Brief Introduction
|  |  |  |
| ------------------------------------- | ----------------------------- | ------------------------------- |
|  |  |  |
|  |  |  |
大概是huggingface 社区首个开源的Stable diffusion 2 中文模型。该模型基于stable diffusion V2.1模型，在约500万条的中国风格特挑中文数据上进行微调，数据来源于多个开源数据集如[LAION-5B](https://laion.ai/blog/laion-5b/), [Noah-Wukong](https://wukong-dataset.github.io/wukong-dataset/), [Zero](https://zero.so.com/)和一些网络数据。
Probably the first open sourced Chinese Stable Diffusion 2 model. This model is finetuned based on stable diffusion V2.1 with 5M chinese style filtered data. Dataset is composed of several different chinese open source dataset such as [LAION-5B](https://laion.ai/blog/laion-5b/), [Noah-Wukong](https://wukong-dataset.github.io/wukong-dataset/), [Zero](https://zero.so.com/) and some web data.
## Model Details
### 文本编码器
文本编码器使用冻结参数的[lyua1225/clip-huge-zh-75k-steps-bs4096](https://huggingface.co/lyua1225/clip-huge-zh-75k-steps-bs4096)。
Text encoder is frozen [lyua1225/clip-huge-zh-75k-steps-bs4096](https://huggingface.co/lyua1225/clip-huge-zh-75k-steps-bs4096) .
### Unet
在特挑的500万中文数据集上训练了150K steps，使用指数移动平均值(EMA)做原绘画能力保留，使模型能够在中文风格和原绘画能力之间获得权衡。
Training on 5M chinese style filtered data for 150k steps. Exponential moving average(EMA) is applied to keep the original Stable Diffusion 2 drawing capability and reach a balance between chinese style and original drawing capability.
## Usage
因为使用了customed tokenizer, 所以需要优先加载一下tokenizer
```py
# !pip install git+https://github.com/huggingface/accelerate
import torch
from diffusers import StableDiffusionPipeline
torch.backends.cudnn.benchmark = True
pipe = StableDiffusionPipeline.from_pretrained("IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1", torch_dtype=torch.float16)
pipe.to('cuda')
prompt = '飞流直下三千尺，油画'
image = pipe(prompt, guidance_scale=7.5).images[0]
image.save("飞流.png")
```
|