release

README.md

@@ -0,0 +1,99 @@
+---
+license: apache-2.0
+license_link: LICENSE.md
+
+language:
+  - en
+  
+tags:
+- text-to-image
+- image-generation
+- cogview
+  
+inference: false
+---
+
+# CogView3-Plus-3B
+
+<p style="text-align: center;">
+  <div align="center">
+  <img src=https://github.com/THUDM/CogView3/raw/main/resources/logo.svg width="50%"/>
+  </div>
+  <p align="center">
+  <a href="README_zh.md">📄 中文阅读 </a> | 
+  <a href="https://huggingface.co/spaces/THUDM-HF-SPACE/CogView-3-Plus">🤗 Hugging Face Space | </a> 
+  <a href="https://github.com/THUDM/CogView3">🌐 Github </a> | 
+  <a href="https://arxiv.org/pdf/2403.05121">📜 arxiv </a>
+</p>
+<p align="center">
+📍 Visit <a href="https://chatglm.cn/main/gdetail/65a232c082ff90a2ad2f15e2?fr=osm_cogvideox&lang=zh"> Qingyan </a> and <a href="https://open.bigmodel.cn/?utm_campaign=open&_channel_track_key=OWTVNma9"> API Platform</a> to experience larger-scale commercial video generation models.
+</p>
+
+## Inference Requirements and Model Overview
+
+This model is the DiT version of CogView3, a text-to-image generation model, supporting image generation from 512 to 2048px.
+
++ Resolution: Width and height must meet the range from 512px to 2048px and must be divisible by 32.
++ Inference Speed: 1s / step (tested on A100)
++ Precision: BF16 / FP32 (FP16 is not supported, as it leads to overflow causing black images)
+
+## Memory Consumption
+
+We tested memory consumption at several common resolutions on A100 devices, `batchsize=1, BF16`, as shown in the table below:
+
+| 分辨率         | enable_model_cpu_offload OFF | enable_model_cpu_offload ON |
+|-------------|------------------------------|-----------------------------|
+| 512 * 512   | 19GB                         | 11GB                        |
+| 720 * 480   | 20GB                         | 11GB                        |
+| 1024 * 1024 | 23GB                         | 11GB                        |
+| 1280 * 720  | 24GB                         | 11GB                        |
+| 2048 * 2048 | 25GB                         | 11GB                        |
+
+## Quick Start
+
+First, ensure the `diffusers` library is installed from source. Then, run the following code:
+
+```python
+from diffusers import CogView3PlusPipeline
+import torch
+
+pipe = CogView3PlusPipeline.from_pretrained("THUDM/CogView3-Plus-3B", torch_dtype=torch.float16).to("cuda")
+
+# Enable it to reduce GPU memory usage
+pipe.enable_model_cpu_offload()
+pipe.vae.enable_slicing()
+pipe.vae.enable_tiling()
+
+prompt = "A vibrant cherry red sports car sits proudly under the gleaming sun, its polished exterior smooth and flawless, casting a mirror-like reflection. The car features a low, aerodynamic body, angular headlights that gaze forward like predatory eyes, and a set of black, high-gloss racing rims that contrast starkly with the red. A subtle hint of chrome embellishes the grille and exhaust, while the tinted windows suggest a luxurious and private interior. The scene conveys a sense of speed and elegance, the car appearing as if it's about to burst into a sprint along a coastal road, with the ocean's azure waves crashing in the background."
+image = pipe(
+    prompt=prompt,
+    guidance_scale=7.0,
+    num_images_per_prompt=1,
+    num_inference_steps=50,
+    width=1024,
+    height=1024,
+).images[0]
+
+image.save("cogview3.png")
+```
+
+For more content and to download the original SAT weights, please visit our [GitHub](https://github.com/THUDM/CogView3).
+
+## Citation
+
+🌟 If you find our work helpful, feel free to cite our paper and leave a star:
+
+```
+@article{zheng2024cogview3,
+  title={Cogview3: Finer and faster text-to-image generation via relay diffusion},
+  author={Zheng, Wendi and Teng, Jiayan and Yang, Zhuoyi and Wang, Weihan and Chen, Jidong and Gu, Xiaotao and Dong, Yuxiao and Ding, Ming and Tang, Jie},
+  journal={arXiv preprint arXiv:2403.05121},
+  year={2024}
+}
+```
+
+We welcome your contributions, and you can click [here](resources/contribute_zh.md) for more information.
+
+## Model License
+
+This Model is released under the [Apache 2.0 License](LICENSE).

README_zh.md

@@ -0,0 +1,84 @@
+# CogView3-Plus-3B
+
+<p style="text-align: center;">
+  <div align="center">
+  <img src=https://github.com/THUDM/CogView3/raw/main/resources/logo.svg width="50%"/>
+  </div>
+  <p align="center">
+  <a href="README.md">📄 Read in English</a> | 
+  <a href="https://huggingface.co/spaces/THUDM-HF-SPACE/CogView-3-Plus">🤗 Hugging Face Space | </a> 
+  <a href="https://github.com/THUDM/CogView3">🌐 Github </a> | 
+  <a href="https://arxiv.org/pdf/2403.05121">📜 arxiv </a>
+</p>
+<p align="center">
+📍 前往<a href="https://chatglm.cn/main/gdetail/65a232c082ff90a2ad2f15e2?fr=osm_cogvideox&lang=zh"> 清言 </a> 和 <a href="https://open.bigmodel.cn/?utm_campaign=open&_channel_track_key=OWTVNma9"> API平台</a> 体验更大规模的商业版视频生成模型。
+</p>
+
+## 推理要求和模型介绍
+
+该模型是 CogView3 的 DiT 版本图像生成模型，支持从 512 到 2048 范围内的图像生成。
+
++ 分辨率: 长宽均需满足 512px - 2048px 之间，均需被32整除。
++ 推理速度: 1s / step (在 A100 进行测试)
++ 精度: BF16 / FP32 (不支持FP16，会出现溢出导致纯黑图片)
+
+## 显存消耗
+
+我们在A100设备上对几个常见分辨率的显存消耗进行了测试，`batchsize=1, BF16`, 如下表所示：
+
+| 分辨率         | enable_model_cpu_offload OFF | enable_model_cpu_offload ON |
+|-------------|------------------------------|-----------------------------|
+| 512 * 512   | 19GB                         | 11GB                        |
+| 720 * 480   | 20GB                         | 11GB                        |
+| 1024 * 1024 | 23GB                         | 11GB                        |
+| 1280 * 720  | 24GB                         | 11GB                        |
+| 2048 * 2048 | 25GB                         | 11GB                        |
+
+## 快速开始
+
+首先，确保从源代码安装`diffusers`库。接着，运行以下代码：
+
+```python
+from diffusers import CogView3PlusPipeline
+import torch
+
+pipe = CogView3PlusPipeline.from_pretrained("THUDM/CogView3-Plus-3B", torch_dtype=torch.float16).to("cuda")
+
+# Open it for reduce GPU memory usage
+pipe.enable_model_cpu_offload()
+pipe.vae.enable_slicing()
+pipe.vae.enable_tiling()
+
+prompt = "A vibrant cherry red sports car sits proudly under the gleaming sun, its polished exterior smooth and flawless, casting a mirror-like reflection. The car features a low, aerodynamic body, angular headlights that gaze forward like predatory eyes, and a set of black, high-gloss racing rims that contrast starkly with the red. A subtle hint of chrome embellishes the grille and exhaust, while the tinted windows suggest a luxurious and private interior. The scene conveys a sense of speed and elegance, the car appearing as if it's about to burst into a sprint along a coastal road, with the ocean's azure waves crashing in the background."
+image = pipe(
+    prompt=prompt,
+    guidance_scale=7.0,
+    num_images_per_prompt=1,
+    num_inference_steps=50,
+    width=1024,
+    height=1024,
+).images[0]
+
+image.save("cogview3.png")
+```
+
+更多内容以及下载 SAT 原始权重，请前往我们的 [github](https://github.com/THUDM/CogView3)。
+
+## 引用
+
+🌟 如果您发现我们的工作有所帮助，欢迎引用我们的文章，留下宝贵的stars
+
+```
+@article{zheng2024cogview3,
+  title={Cogview3: Finer and faster text-to-image generation via relay diffusion},
+  author={Zheng, Wendi and Teng, Jiayan and Yang, Zhuoyi and Wang, Weihan and Chen, Jidong and Gu, Xiaotao and Dong, Yuxiao and Ding, Ming and Tang, Jie},
+  journal={arXiv preprint arXiv:2403.05121},
+  year={2024}
+}
+```
+
+我们欢迎您的贡献，您可以点击[这里](resources/contribute_zh.md)查看更多信息。
+
+## 模型协议
+
+该模型基于 [Apache 2.0 License](LICENSE) 协议发布。