File size: 3,406 Bytes
dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b d662d6d dbb180b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
---
base_model:
- black-forest-labs/FLUX.1-dev
base_model_relation: quantized
pipeline_tag: text-to-image
tags:
- dfloat11
- df11
- lossless compression
- 70% size, 100% accuracy
---
## DFloat11 Compressed Model: `black-forest-labs/FLUX.1-dev`
This is a **losslessly compressed** version of [`black-forest-labs/FLUX.1-dev`](https://huggingface.co/black-forest-labs/FLUX.1-dev) using our custom **DFloat11** format. The outputs of this compressed model are **bit-for-bit identical** to the original BFloat16 model, while reducing GPU memory consumption by approximately **30%**.
### 🔍 How It Works
DFloat11 compresses model weights using **Huffman coding** of BFloat16 exponent bits, combined with **hardware-aware algorithmic designs** that enable efficient on-the-fly decompression directly on the GPU. During inference, the weights remain compressed in GPU memory and are **decompressed just before matrix multiplications**, then **immediately discarded after use** to minimize memory footprint.
Key benefits:
* **No CPU decompression or host-device data transfer** --- all operations are handled entirely on the GPU.
* DFloat11 is **much faster than CPU-offloading approaches**, enabling practical deployment in memory-constrained environments.
* The compression is **fully lossless**, guaranteeing that the model’s outputs are **bit-for-bit identical** to those of the original model.
### 🔧 How to Use
1. Install or upgrade the DFloat11 package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
```bash
pip install dfloat11[cuda12]
# or if you have CUDA version 11:
# pip install dfloat11[cuda11]
```
2. Install or upgrade the diffusers package:
```bash
pip install -U diffusers
```
3. Save the following code as a Python file `flux1.py`:
```python
import torch
from diffusers import FluxPipeline, FluxTransformer2DModel
from dfloat11 import DFloat11Model
from transformers.modeling_utils import no_init_weights
with no_init_weights():
transformer = FluxTransformer2DModel.from_config(
FluxTransformer2DModel.load_config(
"black-forest-labs/FLUX.1-dev", subfolder="transformer"
)
).to(torch.bfloat16)
pipe = FluxPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
transformer=transformer,
torch_dtype=torch.bfloat16
)
DFloat11Model.from_pretrained(
'DFloat11/FLUX.1-dev-DF11',
device='cpu',
bfloat16_model=pipe.transformer,
)
pipe.enable_model_cpu_offload()
prompt = "A scenic landscape with mountains, a river, and a clear sky."
image = pipe(
prompt,
width=1024,
height=1024,
guidance_scale=3.5,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator(device="cuda").manual_seed(0)
).images[0]
image.save("image.png")
```
4. Run `python flux1.py` in your terminal.
### 📄 Learn More
* **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
* **GitHub**: [https://github.com/LeanModels/DFloat11](https://github.com/LeanModels/DFloat11)
* **HuggingFace**: [https://huggingface.co/DFloat11](https://huggingface.co/DFloat11) |