LeanQuant commited on
Commit
d662d6d
·
verified ·
1 Parent(s): dbb180b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -12
README.md CHANGED
@@ -26,7 +26,7 @@ Key benefits:
26
 
27
  ### 🔧 How to Use
28
 
29
- 1. Install the DFloat11 pip package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
30
 
31
  ```bash
32
  pip install dfloat11[cuda12]
@@ -34,31 +34,56 @@ Key benefits:
34
  # pip install dfloat11[cuda11]
35
  ```
36
 
37
- 2. To use the DFloat11 model, run the following example code in Python:
 
 
 
 
 
 
38
  ```python
39
  import torch
40
- from diffusers import FluxPipeline
41
  from dfloat11 import DFloat11Model
42
-
43
- pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  pipe.enable_model_cpu_offload()
45
-
46
- DFloat11Model.from_pretrained('DFloat11/FLUX.1-dev-DF11', device='cpu', bfloat16_model=pipe.transformer)
47
-
48
- prompt = "A futuristic cityscape at sunset, with flying cars, neon lights, and reflective water canals"
49
  image = pipe(
50
  prompt,
51
- width=1920,
52
- height=1440,
53
  guidance_scale=3.5,
54
  num_inference_steps=50,
55
  max_sequence_length=512,
56
  generator=torch.Generator(device="cuda").manual_seed(0)
57
  ).images[0]
58
-
59
  image.save("image.png")
60
  ```
61
 
 
 
62
  ### 📄 Learn More
63
 
64
  * **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)
 
26
 
27
  ### 🔧 How to Use
28
 
29
+ 1. Install or upgrade the DFloat11 package *(installs the CUDA kernel automatically; requires a CUDA-compatible GPU and PyTorch installed)*:
30
 
31
  ```bash
32
  pip install dfloat11[cuda12]
 
34
  # pip install dfloat11[cuda11]
35
  ```
36
 
37
+ 2. Install or upgrade the diffusers package:
38
+
39
+ ```bash
40
+ pip install -U diffusers
41
+ ```
42
+
43
+ 3. Save the following code as a Python file `flux1.py`:
44
  ```python
45
  import torch
46
+ from diffusers import FluxPipeline, FluxTransformer2DModel
47
  from dfloat11 import DFloat11Model
48
+ from transformers.modeling_utils import no_init_weights
49
+
50
+ with no_init_weights():
51
+ transformer = FluxTransformer2DModel.from_config(
52
+ FluxTransformer2DModel.load_config(
53
+ "black-forest-labs/FLUX.1-dev", subfolder="transformer"
54
+ )
55
+ ).to(torch.bfloat16)
56
+
57
+ pipe = FluxPipeline.from_pretrained(
58
+ "black-forest-labs/FLUX.1-dev",
59
+ transformer=transformer,
60
+ torch_dtype=torch.bfloat16
61
+ )
62
+
63
+ DFloat11Model.from_pretrained(
64
+ 'DFloat11/FLUX.1-dev-DF11',
65
+ device='cpu',
66
+ bfloat16_model=pipe.transformer,
67
+ )
68
+
69
  pipe.enable_model_cpu_offload()
70
+
71
+ prompt = "A scenic landscape with mountains, a river, and a clear sky."
 
 
72
  image = pipe(
73
  prompt,
74
+ width=1024,
75
+ height=1024,
76
  guidance_scale=3.5,
77
  num_inference_steps=50,
78
  max_sequence_length=512,
79
  generator=torch.Generator(device="cuda").manual_seed(0)
80
  ).images[0]
81
+
82
  image.save("image.png")
83
  ```
84
 
85
+ 4. Run `python flux1.py` in your terminal.
86
+
87
  ### 📄 Learn More
88
 
89
  * **Paper**: [70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float](https://arxiv.org/abs/2504.11651)