pytorch
/

Qwen3-4B-INT8-INT4

Text Generation

text-generation-inference

Model card Files Files and versions

Qwen3-4B-INT8-INT4

12.5 GB

2 contributors

History: 10 commits

metascroy's picture

Rename qwen3-4b-1024-ctx.pte to qwen3-4b-8da4w-1024-ctx.pte

8ad2519 verified 7 months ago

.gitattributes

1.69 kB

Rename qwen3-4b-1024-ctx.pte to qwen3-4b-8da4w-1024-ctx.pte 7 months ago
README.md

9.48 kB

Update README.md 7 months ago
added_tokens.json

707 Bytes

Upload tokenizer 7 months ago
chat_template.jinja

4.12 kB

Upload tokenizer 7 months ago
config.json

3.02 kB

Upload Qwen3ForCausalLM 7 months ago
generation_config.json

219 Bytes

Upload Qwen3ForCausalLM 7 months ago
merges.txt

1.67 MB

Upload tokenizer 7 months ago
pytorch_model.bin
Detected Pickle imports (17)
- "torch._utils._rebuild_tensor_v2",
- "torch._utils._rebuild_wrapper_subclass",
- "torchao.quantization.linear_activation_quantized_tensor.LinearActivationQuantizedTensor",
- "torchao.quantization.quant_api._int8_asymm_per_token_quant",
- "torch.serialization._get_layout",
- "torch.device",
- "collections.OrderedDict",
- "torchao.dtypes.uintx.q_dq_layout.QDQLayout",
- "torch.BFloat16Storage",
- "torchao.dtypes.uintx.q_dq_layout.QDQTensorImpl",
- "torch.int8",
- "torch.CharStorage",
- "torch._tensor._rebuild_from_type_v2",
- "torchao.dtypes.affine_quantized_tensor.AffineQuantizedTensor",
- "torchao.quantization.quant_primitives.ZeroPointDomain",
- "torch.FloatStorage",
- "torch.float32"
How to fix it?
4.79 GB
xet

Upload Qwen3ForCausalLM 7 months ago
pytorch_model_converted.bin
Detected Pickle imports (17)
- "torch.BFloat16Storage",
- "torch.CharStorage",
- "torch._utils._rebuild_wrapper_subclass",
- "collections.OrderedDict",
- "torchao.dtypes.uintx.q_dq_layout.QDQLayout",
- "torch.FloatStorage",
- "torch.float32",
- "torchao.quantization.quant_primitives.ZeroPointDomain",
- "torchao.quantization.linear_activation_quantized_tensor.LinearActivationQuantizedTensor",
- "torchao.dtypes.uintx.q_dq_layout.QDQTensorImpl",
- "torchao.quantization.quant_api._int8_asymm_per_token_quant",
- "torch.int8",
- "torch._utils._rebuild_tensor_v2",
- "torch._tensor._rebuild_from_type_v2",
- "torchao.dtypes.affine_quantized_tensor.AffineQuantizedTensor",
- "torch.serialization._get_layout",
- "torch.device"
How to fix it?
4.79 GB
xet

Rename qwen3-4b-converted.bin to pytorch_model_converted.bin 7 months ago
qwen3-4b-8da4w-1024-ctx.pte

2.91 GB
xet

Rename qwen3-4b-1024-ctx.pte to qwen3-4b-8da4w-1024-ctx.pte 7 months ago
special_tokens_map.json

613 Bytes

Upload tokenizer 7 months ago
tokenizer.json

11.4 MB
xet

Upload tokenizer 7 months ago
tokenizer_config.json

5.4 kB

Upload tokenizer 7 months ago
vocab.json

2.78 MB

Upload tokenizer 7 months ago