Created using the fork pwilkin/llama.cpp, commit 8f64302. The main repo now supports the models! The quantization process is still the same, no re-making of the models is needed.

This is still in development, expect issues.

Settings
- temp: 1.1
- top-p: 0.95

The IQ models are made using bartowski1182/calibration_datav3.txt.

Downloads last month: 580

GGUF

Model size

4.06M params

Architecture

undefined

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

View +1 variant

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for RDson/Seed-OSS-36B-Instruct-GGUF

Base model

ByteDance-Seed/Seed-OSS-36B-Instruct

Quantized

(19)

this model