README.md

---
base_model: onekq-ai/OneSQL-v0.2-Qwen-3B
tags:
- text-generation-inference
- transformers
- qwen2
- gguf
license: apache-2.0
language:
- en
---

# Disclaimer
Your email will be used for anonymous survey. It will NOT be shared with anyone.

# Introduction

This model is the GGUF version of [OneSQL-v0.2-Qwen-3B](https://huggingface.co/onekq-ai/OneSQL-v0.2-Qwen-3B).

# Performances

Below is the self-evaluation results for each quantization and its improvement over [OneSQL-v0.1-Qwen-3B-GGUF](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-3B-GGUF).

| Quantization |EX score|v0.1 EX score
|------------|-------|-------|
| Q4_0       | 29.59 | 16.83 |
| Q4_1       | 32.35 | 21.85 |
| Q4_K_S     | 31.16 | 22.49 |
| Q4_K_M     | 31.03 | 21.85 |
| Q5_0       | 31.24 | 23.40 |
| Q5_1       | 33.27 | 23.53 |
| Q5_K_S     | 34.38 | 22.77 |
| Q5_K_M     | **34.49** | 23.73 |
| Q6_K       | 32.68 | 24.51 |
| Q8_0       | 32.59 | **24.90** |

# Quick start

To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**.
Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit.

```sh
PROMPT="CREATE TABLE students (
    id INTEGER PRIMARY KEY,
    name TEXT,
    age INTEGER,
    grade TEXT
);

-- Find the three youngest students
SELECT "

PROMPT=$(printf "<|im_start|>system\nYou are a SQL expert. Return code only.<|im_end|>\n<|im_start|>user\n%s<|im_end|>\n<|im_start|>assistant\n" "$PROMPT")

llama.cpp/build/bin/llama-run file://OneSQL-v0.2-Qwen-3B-Q4_K_M.gguf "$PROMPT"
```

The model response is the finished SQL query without **SELECT**
```sql
* FROM students ORDER BY age ASC LIMIT 3
```

# Caveats

* The performance drop from the original model is due to quantization itself, and the lack of beam search support in llama.cpp framework. Use at your own discretion.
* The 2-bit and 3-bit quantizations suffer from repetitive and unrelevant output token, hence are not recommended for usage.