|
|
--- |
|
|
base_model: onekq-ai/OneSQL-v0.2-Qwen-3B |
|
|
tags: |
|
|
- text-generation-inference |
|
|
- transformers |
|
|
- qwen2 |
|
|
- gguf |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
--- |
|
|
|
|
|
# Disclaimer |
|
|
Your email will be used for anonymous survey. It will NOT be shared with anyone. |
|
|
|
|
|
# Introduction |
|
|
|
|
|
This model is the GGUF version of [OneSQL-v0.2-Qwen-3B](https://huggingface.co/onekq-ai/OneSQL-v0.2-Qwen-3B). |
|
|
|
|
|
# Performances |
|
|
|
|
|
Below is the self-evaluation results for each quantization and its improvement over [OneSQL-v0.1-Qwen-3B-GGUF](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-3B-GGUF). |
|
|
|
|
|
| Quantization |EX score|v0.1 EX score |
|
|
|------------|-------|-------| |
|
|
| Q4_0 | 29.59 | 16.83 | |
|
|
| Q4_1 | 32.35 | 21.85 | |
|
|
| Q4_K_S | 31.16 | 22.49 | |
|
|
| Q4_K_M | 31.03 | 21.85 | |
|
|
| Q5_0 | 31.24 | 23.40 | |
|
|
| Q5_1 | 33.27 | 23.53 | |
|
|
| Q5_K_S | 34.38 | 22.77 | |
|
|
| Q5_K_M | **34.49** | 23.73 | |
|
|
| Q6_K | 32.68 | 24.51 | |
|
|
| Q8_0 | 32.59 | **24.90** | |
|
|
|
|
|
# Quick start |
|
|
|
|
|
To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**. |
|
|
Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit. |
|
|
|
|
|
```sh |
|
|
PROMPT="CREATE TABLE students ( |
|
|
id INTEGER PRIMARY KEY, |
|
|
name TEXT, |
|
|
age INTEGER, |
|
|
grade TEXT |
|
|
); |
|
|
|
|
|
-- Find the three youngest students |
|
|
SELECT " |
|
|
|
|
|
PROMPT=$(printf "<|im_start|>system\nYou are a SQL expert. Return code only.<|im_end|>\n<|im_start|>user\n%s<|im_end|>\n<|im_start|>assistant\n" "$PROMPT") |
|
|
|
|
|
llama.cpp/build/bin/llama-run file://OneSQL-v0.2-Qwen-3B-Q4_K_M.gguf "$PROMPT" |
|
|
``` |
|
|
|
|
|
The model response is the finished SQL query without **SELECT** |
|
|
```sql |
|
|
* FROM students ORDER BY age ASC LIMIT 3 |
|
|
``` |
|
|
|
|
|
# Caveats |
|
|
|
|
|
* The performance drop from the original model is due to quantization itself, and the lack of beam search support in llama.cpp framework. Use at your own discretion. |
|
|
* The 2-bit and 3-bit quantizations suffer from repetitive and unrelevant output token, hence are not recommended for usage. |
