Instructions to use SeongryongJung/Qwen-4b-base-RLSD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SeongryongJung/Qwen-4b-base-RLSD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SeongryongJung/Qwen-4b-base-RLSD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SeongryongJung/Qwen-4b-base-RLSD")
model = AutoModelForCausalLM.from_pretrained("SeongryongJung/Qwen-4b-base-RLSD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SeongryongJung/Qwen-4b-base-RLSD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SeongryongJung/Qwen-4b-base-RLSD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SeongryongJung/Qwen-4b-base-RLSD

SGLang

How to use SeongryongJung/Qwen-4b-base-RLSD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SeongryongJung/Qwen-4b-base-RLSD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SeongryongJung/Qwen-4b-base-RLSD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SeongryongJung/Qwen-4b-base-RLSD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SeongryongJung/Qwen-4b-base-RLSD with Docker Model Runner:
```
docker model run hf.co/SeongryongJung/Qwen-4b-base-RLSD
```

Qwen3-4B-Base RLSD

RLSD self-distillation reinforcement learning on the local math training split.

This repository contains the final merged Hugging Face checkpoint from global_step_100. The training checkpoint was saved from FSDP shards and merged to safetensors for this upload.

Training Method

Policy loss mode: rlsd.
Self-distillation uses reprompt feedback and token reweighting.
Token reweighting: lambda 0.5, eps_w 0.2, decay steps 50.
Advantage estimator remains grpo in the trainer config.
Reward function: local math compute_score reward manager.
Fine-tuning type: full-parameter FSDP training, not LoRA.

Training Hyperparameters

Field	Value
Base model	`Qwen/Qwen3-4B-Base`
Train file	`/home1/irteam/SDPO/self-distillation-analysis/data/math/train.parquet`
Validation file	`/home1/irteam/SDPO/self-distillation-analysis/data/math/evaluation/aime24.parquet`
Train max samples	25600
Train batch size	256
Rollouts per prompt	8
PPO mini batch size	128
PPO micro batch size per GPU	1
Optimizer	AdamW
Learning rate	1e-06
Weight decay	0.01
LR warmup steps	10
Total training steps	100
Save frequency	every 10 steps
Validation frequency	every 10 steps
Max prompt length	2048
Max response length	20480
Rollout backend	vllm
Rollout temperature	1
Rollout top_p	1
vLLM GPU memory utilization	0.75
Actor strategy	fsdp
Dtype	bfloat16
Advantage estimator	grpo
Gamma / Lambda	1 / 1
KL loss enabled	False
KL loss coefficient	0.001
Checkpoint uploaded	`math-RLSD-Qwen3-4B-Base-128-train256-rollout8-lr1e-6-vllm0.75-modelQwen-Qwen3-4B-Base/global_step_100`
W&B run id	`3tuehy90`

Training Score

The plot below shows critic/score/mean logged during training.

CSV data is included in training_score.csv.

Metric	Value
Final training step	100
Final `critic/score/mean`	0.304199
Final `val-core/math_dapo/acc/mean@1`	0.1

Intended Use

This model is intended for internal research and analysis of math-focused RL fine-tuning methods. It has not been broadly safety evaluated for production use.

Limitations

The model was trained for 100 optimization steps on a local math dataset split. Reported scores are training-time reward/validation metrics from the same experiment setup and should not be treated as broad benchmark results.

Downloads last month: 11

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for SeongryongJung/Qwen-4b-base-RLSD

Base model

Qwen/Qwen3-4B-Base

Finetuned

(350)

this model