Qwen3-32B-alpaca-th-52k-dolly-th-15k-wangchan-instruct-seed-4201

This model is a fine-tuned version of Qwen/Qwen3-32B on the alpaca-th-52k, the dolly-th-15k and the wangchan-instruct datasets. It achieves the following results on the evaluation set:

• Loss: 0.6413

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 2
• eval_batch_size: 2
• seed: 4201
• distributed_type: multi-GPU
• num_devices: 32
• gradient_accumulation_steps: 8
• total_train_batch_size: 512
• total_eval_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.9293	0.0575	10	1.0471
0.8085	0.1149	20	0.8245
0.7547	0.1724	30	0.7581
0.7289	0.2299	40	0.7386
0.6965	0.2874	50	0.7242
0.6848	0.3448	60	0.7109
0.693	0.4023	70	0.7022
0.7101	0.4598	80	0.6947
0.7293	0.5172	90	0.6888
0.6852	0.5747	100	0.6822
0.7033	0.6322	110	0.6770
0.6815	0.6897	120	0.6736
0.679	0.7471	130	0.6707
0.6571	0.8046	140	0.6682
0.6491	0.8621	150	0.6660
0.7015	0.9195	160	0.6636
0.6523	0.9770	170	0.6619
0.6672	1.0345	180	0.6602
0.6862	1.0920	190	0.6588
0.6755	1.1494	200	0.6577
0.6279	1.2069	210	0.6563
0.6622	1.2644	220	0.6551
0.6329	1.3218	230	0.6542
0.6559	1.3793	240	0.6528
0.6389	1.4368	250	0.6517
0.6476	1.4943	260	0.6506
0.6412	1.5517	270	0.6497
0.6232	1.6092	280	0.6485
0.6243	1.6667	290	0.6478
0.6467	1.7241	300	0.6469
0.6146	1.7816	310	0.6460
0.6386	1.8391	320	0.6450
0.6456	1.8966	330	0.6443
0.6402	1.9540	340	0.6437
0.6455	2.0115	350	0.6434
0.5888	2.0690	360	0.6437
0.6267	2.1264	370	0.6435
0.6292	2.1839	380	0.6434
0.6058	2.2414	390	0.6432
0.6221	2.2989	400	0.6427
0.6254	2.3563	410	0.6428
0.6178	2.4138	420	0.6423
0.6161	2.4713	430	0.6420
0.634	2.5287	440	0.6419
0.6241	2.5862	450	0.6418
0.6084	2.6437	460	0.6416
0.6264	2.7011	470	0.6415
0.608	2.7586	480	0.6413
0.6039	2.8161	490	0.6413
0.6445	2.8736	500	0.6413
0.6249	2.9310	510	0.6413
0.6006	2.9885	520	0.6413

Framework versions

• PEFT 0.15.2
• Transformers 4.52.3
• Pytorch 2.7.0+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1