Instructions to use muhtasham/spark-llm-finetune-tj with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use muhtasham/spark-llm-finetune-tj with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="muhtasham/spark-llm-finetune-tj")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("muhtasham/spark-llm-finetune-tj")
model = AutoModelForCausalLM.from_pretrained("muhtasham/spark-llm-finetune-tj", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use muhtasham/spark-llm-finetune-tj with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "muhtasham/spark-llm-finetune-tj"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "muhtasham/spark-llm-finetune-tj",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/muhtasham/spark-llm-finetune-tj

SGLang

How to use muhtasham/spark-llm-finetune-tj with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "muhtasham/spark-llm-finetune-tj" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "muhtasham/spark-llm-finetune-tj",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "muhtasham/spark-llm-finetune-tj" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "muhtasham/spark-llm-finetune-tj",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use muhtasham/spark-llm-finetune-tj with Docker Model Runner:
```
docker model run hf.co/muhtasham/spark-llm-finetune-tj
```

See axolotl config

axolotl version: 0.9.2

base_model: pretrained_models/Spark-TTS-0.5B/LLM
# Automatically upload checkpoint and final model to HF
hub_model_id: muhtasham/spark-llm-finetune-tj

trust_remote_code: true

strict: false

datasets:
  - path: data/output_prompt.jsonl
    type: completion

dataset_prepared_path:
val_set_size: 0.05
output_dir: ./outputs/out


sequence_len: 4098
sample_packing: true
eval_sample_packing: true
pad_to_sequence_len: true

wandb_project: spark-tts
wandb_entity:
wandb_watch:
wandb_name:
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 4
num_epochs: 50
optimizer: adamw_torch_fused
lr_scheduler: cosine
learning_rate: 0.0002

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: true

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 50
xformers_attention:
flash_attention: true

warmup_steps: 10
evals_per_epoch: 1
save_steps: 5000
debug:
deepspeed:
weight_decay: 0.0

spark-llm-finetune-tj

This model was trained from scratch on the data/output_prompt.jsonl dataset. It achieves the following results on the evaluation set:

Loss: 5.2546

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0002
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 4
total_train_batch_size: 16
optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 10
num_epochs: 50.0

Training results

Training Loss	Epoch	Step	Validation Loss
No log	0.0088	1	9.9240
5.5236	0.9978	114	5.5667
5.0799	1.9891	228	5.3932
4.9292	2.9803	342	5.3107
4.7729	3.9716	456	5.2529
4.7022	4.9628	570	5.2174
4.6598	5.9540	684	5.1988
4.6176	6.9453	798	5.1833
4.5814	7.9365	912	5.1737
4.5422	8.9278	1026	5.1687
4.506	9.9190	1140	5.1643
4.492	10.9103	1254	5.1646
4.4605	11.9015	1368	5.1670
4.4384	12.8928	1482	5.1699
4.4151	13.8840	1596	5.1751
4.4053	14.8753	1710	5.1766
4.3875	15.8665	1824	5.1807
4.3684	16.8578	1938	5.1879
4.3624	17.8490	2052	5.1921
4.3413	18.8403	2166	5.1983
4.3302	19.8315	2280	5.2020
4.3179	20.8228	2394	5.2081
4.3152	21.8140	2508	5.2157
4.306	22.8053	2622	5.2180
4.2989	23.7965	2736	5.2243
4.2982	24.7877	2850	5.2282
4.2862	25.7790	2964	5.2328
4.2827	26.7702	3078	5.2339
4.2775	27.7615	3192	5.2368
4.2802	28.7527	3306	5.2417
4.2686	29.7440	3420	5.2434
4.2713	30.7352	3534	5.2432
4.2689	31.7265	3648	5.2476
4.2687	32.7177	3762	5.2481
4.2651	33.7090	3876	5.2508
4.266	34.7002	3990	5.2509
4.2644	35.6915	4104	5.2517
4.2626	36.6827	4218	5.2517
4.2646	37.6740	4332	5.2525
4.2617	38.6652	4446	5.2524
4.2603	39.6565	4560	5.2544
4.2633	40.6477	4674	5.2537
4.2561	41.6389	4788	5.2522
4.2612	42.6302	4902	5.2546
4.2618	43.6214	5016	5.2530
4.2602	44.6127	5130	5.2540
4.2619	45.6039	5244	5.2543
4.263	46.5952	5358	5.2549
4.2625	47.5864	5472	5.2547
4.2611	48.5777	5586	5.2545
4.2621	49.5689	5700	5.2546

Framework versions

Transformers 4.51.3
Pytorch 2.7.1+cu126
Datasets 3.5.1
Tokenizers 0.21.1

Downloads last month: 12

Safetensors

Model size

0.5B params

Tensor type

BF16