metadata
language:
- en
tags:
- text-to-sql
- gpt2
- gpt2-medium
- nlp-to-sql
- text2sql
- sql
Model Card for Model ID
This is my first fine tuned LLM project.
Prompt
query = List the creation year, name and budget of each department
f"Translate the following English question to SQL: {query}
Output
SELECT zip_code FROM Visibility WHERE AVG(visibility) < 10
[More Information Needed]
Training Hyperparameters
num_train_epochs=1 per_device_train_batch_size=3 gradient_accumulation_steps=9 learning_rate=5e-5 weight_decay=0.01
Evaluation
Step Training Loss
500 0.337800 1000 0.262900 1500 0.253200 2000 0.246400
{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}