metadata
license: mit
license: mit tags: - tinyllama - lora - cli - fine-tuning - qna - transformers - peft library_name: transformers datasets: - custom language: en model_type: causal-lm
🔧 CLI LoRA-TinyLlama
A fine-tuned version of TinyLlama-1.1B-Chat-v1.0 on a custom dataset of command-line Q&A, using LoRA (Low-Rank Adaptation). Built for fast, accurate help on common CLI topics.
🧩 Base Model
- Model:
TinyLlama/TinyLlama-1.1B-Chat-v1.0 - Fine-Tuning Method: LoRA
- Libraries Used:
transformers,peft,datasets,accelerate
📚 Dataset
- Custom dataset with 150+ Q&A pairs covering:
git,bash,grep,tar,venv
- Raw file:
cli_questions.json - Tokenized version:
tokenized_dataset/
🛠️ Training Configuration
from peft import LoraConfig
base_model = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
lora_config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.1,
bias="none",
task_type="CAUSAL_LM"
)