metadata

license: mit
tags:
  - tinyllama
  - lora
  - cli
  - fine-tuning
  - qna
  - transformers
  - peft
library_name: transformers
datasets:
  - custom
language: en
model_type: causal-lm

🔧 CLI LoRA-TinyLlama

A fine-tuned version of TinyLlama-1.1B-Chat-v1.0 on a custom dataset of command-line Q&A, using LoRA (Low-Rank Adaptation). Built for fast, accurate help on common CLI topics.

🧩 Base Model

Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
Fine-Tuning Method: LoRA
Libraries Used: transformers, peft, datasets, accelerate

📚 Dataset

Custom dataset with 150+ Q&A pairs covering:
- git, bash, grep, tar, venv
Raw file: cli_questions.json
Tokenized version: tokenized_dataset/

🛠️ Training Configuration

from peft import LoraConfig

base_model = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM"
)