---
license: mit
tags:
  - tinyllama
  - lora
  - cli
  - fine-tuning
  - qna
  - transformers
  - peft
library_name: transformers
datasets:
  - custom
language: en
model_type: causal-lm
---

# 🔧 CLI LoRA-TinyLlama

A fine-tuned version of [TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on a custom dataset of command-line Q&A, using **LoRA** (Low-Rank Adaptation). Built for fast, accurate help on common CLI topics.

---

## 🧩 Base Model

- Model: `TinyLlama/TinyLlama-1.1B-Chat-v1.0`
- Fine-Tuning Method: [LoRA](https://arxiv.org/abs/2106.09685)
- Libraries Used: `transformers`, `peft`, `datasets`, `accelerate`

---

## 📚 Dataset

- Custom dataset with **150+ Q&A pairs** covering:
  - `git`, `bash`, `grep`, `tar`, `venv`
- Raw file: `cli_questions.json`
- Tokenized version: `tokenized_dataset/`

---

## 🛠️ Training Configuration

```python
from peft import LoraConfig

base_model = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.1,
    bias="none",
    task_type="CAUSAL_LM"
)