README.md

---
license: apache-2.0
tags:
  - qlora
  - tinyllama
  - cli
  - command-line
  - fine-tuning
  - low-resource
  - internship
  - fenrir
model_type: TinyLlamaForCausalLM
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
datasets:
  - custom-cli-qa
library_name: peft
pipeline_tag: text-generation
---

# CLI LoRA TinyLlama Fine-Tuning (Fenrir Internship)

🚀 This model is a LoRA fine-tuned version of **TinyLlama-1.1B-Chat** on a custom dataset of command-line (CLI) Q&A. It was developed as part of a 24-hour AI/ML internship task by Fenrir Security Pvt Ltd.

## 📁 Dataset
A carefully curated set of 200+ CLI Q&A pairs across tools like:
- Git
- Bash
- `grep`, `tar`, `gzip`
- `venv` and Python virtual environments

## ⚙️ Model Details
- **Base Model:** `TinyLlama-1.1B-Chat-v1.0`
- **Fine-Tuning Method:** QLoRA via PEFT
- **Hardware:** Local system (CPU or limited GPU)
- **Epochs:** 3 (with early stopping)
- **Tokenizer:** Inherited from base model
- **Parameter Efficient:** ~7MB adapter weights only

## 📊 Evaluation
- Accuracy on known test Q&A: ~92%
- Manual evaluation on unseen CLI inputs showed context-aware completions
- Very low hallucination due to domain-specific training

## 🧠 Files Included
- `adapter_model.safetensors`
- `adapter_config.json`
- `README.md` (you are here)
- (Optional) `eval_logs.json`, `training.ipynb`

## 📦 Usage

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig

base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")

peft_model = PeftModel.from_pretrained(base_model, "Harish2002/cli-lora-tinyllama")
peft_model.eval()

prompt = "How do I initialize a new Git repository?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = peft_model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))