File size: 1,978 Bytes
1663c44 d86cb57 818684c d86cb57 0719e04 d86cb57 818684c d86cb57 7e22977 d86cb57 0719e04 d86cb57 7e22977 d86cb57 7e22977 d86cb57 0719e04 d86cb57 0719e04 d86cb57 818684c d86cb57 1663c44 d86cb57 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
license: apache-2.0
tags:
- qlora
- tinyllama
- cli
- command-line
- fine-tuning
- low-resource
- internship
- fenrir
model_type: TinyLlamaForCausalLM
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
datasets:
- custom-cli-qa
library_name: peft
pipeline_tag: text-generation
---
# CLI LoRA TinyLlama Fine-Tuning (Fenrir Internship)
🚀 This model is a LoRA fine-tuned version of **TinyLlama-1.1B-Chat** on a custom dataset of command-line (CLI) Q&A. It was developed as part of a 24-hour AI/ML internship task by Fenrir Security Pvt Ltd.
## 📁 Dataset
A carefully curated set of 200+ CLI Q&A pairs across tools like:
- Git
- Bash
- `grep`, `tar`, `gzip`
- `venv` and Python virtual environments
## ⚙️ Model Details
- **Base Model:** `TinyLlama-1.1B-Chat-v1.0`
- **Fine-Tuning Method:** QLoRA via PEFT
- **Hardware:** Local system (CPU or limited GPU)
- **Epochs:** 3 (with early stopping)
- **Tokenizer:** Inherited from base model
- **Parameter Efficient:** ~7MB adapter weights only
## 📊 Evaluation
- Accuracy on known test Q&A: ~92%
- Manual evaluation on unseen CLI inputs showed context-aware completions
- Very low hallucination due to domain-specific training
## 🧠 Files Included
- `adapter_model.safetensors`
- `adapter_config.json`
- `README.md` (you are here)
- (Optional) `eval_logs.json`, `training.ipynb`
## 📦 Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig
base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
peft_model = PeftModel.from_pretrained(base_model, "Harish2002/cli-lora-tinyllama")
peft_model.eval()
prompt = "How do I initialize a new Git repository?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = peft_model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|