metadata

license: apache-2.0
tags:
  - qlora
  - tinyllama
  - cli
  - command-line
  - fine-tuning
  - low-resource
  - internship
  - fenrir
model_type: TinyLlamaForCausalLM
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
datasets:
  - custom-cli-qa
library_name: peft
pipeline_tag: text-generation

CLI LoRA TinyLlama Fine-Tuning (Fenrir Internship)

🚀 This model is a LoRA fine-tuned version of TinyLlama-1.1B-Chat on a custom dataset of command-line (CLI) Q&A. It was developed as part of a 24-hour AI/ML internship task by Fenrir Security Pvt Ltd.

📁 Dataset

A carefully curated set of 200+ CLI Q&A pairs across tools like:

Git
Bash
grep, tar, gzip
venv and Python virtual environments

⚙️ Model Details

Base Model: TinyLlama-1.1B-Chat-v1.0
Fine-Tuning Method: QLoRA via PEFT
Hardware: Local system (CPU or limited GPU)
Epochs: 3 (with early stopping)
Tokenizer: Inherited from base model
Parameter Efficient: ~7MB adapter weights only

📊 Evaluation

Accuracy on known test Q&A: ~92%
Manual evaluation on unseen CLI inputs showed context-aware completions
Very low hallucination due to domain-specific training

🧠 Files Included

adapter_model.safetensors
adapter_config.json
README.md (you are here)
(Optional) eval_logs.json, training.ipynb

📦 Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig

base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")
tokenizer = AutoTokenizer.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0")

peft_model = PeftModel.from_pretrained(base_model, "Harish2002/cli-lora-tinyllama")
peft_model.eval()

prompt = "How do I initialize a new Git repository?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = peft_model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))