Model Card for llama3-codeweaver-lora

Model Details

• Model name: llama3-codeweaver-lora
• Developed by: mahmoudalrefaey
• Funded by: None (personal project)
• Finetuned from: meta-llama/Meta-Llama-3-8B
• License: LLaMA 3 license

This is a LLaMA-3 8B model fine-tuned with QLoRA on the CoNaLa mined-curated dataset for code generation tasks.
The adapter was trained on Google Colab T4 (16GB) using fp16 mixed precision with QLoRA for efficiency.

---

Uses

Direct Use

• Intended for code generation assistant tasks such as transforming natural language instructions into Python snippets.
• Educational use for learning about LLM fine-tuning with LoRA adapters.

Downstream Use

• Can be further fine-tuned on specialized coding datasets (e.g. SQL, JS).
• Integration into coding assistants and research projects.

Out-of-Scope Use

• Not intended for production-critical code security auditing.
• Not guaranteed to generate safe or fully optimized code.
• Should not be used in environments where code execution safety is critical without sandboxing.

---

Training Details

Training Data

• Dataset: CoNaLa mined-curated
• Dataset size used: ~7,000 samples

Training Procedure

• Method: QLoRA fine-tuning with 4-bit quantization
• Precision: fp16 mixed precision
• Hardware: Google Colab T4 (16GB GPU)
• Batch size: 2 → effective batch 4 with accumulation
• Epochs: 3
• Training time: ~1h 30m

---

Evaluation

Testing Data

• Held-out validation split (10% of dataset)

Metrics

• Validation Loss decreased steadily across epochs
• Qualitative Evaluation: Generated Python snippets from validation prompts
• Example outputs matched reference solutions for common coding tasks

Example Prompt & Output

Prompt:
### Instruction:
Write code to convert integer num to list

### Code:

Generated:
[int(x) for x in str(num)]

Environmental Impact

• Hardware: NVIDIA T4 (16 GB VRAM)
• Cloud Provider: Google Colab
• Compute Region: Unknown
• Training Duration: ~1.5 hours

Citation

@misc{llama3-codeweaver-lora, author = {Mahmoud Alrefaey}, title = {llama3-codeweaver-lora: A QLoRA fine-tuned LLaMA-3 model for code generation}, year = {2025}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/mahmoudalrefaey/llama3-codeweaver-lora}}, }