README.md

---
library_name: transformers
tags:
- unsloth
- trl
- sft
license: apache-2.0
language:
- en
base_model:
- meta-llama/Llama-3.2-3B-Instruct
pipeline_tag: text-generation
metrics:
- accuracy
- bleu
- rouge
---

# Model Card for MediLlama-3.2

A fine-tuned version of Meta's LLaMA 3.2 (3B Instruct) for domain-specific applications in healthcare and medicine. This model is optimized for tasks such as medical Q&A, symptom checking, and patient education.

## Model Details

### Model Description

This model is a domain-adapted version of LLaMA 3.2 3B Instruct. It has been fine-tuned using supervised fine-tuning (SFT) on medical datasets to handle English-language healthcare scenarios including diagnostic queries, treatment suggestions, and general medical advice.

- **Developed by:** InferenceLab  
- **Model type:** Medical Chatbot  
- **Language(s) (NLP):** English  
- **License:** Apache 2.0  
- **Finetuned from model:** meta-llama/Llama-3.2-3B-Instruct  

## Uses

### Direct Use

MediLlama-3.2 can be used directly as a chatbot or virtual assistant in medical and health-related applications. Ideal for educational content, initial symptom triage, and research purposes.

### Downstream Use

Can be integrated into larger telehealth systems, clinical documentation tools, or diagnostic assistants after further task-specific fine-tuning.

### Out-of-Scope Use

- Should not be used for real-time diagnosis or treatment decisions without expert validation.  
- Not suitable for high-risk or life-threatening emergency response.  
- Not trained on pediatric or highly specialized medical domains.  

## Bias, Risks, and Limitations

While the model is trained on medical data, it may still exhibit:  
- Biases from source data  
- Hallucinations or incorrect suggestions  
- Outdated or non-region-specific medical advice  

### Recommendations

Users should validate outputs with certified medical professionals. This model is for research and prototyping only, not for clinical deployment without regulatory compliance.

## How to Get Started with the Model

```python
import torch
from transformers import pipeline

model_id = "InferenceLab/MediLlama-3.2"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful Medical assistant."},
    {"role": "user", "content": "Hi! How are you?"},
]
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

````

## Training Details

### Training Data

Model trained using cleaned and preprocessed medical QA datasets, synthetic doctor-patient conversations, and publicly available health forums. Protected health information (PHI) was removed.

### Training Procedure

Supervised fine-tuning (SFT) using TRL and Unsloth libraries.

#### Preprocessing

Tokenization using LLaMA tokenizer with special medical instruction formatting.

#### Training Hyperparameters

* **Training regime:** bf16 mixed precision
* **Learning rate:** 1e-5

#### Speeds, Sizes, Times

* **Training time:** \~12 hours on 4×A100 GPUs

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

Subset of unseen medical QA pairs, synthetic test cases, and MedQA-derived examples.

#### Factors

* Input prompt complexity
* Use of medical terminology
* Chat length

#### Metrics

* **Accuracy:** 81.3%
* **BLEU:** 34.5
* **ROUGE-L:** 62.2

### Results

#### Summary

Model shows good generalization to unseen prompts and performs competitively for general medical dialogue. Further tuning needed for specialty areas like oncology or rare diseases.

## Model Examination

Explainability tools like LLaMA-MedLens (if available) are suggested to interpret model decisions.

## Environmental Impact

* **Hardware Type:** 4×NVIDIA A100 40GB
* **Hours used:** 12
* **Cloud Provider:** AWS
* **Compute Region:** us-west-2
* **Carbon Emitted:** \~35.8 kg CO2eq (estimated)

## Technical Specifications

### Model Architecture and Objective

* Based on Meta LLaMA 3.2 3B Instruct
* Decoder-only transformer
* Objective: Causal Language Modeling (CLM) with instruction fine-tuning

### Compute Infrastructure

#### Hardware

* 4×NVIDIA A100 40GB

#### Software

* Python 3.10
* Transformers (v4.40+)
* TRL
* Unsloth
* PyTorch 2.1


## Glossary

* **SFT**: Supervised Fine-Tuning
* **BLEU**: Bilingual Evaluation Understudy
* **ROUGE**: Recall-Oriented Understudy for Gisting Evaluation

## More Information

For collaborations, deployment help, or fine-tuning extensions, please contact the developers.

## Model Card Authors

* InferenceLab Team