Ayurveda Question Answering - LLaMA 3 8B LoRA Fine-tuned

This model is a fine-tuned version of Meta-Llama-3-8B-Instruct using PEFT (LoRA) on the Macromrit/ayurveda-text-based-qanda dataset.
It has been trained for 1 epoch to answer Ayurveda-related questions in a natural and informative way.

Model Details

Model Description

Developed by: Independent contributor (vibhu5u)
Funded by: Self-funded
Model type: Causal Language Model (Instruction-tuned)
Language(s): English
License: MIT
Finetuned from model: meta-llama/Meta-Llama-3-8B-Instruct

Model Sources

Repository: Model on Hugging Face Hub
Dataset: Macromrit/ayurveda-text-based-qanda

Uses

Direct Use

This model can be used for:

Ayurveda question answering
Knowledge-based assistants in wellness applications
Educational content generation about Ayurveda

Downstream Use

Integration into chatbots for Ayurvedic healthcare guidance
Domain-specific retrieval-augmented generation (RAG) pipelines
Fine-tuning further for healthcare knowledge systems

Out-of-Scope Use

Do not use for medical diagnosis, treatment prescription, or emergency medical assistance.
Not intended to replace certified Ayurvedic practitioners.

Bias, Risks, and Limitations

Responses are limited to the scope of the training dataset (Ayurveda Q&A).
May produce hallucinations if asked about unrelated or modern medicine queries.
Could reflect cultural bias as Ayurveda is India-centric.

Recommendations

Always validate model outputs with certified Ayurvedic practitioners.
Use in educational and supportive contexts only, not clinical decisions.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "vibhu5u/llama3b-ayurveda"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "system", "content": "You are an expert Ayurvedic assistant."},
    {"role": "user", "content": "What is Apana Vayu and what does it govern?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Ayurveda Question Answering - LLaMA 3 8B LoRA Fine-Tuned

🧾 Training Details

Training Data

Dataset: Macromrit/ayurveda-text-based-qanda
Contains instruction–answer pairs focused on Ayurveda concepts, terminology, and practices.

Training Procedure

Method: Parameter-Efficient Fine-Tuning (LoRA via PEFT)
Epochs: 1
Precision: bfloat16 mixed precision
Sequence Length: 512 tokens
Optimizer: AdamW
Batch Size: 8 per device

Hardware

GPU: 1 × A100 80GB
Training Time: ~2 hours

✅ Evaluation

Testing Data

Subset of the Ayurveda Q&A dataset (test split).

Metrics

Perplexity (PPL): Evaluated for language modeling quality.
Human Evaluation: Checked for correctness and fluency of Ayurvedic answers.

Results

Model generates fluent, contextually relevant answers for Ayurveda questions.
Performs better than base LLaMA-3 8B on this domain-specific dataset.

🌍 Environmental Impact

Hardware Type: A100 80GB
Hours Used: ~2
Cloud Provider: Kaggle Notebook platform
Carbon Emitted: Estimated ≈ 3.2 kg CO₂eq (via ML CO2 Impact Calculator)

⚙️ Technical Specifications

Architecture: LLaMA-3 8B (decoder-only transformer)
Objective: Causal Language Modeling (Instruction-tuned Q&A)
Fine-tuning Method: LoRA (PEFT)

📖 Citation

If you use this model, please cite:

@misc{macromrit2025ayurvedaqa,
  title = {Ayurveda Question Answering with LLaMA 3 8B LoRA Fine-tuned},
  author = {vibhu5u},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/vibhu5u/llama3b-ayurveda}}
}

vibh5u
/

llama3b-ayurveda