Nutanix/Meta-Llama-3-8B_SFT_lora_cpp_train_dataset_chat_format_less_than_8k

Model Training Configuration

This document outlines the configuration parameters used for training the model.

Training Parameters

• Epochs: 2
• Base Model: LLAMA3-8B Instruct

LoRA Configuration

• lora_alpha: 8
• lora_dropout: 0.05
• lora_rank: 8

General Configuration

• seed: 42
• adam_epsilon: 1e-08
• lr_scheduler_type: "linear"
• logging_steps: 100
• save_steps: 1000
• save_strategy: "steps"
• evaluation_strategy: "steps"
• eval_steps: 100
• max_seq_length: 8192