File size: 2,316 Bytes
d4f8732 bf42bfb d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f d4f8732 540ce3f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 |
---
library_name: transformers
tags:
- llama-factory
base_model:
- mistralai/Mistral-7B-Instruct-v0.3
---
## Summary
This model is a fine-tuned version of the Mistral-7B-Instruct-v0.3 optimised for answering questions in the domain of forensic investigations. The model has been trained using a specialised dataset titled **Advanced_Forensic_Investigations_Knowledge_Library_v1**, which consists of approximately 100 domain-specific question-answer pairs. The objective is to support advanced forensic investigative reasoning, rapid knowledge retrieval, and high-precision forensic domain assistance.
---
## Model Details
### Model Description
- **Model type:** Instruction-following Language Model (LoRA-based fine-tuning)
- **Language(s):** English
- **Fine-tuned from model:** Mistral-7B-Instruct-v0.3
---
## Training Details
### Training Data
- **Dataset:** `Advanced_Forensic_Investigations_Knowledge_Library_v1`
- **Data size:** ~100 high-quality, domain-specific QA pairs
### Training Procedure
#### Preprocessing
- Template: `mistral`
- Token truncation/cutoff: 2048
- No vocab resizing or prompt packing
#### Hyperparameters
- **Finetuning type:** LoRA
- **Precision:** `bf16`
- **LoRA rank:** 16
- **LoRA alpha:** 32
- **Batch size:** 4
- **Gradient accumulation:** 8
- **Learning rate:** `3e-4`
- **Epochs:** 35
- **LR scheduler:** cosine
- **Quantisation:** 4-bit (bitsandbytes)
- **Cutoff length:** 2048
#### Compute
- **Training time:** close to 30 minutes
- **Framework:** LLaMA-Factory
---
## Evaluation
### Testing Data, Factors & Metrics
- **Metrics Used:** BLEU-4, ROUGE-1, ROUGE-2
- **Results:**
- BLEU-4: 100%
- ROUGE-1: 100%
- ROUGE-2: 100%
These scores reflect perfect overlap with reference answers within the scope of the evaluation dataset.
---
## Technical Specifications
### Model Architecture and Objective
- **Base:** Transformer (Mistral-7B architecture)
- **Fine-tuning method:** LoRA
- **Objective:** Instruction-following with forensic legal knowledge adaptation
### Compute Infrastructure
- **Hardware:** 2xL40s
- **Software:** LLaMA-Factory, PyTorch, Transformers, bitsandbytes |