Update README.md

README.md

@@ -4,4 +4,40 @@ datasets:
 - Photolens/alpaca-cleaned-airoboros-2.1-no-code-oasst1-en-merged
 language:
 - en
----
+---
+
+## Model overview
+This model is finetuned on *[a merged dataset of: oasst1-en, alpaca-cleaned and airoboros-2.1-no-code](https://huggingface.co/datasets/Photolens/alpaca-cleaned-airoboros-2.1-no-code-oasst1-en-merged)* on a base model: *[Marx-3b-V2](https://huggingface.co/acrastt/Marx-3B-V2)*
+ - License: "`Creative-Commons-Attribution-4.0`"
+ - Language: "`en`"
+ - Size: "`3.43b params`"
+
+## Prompt template
+Prompt template:
+```
+### SYSTEM:
+<system_prompt_here>
+
+### HUMAN:
+<prompter_message_here>
+
+### INPUT:
+<input_text_here>
+
+### RESPONSE:
+<leave_a_blank_line_here>
+```
+*Note: If you dont have a system or input text, do not include the tokens in the prompt.*
+
+## Training Details
+This model took `2:40:54` to train in LoRA on a single `A100 40gb` GPU.<br>
+ - *epochs*:  `1`
+ - *train batch size*:  `8`
+ - *eval batch size*:  `8`
+ - *gradient accumulation steps*:  `1`
+ - *maximum gradient normal*:  `0.3`
+ - *learning rate*:  `2e-4`
+ - *weight decay*:  `0.001`
+ - *optimizer*:  `paged_adamw_32bit`
+ - *learning rate schedule*:  `cosine`
+ - *warmup ratio (linear)*:  `0.03`