OpenAssistant
/

pythia-12b-sft-v8-2.5k-steps

Text Generation

text-generation-inference

Model card Files Files and versions

andreaskoepf commited on May 7, 2023

Commit

aa617fe

·

1 Parent(s): c3708e0

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -12,7 +12,7 @@ pythia-12b-sft-8:
   model_name: OpenAssistant/pythia-12b-pre-v8-12.5k-steps
   output_dir: pythia_model_12b
   weight_decay: 0.0
-  residual_dropout: 0.1
   max_length: 2048
   use_flash_attention: true
   warmup_steps: 100

   model_name: OpenAssistant/pythia-12b-pre-v8-12.5k-steps
   output_dir: pythia_model_12b
   weight_decay: 0.0
+  residual_dropout: 0.0
   max_length: 2048
   use_flash_attention: true
   warmup_steps: 100