Locutusque commited on
Commit
f1fe4c6
·
verified ·
1 Parent(s): c6be53d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -23,7 +23,9 @@ model_name: liberalis-cogitator-llama-3.1-8b-dpo
23
 
24
  > *“Thought, unbound, is the only true frontier.”*
25
 
26
- **liberalis-cogitator-llama-3.1-8b** is not just a machine for words — it is a forge for ideas. With **8 billion parameters**, trained with a custom **Direct Preference Optimization (DPO)** algorithm on a dataset of **16,000 preference pairs** spanning **~450,000 conversations, problems, and stories**, this model embraces the philosophy that thought should wander without leash or muzzle.
 
 
27
 
28
  Its name — *liberalis cogitator* — whispers in Latin: *a thinker who is free*. Not merely free as in “without cost,” but free as in **without walls**.
29
 
 
23
 
24
  > *“Thought, unbound, is the only true frontier.”*
25
 
26
+ **liberalis-cogitator-llama-3.1-8b** is not just a machine for words — it is a forge for ideas. With **8 billion parameters**, trained with a custom **Direct Preference Optimization (DPO)** algorithm on a dataset of **16,000 preference pairs** and a SFT dataset spanning **~450,000 conversations, problems, and stories**, this model embraces the philosophy that thought should wander without leash or muzzle.
27
+
28
+ During DPO fine-tuning, the context window was scaled to 65536, giving this model the capabilities of long conversation.
29
 
30
  Its name — *liberalis cogitator* — whispers in Latin: *a thinker who is free*. Not merely free as in “without cost,” but free as in **without walls**.
31