End of training

README.md

@@ -1,69 +1,69 @@
 ---
-library_name: transformers
 base_model: Ellio98/mistral-0.5B-base
+library_name: transformers
+model_name: mistral-0.5B-Instruct-v0.1
 tags:
 - generated_from_trainer
-model-index:
-- name: mistral-0.5B-Instruct-v0.1
-  results: []
+- trl
+- gkd
+licence: license
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# mistral-0.5B-Instruct-v0.1
-
-This model is a fine-tuned version of [Ellio98/mistral-0.5B-base](https://huggingface.co/Ellio98/mistral-0.5B-base) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: nan
+# Model Card for mistral-0.5B-Instruct-v0.1
 
-## Model description
+This model is a fine-tuned version of [Ellio98/mistral-0.5B-base](https://huggingface.co/Ellio98/mistral-0.5B-base).
+It has been trained using [TRL](https://github.com/huggingface/trl).
 
-More information needed
+## Quick start
 
-## Intended uses & limitations
+```python
+from transformers import pipeline
 
-More information needed
-
-## Training and evaluation data
-
-More information needed
+question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+generator = pipeline("text-generation", model="Ellio98/mistral-0.5B-Instruct-v0.1", device="cuda")
+output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+print(output["generated_text"])
+```
 
 ## Training procedure
 
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0005
-- train_batch_size: 4
-- eval_batch_size: 4
-- seed: 42
-- gradient_accumulation_steps: 4
-- total_train_batch_size: 16
-- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 100
-- num_epochs: 1
-
-### Training results
+ 
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 0.0           | 0.1000 | 125  | nan             |
-| 0.0           | 0.2000 | 250  | nan             |
-| 0.0           | 0.2999 | 375  | nan             |
-| 0.0           | 0.3999 | 500  | nan             |
-| 0.0           | 0.4999 | 625  | nan             |
-| 0.0           | 0.5999 | 750  | nan             |
-| 0.0           | 0.6999 | 875  | nan             |
-| 0.0           | 0.7998 | 1000 | nan             |
-| 0.0           | 0.8998 | 1125 | nan             |
-| 0.0           | 0.9998 | 1250 | nan             |
 
+This model was trained with GKD, a method introduced in [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://huggingface.co/papers/2306.13649).
 
 ### Framework versions
 
-- Transformers 4.47.0
-- Pytorch 2.5.1+cu121
-- Datasets 3.3.1
-- Tokenizers 0.21.0
+- TRL: 0.15.2
+- Transformers: 4.47.0
+- Pytorch: 2.5.1+cu121
+- Datasets: 3.3.1
+- Tokenizers: 0.21.0
+
+## Citations
+
+Cite GKD as:
+
+```bibtex
+@inproceedings{agarwal2024on-policy,
+    title        = {{On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes}},
+    author       = {Rishabh Agarwal and Nino Vieillard and Yongchao Zhou and Piotr Stanczyk and Sabela Ramos Garea and Matthieu Geist and Olivier Bachem},
+    year         = 2024,
+    booktitle    = {The Twelfth International Conference on Learning Representations, {ICLR} 2024, Vienna, Austria, May 7-11, 2024},
+    publisher    = {OpenReview.net},
+    url          = {https://openreview.net/forum?id=3zKtaqxLhW},
+}
+```
+
+Cite TRL as:
+    
+```bibtex
+@misc{vonwerra2022trl,
+	title        = {{TRL: Transformer Reinforcement Learning}},
+	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+	year         = 2020,
+	journal      = {GitHub repository},
+	publisher    = {GitHub},
+	howpublished = {\url{https://github.com/huggingface/trl}}
+}
+```

config.json

@@ -16,7 +16,7 @@
   "hidden_size": 1536,
   "initializer_range": 0.02,
   "intermediate_size": 4096,
-  "max_position_embeddings": 2048,
+  "max_position_embeddings": 4096,
   "model_type": "mistral",
   "num_attention_heads": 8,
   "num_hidden_layers": 16,
@@ -28,6 +28,6 @@
   "tie_word_embeddings": false,
   "torch_dtype": "float32",
   "transformers_version": "4.47.0",
-  "use_cache": true,
-  "vocab_size": 32000
+  "use_cache": false,
+  "vocab_size": 32768
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:77612e60162ff8aaaa3dac64dbe7c603d1e3dba02a733b4017b6a0da2bec9917
-size 2054379856
+oid sha256:30a5957a9d53c48936412c565e53ca82037d7b2ae88cd5b0b9237e33dd45f800
+size 2063817040

tokenizer.json

@@ -2,20 +2,11 @@
   "version": "1.0",
   "truncation": {
     "direction": "Right",
-    "max_length": 512,
+    "max_length": 199,
     "strategy": "LongestFirst",
     "stride": 0
   },
-  "padding": {
-    "strategy": {
-      "Fixed": 512
-    },
-    "direction": "Left",
-    "pad_to_multiple_of": null,
-    "pad_id": 2,
-    "pad_type_id": 0,
-    "pad_token": "</s>"
-  },
+  "padding": null,
   "added_tokens": [
     {
       "id": 0,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:afd73705e4ffeffaeae1bb1eeceadc6314146a3eed261671e37a854cfed3fbc3
-size 5304
+oid sha256:97c4112d49a5f5502436f07c430a3c5b2116dbc352be2ae2975ebc3a342d79b8
+size 5752