add cot finetuned lora adapters for codegemma7b

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+codegemma_instruct_cot_ft_lora_r64_alpha64/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+codegemma_instruct_cot_ft_lora_r64_alpha64/**/tokenizer.json filter=lfs diff=lfs merge=lfs -text

codegemma_instruct_cot_ft_lora_r64_alpha64/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/codegemma-7b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

codegemma_instruct_cot_ft_lora_r64_alpha64/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/codegemma-7b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b9a683aab0635490d46fb72d7c531a3a443ffe4435c539ad2d7f933ae1397d5
+size 800116456

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/codegemma-7b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/codegemma-7b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64e267050be1a43e3b401da7e45c66497974e0eb05f8f8a684c91b23aad584c4
+size 800116456

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5ec055e0ecb1d77f399dcf0d034378575853e437b45a99193e4df6760dc771c
+size 406743412

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:203e9bfabd925cb4ec7129d24877156fcee87215187c35a867e358e56a9425a4
+size 14244

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd58b39bd6bfb59bb506313d8c2a30b14e9dc11a2d99de48bc99568a9510416f
+size 1064

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d964a2c8346d40f95791533eae48730d5f163c2e65fd16333560fd3e661df318
+size 34362915

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:583f2ebd2a1936009b7da991ea255504db68c7a9713a78673d1335a87098966c
+size 4241023

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/tokenizer_config.json

@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<|file_separator|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/trainer_state.json

@@ -0,0 +1,742 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.2058142526369951,
+  "eval_steps": 100,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0020581425263699513,
+      "grad_norm": 11.994463920593262,
+      "learning_rate": 2.061855670103093e-07,
+      "loss": 2.91,
+      "step": 1
+    },
+    {
+      "epoch": 0.004116285052739903,
+      "grad_norm": 11.769092559814453,
+      "learning_rate": 4.123711340206186e-07,
+      "loss": 2.8686,
+      "step": 2
+    },
+    {
+      "epoch": 0.0061744275791098535,
+      "grad_norm": 13.05551815032959,
+      "learning_rate": 6.185567010309279e-07,
+      "loss": 3.0286,
+      "step": 3
+    },
+    {
+      "epoch": 0.008232570105479805,
+      "grad_norm": 12.334521293640137,
+      "learning_rate": 8.247422680412372e-07,
+      "loss": 2.904,
+      "step": 4
+    },
+    {
+      "epoch": 0.010290712631849755,
+      "grad_norm": 12.075353622436523,
+      "learning_rate": 1.0309278350515464e-06,
+      "loss": 2.8991,
+      "step": 5
+    },
+    {
+      "epoch": 0.012348855158219707,
+      "grad_norm": 11.86032485961914,
+      "learning_rate": 1.2371134020618557e-06,
+      "loss": 3.0007,
+      "step": 6
+    },
+    {
+      "epoch": 0.014406997684589657,
+      "grad_norm": 10.10457992553711,
+      "learning_rate": 1.4432989690721649e-06,
+      "loss": 2.8493,
+      "step": 7
+    },
+    {
+      "epoch": 0.01646514021095961,
+      "grad_norm": 8.56408405303955,
+      "learning_rate": 1.6494845360824744e-06,
+      "loss": 2.9573,
+      "step": 8
+    },
+    {
+      "epoch": 0.01852328273732956,
+      "grad_norm": 6.307392120361328,
+      "learning_rate": 1.8556701030927837e-06,
+      "loss": 2.9507,
+      "step": 9
+    },
+    {
+      "epoch": 0.02058142526369951,
+      "grad_norm": 4.276430130004883,
+      "learning_rate": 2.061855670103093e-06,
+      "loss": 2.8988,
+      "step": 10
+    },
+    {
+      "epoch": 0.022639567790069464,
+      "grad_norm": 2.5912015438079834,
+      "learning_rate": 2.268041237113402e-06,
+      "loss": 2.9926,
+      "step": 11
+    },
+    {
+      "epoch": 0.024697710316439414,
+      "grad_norm": 2.018446207046509,
+      "learning_rate": 2.4742268041237115e-06,
+      "loss": 2.9874,
+      "step": 12
+    },
+    {
+      "epoch": 0.026755852842809364,
+      "grad_norm": 1.8558588027954102,
+      "learning_rate": 2.680412371134021e-06,
+      "loss": 2.8608,
+      "step": 13
+    },
+    {
+      "epoch": 0.028813995369179314,
+      "grad_norm": 1.9658265113830566,
+      "learning_rate": 2.8865979381443297e-06,
+      "loss": 2.8596,
+      "step": 14
+    },
+    {
+      "epoch": 0.030872137895549268,
+      "grad_norm": 1.872044563293457,
+      "learning_rate": 3.0927835051546395e-06,
+      "loss": 2.8836,
+      "step": 15
+    },
+    {
+      "epoch": 0.03293028042191922,
+      "grad_norm": 1.8884096145629883,
+      "learning_rate": 3.298969072164949e-06,
+      "loss": 2.9383,
+      "step": 16
+    },
+    {
+      "epoch": 0.03498842294828917,
+      "grad_norm": 1.8795744180679321,
+      "learning_rate": 3.5051546391752577e-06,
+      "loss": 2.883,
+      "step": 17
+    },
+    {
+      "epoch": 0.03704656547465912,
+      "grad_norm": 1.783678412437439,
+      "learning_rate": 3.7113402061855674e-06,
+      "loss": 2.8019,
+      "step": 18
+    },
+    {
+      "epoch": 0.039104708001029075,
+      "grad_norm": 1.820617914199829,
+      "learning_rate": 3.917525773195877e-06,
+      "loss": 2.8813,
+      "step": 19
+    },
+    {
+      "epoch": 0.04116285052739902,
+      "grad_norm": 1.8188731670379639,
+      "learning_rate": 4.123711340206186e-06,
+      "loss": 2.8401,
+      "step": 20
+    },
+    {
+      "epoch": 0.043220993053768975,
+      "grad_norm": 1.7305251359939575,
+      "learning_rate": 4.329896907216495e-06,
+      "loss": 2.7478,
+      "step": 21
+    },
+    {
+      "epoch": 0.04527913558013893,
+      "grad_norm": 1.7014551162719727,
+      "learning_rate": 4.536082474226804e-06,
+      "loss": 2.7356,
+      "step": 22
+    },
+    {
+      "epoch": 0.047337278106508875,
+      "grad_norm": 1.677381157875061,
+      "learning_rate": 4.742268041237113e-06,
+      "loss": 2.7593,
+      "step": 23
+    },
+    {
+      "epoch": 0.04939542063287883,
+      "grad_norm": 1.628554344177246,
+      "learning_rate": 4.948453608247423e-06,
+      "loss": 2.7689,
+      "step": 24
+    },
+    {
+      "epoch": 0.051453563159248775,
+      "grad_norm": 1.4968128204345703,
+      "learning_rate": 5.154639175257732e-06,
+      "loss": 2.6613,
+      "step": 25
+    },
+    {
+      "epoch": 0.05351170568561873,
+      "grad_norm": 1.4734832048416138,
+      "learning_rate": 5.360824742268042e-06,
+      "loss": 2.7095,
+      "step": 26
+    },
+    {
+      "epoch": 0.05556984821198868,
+      "grad_norm": 1.3745571374893188,
+      "learning_rate": 5.567010309278351e-06,
+      "loss": 2.655,
+      "step": 27
+    },
+    {
+      "epoch": 0.05762799073835863,
+      "grad_norm": 1.3381729125976562,
+      "learning_rate": 5.7731958762886594e-06,
+      "loss": 2.55,
+      "step": 28
+    },
+    {
+      "epoch": 0.05968613326472858,
+      "grad_norm": 1.3388073444366455,
+      "learning_rate": 5.979381443298969e-06,
+      "loss": 2.5219,
+      "step": 29
+    },
+    {
+      "epoch": 0.061744275791098535,
+      "grad_norm": 1.317008376121521,
+      "learning_rate": 6.185567010309279e-06,
+      "loss": 2.4491,
+      "step": 30
+    },
+    {
+      "epoch": 0.06380241831746848,
+      "grad_norm": 1.3210794925689697,
+      "learning_rate": 6.391752577319588e-06,
+      "loss": 2.4358,
+      "step": 31
+    },
+    {
+      "epoch": 0.06586056084383844,
+      "grad_norm": 1.182519555091858,
+      "learning_rate": 6.597938144329898e-06,
+      "loss": 2.4514,
+      "step": 32
+    },
+    {
+      "epoch": 0.06791870337020839,
+      "grad_norm": 1.2238099575042725,
+      "learning_rate": 6.804123711340207e-06,
+      "loss": 2.442,
+      "step": 33
+    },
+    {
+      "epoch": 0.06997684589657834,
+      "grad_norm": 1.1793314218521118,
+      "learning_rate": 7.010309278350515e-06,
+      "loss": 2.3864,
+      "step": 34
+    },
+    {
+      "epoch": 0.0720349884229483,
+      "grad_norm": 1.1983020305633545,
+      "learning_rate": 7.216494845360825e-06,
+      "loss": 2.3796,
+      "step": 35
+    },
+    {
+      "epoch": 0.07409313094931824,
+      "grad_norm": 1.2189652919769287,
+      "learning_rate": 7.422680412371135e-06,
+      "loss": 2.4152,
+      "step": 36
+    },
+    {
+      "epoch": 0.07615127347568819,
+      "grad_norm": 1.14923095703125,
+      "learning_rate": 7.628865979381444e-06,
+      "loss": 2.3298,
+      "step": 37
+    },
+    {
+      "epoch": 0.07820941600205815,
+      "grad_norm": 1.147013545036316,
+      "learning_rate": 7.835051546391754e-06,
+      "loss": 2.2488,
+      "step": 38
+    },
+    {
+      "epoch": 0.0802675585284281,
+      "grad_norm": 1.133981466293335,
+      "learning_rate": 8.041237113402063e-06,
+      "loss": 2.1825,
+      "step": 39
+    },
+    {
+      "epoch": 0.08232570105479804,
+      "grad_norm": 1.1686867475509644,
+      "learning_rate": 8.247422680412371e-06,
+      "loss": 2.2282,
+      "step": 40
+    },
+    {
+      "epoch": 0.084383843581168,
+      "grad_norm": 1.131690502166748,
+      "learning_rate": 8.453608247422681e-06,
+      "loss": 2.0962,
+      "step": 41
+    },
+    {
+      "epoch": 0.08644198610753795,
+      "grad_norm": 1.1626195907592773,
+      "learning_rate": 8.65979381443299e-06,
+      "loss": 2.1161,
+      "step": 42
+    },
+    {
+      "epoch": 0.0885001286339079,
+      "grad_norm": 1.1508581638336182,
+      "learning_rate": 8.865979381443299e-06,
+      "loss": 1.9856,
+      "step": 43
+    },
+    {
+      "epoch": 0.09055827116027786,
+      "grad_norm": 1.2286733388900757,
+      "learning_rate": 9.072164948453609e-06,
+      "loss": 2.076,
+      "step": 44
+    },
+    {
+      "epoch": 0.0926164136866478,
+      "grad_norm": 1.82068932056427,
+      "learning_rate": 9.278350515463918e-06,
+      "loss": 1.9995,
+      "step": 45
+    },
+    {
+      "epoch": 0.09467455621301775,
+      "grad_norm": 2.079101324081421,
+      "learning_rate": 9.484536082474226e-06,
+      "loss": 1.9601,
+      "step": 46
+    },
+    {
+      "epoch": 0.0967326987393877,
+      "grad_norm": 1.1209226846694946,
+      "learning_rate": 9.690721649484536e-06,
+      "loss": 1.9346,
+      "step": 47
+    },
+    {
+      "epoch": 0.09879084126575766,
+      "grad_norm": 1.0579711198806763,
+      "learning_rate": 9.896907216494846e-06,
+      "loss": 1.8764,
+      "step": 48
+    },
+    {
+      "epoch": 0.1008489837921276,
+      "grad_norm": 1.0434011220932007,
+      "learning_rate": 1.0103092783505156e-05,
+      "loss": 1.8483,
+      "step": 49
+    },
+    {
+      "epoch": 0.10290712631849755,
+      "grad_norm": 1.0089991092681885,
+      "learning_rate": 1.0309278350515464e-05,
+      "loss": 1.8018,
+      "step": 50
+    },
+    {
+      "epoch": 0.10496526884486751,
+      "grad_norm": 1.0117324590682983,
+      "learning_rate": 1.0515463917525775e-05,
+      "loss": 1.8003,
+      "step": 51
+    },
+    {
+      "epoch": 0.10702341137123746,
+      "grad_norm": 1.0006697177886963,
+      "learning_rate": 1.0721649484536083e-05,
+      "loss": 1.7482,
+      "step": 52
+    },
+    {
+      "epoch": 0.1090815538976074,
+      "grad_norm": 2.1164329051971436,
+      "learning_rate": 1.0927835051546391e-05,
+      "loss": 1.7363,
+      "step": 53
+    },
+    {
+      "epoch": 0.11113969642397736,
+      "grad_norm": 0.9573502540588379,
+      "learning_rate": 1.1134020618556703e-05,
+      "loss": 1.661,
+      "step": 54
+    },
+    {
+      "epoch": 0.11319783895034731,
+      "grad_norm": 1.0059764385223389,
+      "learning_rate": 1.134020618556701e-05,
+      "loss": 1.6979,
+      "step": 55
+    },
+    {
+      "epoch": 0.11525598147671726,
+      "grad_norm": 0.9719656109809875,
+      "learning_rate": 1.1546391752577319e-05,
+      "loss": 1.6318,
+      "step": 56
+    },
+    {
+      "epoch": 0.11731412400308722,
+      "grad_norm": 1.0024539232254028,
+      "learning_rate": 1.175257731958763e-05,
+      "loss": 1.6283,
+      "step": 57
+    },
+    {
+      "epoch": 0.11937226652945716,
+      "grad_norm": 0.9772456288337708,
+      "learning_rate": 1.1958762886597938e-05,
+      "loss": 1.5611,
+      "step": 58
+    },
+    {
+      "epoch": 0.12143040905582711,
+      "grad_norm": 0.9947625994682312,
+      "learning_rate": 1.2164948453608248e-05,
+      "loss": 1.6073,
+      "step": 59
+    },
+    {
+      "epoch": 0.12348855158219707,
+      "grad_norm": 2.112889051437378,
+      "learning_rate": 1.2371134020618558e-05,
+      "loss": 1.6208,
+      "step": 60
+    },
+    {
+      "epoch": 0.12554669410856703,
+      "grad_norm": 1.0515345335006714,
+      "learning_rate": 1.2577319587628866e-05,
+      "loss": 1.569,
+      "step": 61
+    },
+    {
+      "epoch": 0.12760483663493696,
+      "grad_norm": 1.0782145261764526,
+      "learning_rate": 1.2783505154639176e-05,
+      "loss": 1.5097,
+      "step": 62
+    },
+    {
+      "epoch": 0.12966297916130692,
+      "grad_norm": 1.154104232788086,
+      "learning_rate": 1.2989690721649485e-05,
+      "loss": 1.5472,
+      "step": 63
+    },
+    {
+      "epoch": 0.13172112168767688,
+      "grad_norm": 1.1614656448364258,
+      "learning_rate": 1.3195876288659795e-05,
+      "loss": 1.4833,
+      "step": 64
+    },
+    {
+      "epoch": 0.13377926421404682,
+      "grad_norm": 1.1720911264419556,
+      "learning_rate": 1.3402061855670103e-05,
+      "loss": 1.4644,
+      "step": 65
+    },
+    {
+      "epoch": 0.13583740674041678,
+      "grad_norm": 1.8903896808624268,
+      "learning_rate": 1.3608247422680415e-05,
+      "loss": 1.4286,
+      "step": 66
+    },
+    {
+      "epoch": 0.13789554926678674,
+      "grad_norm": 1.2675013542175293,
+      "learning_rate": 1.3814432989690723e-05,
+      "loss": 1.416,
+      "step": 67
+    },
+    {
+      "epoch": 0.13995369179315667,
+      "grad_norm": 1.266434907913208,
+      "learning_rate": 1.402061855670103e-05,
+      "loss": 1.3171,
+      "step": 68
+    },
+    {
+      "epoch": 0.14201183431952663,
+      "grad_norm": 1.3408889770507812,
+      "learning_rate": 1.4226804123711342e-05,
+      "loss": 1.3396,
+      "step": 69
+    },
+    {
+      "epoch": 0.1440699768458966,
+      "grad_norm": 1.3862446546554565,
+      "learning_rate": 1.443298969072165e-05,
+      "loss": 1.2642,
+      "step": 70
+    },
+    {
+      "epoch": 0.14612811937226652,
+      "grad_norm": 2.110553026199341,
+      "learning_rate": 1.4639175257731958e-05,
+      "loss": 1.2593,
+      "step": 71
+    },
+    {
+      "epoch": 0.14818626189863648,
+      "grad_norm": 1.7017499208450317,
+      "learning_rate": 1.484536082474227e-05,
+      "loss": 1.24,
+      "step": 72
+    },
+    {
+      "epoch": 0.15024440442500644,
+      "grad_norm": 1.9851700067520142,
+      "learning_rate": 1.5051546391752578e-05,
+      "loss": 1.2313,
+      "step": 73
+    },
+    {
+      "epoch": 0.15230254695137638,
+      "grad_norm": 2.009608030319214,
+      "learning_rate": 1.5257731958762888e-05,
+      "loss": 1.1281,
+      "step": 74
+    },
+    {
+      "epoch": 0.15436068947774634,
+      "grad_norm": 2.7587485313415527,
+      "learning_rate": 1.5463917525773197e-05,
+      "loss": 1.1248,
+      "step": 75
+    },
+    {
+      "epoch": 0.1564188320041163,
+      "grad_norm": 2.780954599380493,
+      "learning_rate": 1.5670103092783507e-05,
+      "loss": 1.0797,
+      "step": 76
+    },
+    {
+      "epoch": 0.15847697453048623,
+      "grad_norm": 3.1470866203308105,
+      "learning_rate": 1.5876288659793813e-05,
+      "loss": 1.0064,
+      "step": 77
+    },
+    {
+      "epoch": 0.1605351170568562,
+      "grad_norm": 4.653595447540283,
+      "learning_rate": 1.6082474226804127e-05,
+      "loss": 0.9219,
+      "step": 78
+    },
+    {
+      "epoch": 0.16259325958322615,
+      "grad_norm": 4.157363414764404,
+      "learning_rate": 1.6288659793814433e-05,
+      "loss": 0.8709,
+      "step": 79
+    },
+    {
+      "epoch": 0.16465140210959608,
+      "grad_norm": 4.5814924240112305,
+      "learning_rate": 1.6494845360824743e-05,
+      "loss": 0.7693,
+      "step": 80
+    },
+    {
+      "epoch": 0.16670954463596604,
+      "grad_norm": 5.096139907836914,
+      "learning_rate": 1.6701030927835052e-05,
+      "loss": 0.6868,
+      "step": 81
+    },
+    {
+      "epoch": 0.168767687162336,
+      "grad_norm": 4.858880519866943,
+      "learning_rate": 1.6907216494845362e-05,
+      "loss": 0.5971,
+      "step": 82
+    },
+    {
+      "epoch": 0.17082582968870594,
+      "grad_norm": 4.42564582824707,
+      "learning_rate": 1.7113402061855672e-05,
+      "loss": 0.4719,
+      "step": 83
+    },
+    {
+      "epoch": 0.1728839722150759,
+      "grad_norm": 7.720851421356201,
+      "learning_rate": 1.731958762886598e-05,
+      "loss": 0.3943,
+      "step": 84
+    },
+    {
+      "epoch": 0.17494211474144586,
+      "grad_norm": 0.41923192143440247,
+      "learning_rate": 1.752577319587629e-05,
+      "loss": 0.3635,
+      "step": 85
+    },
+    {
+      "epoch": 0.1770002572678158,
+      "grad_norm": 0.2771846354007721,
+      "learning_rate": 1.7731958762886598e-05,
+      "loss": 0.3597,
+      "step": 86
+    },
+    {
+      "epoch": 0.17905839979418575,
+      "grad_norm": 0.24761857092380524,
+      "learning_rate": 1.793814432989691e-05,
+      "loss": 0.3735,
+      "step": 87
+    },
+    {
+      "epoch": 0.1811165423205557,
+      "grad_norm": 0.23277048766613007,
+      "learning_rate": 1.8144329896907217e-05,
+      "loss": 0.3643,
+      "step": 88
+    },
+    {
+      "epoch": 0.18317468484692565,
+      "grad_norm": 0.22931228578090668,
+      "learning_rate": 1.8350515463917527e-05,
+      "loss": 0.3519,
+      "step": 89
+    },
+    {
+      "epoch": 0.1852328273732956,
+      "grad_norm": 0.20750615000724792,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 0.3431,
+      "step": 90
+    },
+    {
+      "epoch": 0.18729096989966554,
+      "grad_norm": 0.2080322951078415,
+      "learning_rate": 1.8762886597938147e-05,
+      "loss": 0.3632,
+      "step": 91
+    },
+    {
+      "epoch": 0.1893491124260355,
+      "grad_norm": 0.20186181366443634,
+      "learning_rate": 1.8969072164948453e-05,
+      "loss": 0.3492,
+      "step": 92
+    },
+    {
+      "epoch": 0.19140725495240546,
+      "grad_norm": 0.19172786176204681,
+      "learning_rate": 1.9175257731958766e-05,
+      "loss": 0.3552,
+      "step": 93
+    },
+    {
+      "epoch": 0.1934653974787754,
+      "grad_norm": 0.1747850626707077,
+      "learning_rate": 1.9381443298969072e-05,
+      "loss": 0.3355,
+      "step": 94
+    },
+    {
+      "epoch": 0.19552354000514535,
+      "grad_norm": 0.196411594748497,
+      "learning_rate": 1.9587628865979382e-05,
+      "loss": 0.3271,
+      "step": 95
+    },
+    {
+      "epoch": 0.1975816825315153,
+      "grad_norm": 0.20063228905200958,
+      "learning_rate": 1.9793814432989692e-05,
+      "loss": 0.3351,
+      "step": 96
+    },
+    {
+      "epoch": 0.19963982505788525,
+      "grad_norm": 0.19240939617156982,
+      "learning_rate": 2e-05,
+      "loss": 0.3266,
+      "step": 97
+    },
+    {
+      "epoch": 0.2016979675842552,
+      "grad_norm": 0.18206572532653809,
+      "learning_rate": 1.997709049255441e-05,
+      "loss": 0.3393,
+      "step": 98
+    },
+    {
+      "epoch": 0.20375611011062517,
+      "grad_norm": 0.20384562015533447,
+      "learning_rate": 1.9954180985108823e-05,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "grad_norm": 0.19944581389427185,
+      "learning_rate": 1.9931271477663232e-05,
+      "loss": 0.3268,
+      "step": 100
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "eval_loss": 0.3456890285015106,
+      "eval_runtime": 2114.0178,
+      "eval_samples_per_second": 3.677,
+      "eval_steps_per_second": 0.92,
+      "step": 100
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 970,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 3.305891627680727e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee55a11f5a306eb7c39b536fdfe2459bc279e468da50f6adda478c4deffcb812
+size 5688

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/codegemma-7b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/codegemma-7b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97f49550075985c6448d99511eea8c0d140b5aa2a60faa2277a7890044a1b911
+size 800116456

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a9e27cacef57683319dff6c0d7b7cf66889116f9593d5c58fbafe7bfe73c72e
+size 406743412

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4981123ff3cf7bd5b7f76839e90e4776f747ca4c38dcb41876fa010c0dea8b23
+size 14244

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83f316812b0c43f9354e615bde4cea4403d3ff2b0ee5dd42dfeae9f021d9e696
+size 1064

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d964a2c8346d40f95791533eae48730d5f163c2e65fd16333560fd3e661df318
+size 34362915

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:583f2ebd2a1936009b7da991ea255504db68c7a9713a78673d1335a87098966c
+size 4241023

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/tokenizer_config.json

@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<|file_separator|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/trainer_state.json

@@ -0,0 +1,1450 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.4116285052739902,
+  "eval_steps": 100,
+  "global_step": 200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0020581425263699513,
+      "grad_norm": 11.994463920593262,
+      "learning_rate": 2.061855670103093e-07,
+      "loss": 2.91,
+      "step": 1
+    },
+    {
+      "epoch": 0.004116285052739903,
+      "grad_norm": 11.769092559814453,
+      "learning_rate": 4.123711340206186e-07,
+      "loss": 2.8686,
+      "step": 2
+    },
+    {
+      "epoch": 0.0061744275791098535,
+      "grad_norm": 13.05551815032959,
+      "learning_rate": 6.185567010309279e-07,
+      "loss": 3.0286,
+      "step": 3
+    },
+    {
+      "epoch": 0.008232570105479805,
+      "grad_norm": 12.334521293640137,
+      "learning_rate": 8.247422680412372e-07,
+      "loss": 2.904,
+      "step": 4
+    },
+    {
+      "epoch": 0.010290712631849755,
+      "grad_norm": 12.075353622436523,
+      "learning_rate": 1.0309278350515464e-06,
+      "loss": 2.8991,
+      "step": 5
+    },
+    {
+      "epoch": 0.012348855158219707,
+      "grad_norm": 11.86032485961914,
+      "learning_rate": 1.2371134020618557e-06,
+      "loss": 3.0007,
+      "step": 6
+    },
+    {
+      "epoch": 0.014406997684589657,
+      "grad_norm": 10.10457992553711,
+      "learning_rate": 1.4432989690721649e-06,
+      "loss": 2.8493,
+      "step": 7
+    },
+    {
+      "epoch": 0.01646514021095961,
+      "grad_norm": 8.56408405303955,
+      "learning_rate": 1.6494845360824744e-06,
+      "loss": 2.9573,
+      "step": 8
+    },
+    {
+      "epoch": 0.01852328273732956,
+      "grad_norm": 6.307392120361328,
+      "learning_rate": 1.8556701030927837e-06,
+      "loss": 2.9507,
+      "step": 9
+    },
+    {
+      "epoch": 0.02058142526369951,
+      "grad_norm": 4.276430130004883,
+      "learning_rate": 2.061855670103093e-06,
+      "loss": 2.8988,
+      "step": 10
+    },
+    {
+      "epoch": 0.022639567790069464,
+      "grad_norm": 2.5912015438079834,
+      "learning_rate": 2.268041237113402e-06,
+      "loss": 2.9926,
+      "step": 11
+    },
+    {
+      "epoch": 0.024697710316439414,
+      "grad_norm": 2.018446207046509,
+      "learning_rate": 2.4742268041237115e-06,
+      "loss": 2.9874,
+      "step": 12
+    },
+    {
+      "epoch": 0.026755852842809364,
+      "grad_norm": 1.8558588027954102,
+      "learning_rate": 2.680412371134021e-06,
+      "loss": 2.8608,
+      "step": 13
+    },
+    {
+      "epoch": 0.028813995369179314,
+      "grad_norm": 1.9658265113830566,
+      "learning_rate": 2.8865979381443297e-06,
+      "loss": 2.8596,
+      "step": 14
+    },
+    {
+      "epoch": 0.030872137895549268,
+      "grad_norm": 1.872044563293457,
+      "learning_rate": 3.0927835051546395e-06,
+      "loss": 2.8836,
+      "step": 15
+    },
+    {
+      "epoch": 0.03293028042191922,
+      "grad_norm": 1.8884096145629883,
+      "learning_rate": 3.298969072164949e-06,
+      "loss": 2.9383,
+      "step": 16
+    },
+    {
+      "epoch": 0.03498842294828917,
+      "grad_norm": 1.8795744180679321,
+      "learning_rate": 3.5051546391752577e-06,
+      "loss": 2.883,
+      "step": 17
+    },
+    {
+      "epoch": 0.03704656547465912,
+      "grad_norm": 1.783678412437439,
+      "learning_rate": 3.7113402061855674e-06,
+      "loss": 2.8019,
+      "step": 18
+    },
+    {
+      "epoch": 0.039104708001029075,
+      "grad_norm": 1.820617914199829,
+      "learning_rate": 3.917525773195877e-06,
+      "loss": 2.8813,
+      "step": 19
+    },
+    {
+      "epoch": 0.04116285052739902,
+      "grad_norm": 1.8188731670379639,
+      "learning_rate": 4.123711340206186e-06,
+      "loss": 2.8401,
+      "step": 20
+    },
+    {
+      "epoch": 0.043220993053768975,
+      "grad_norm": 1.7305251359939575,
+      "learning_rate": 4.329896907216495e-06,
+      "loss": 2.7478,
+      "step": 21
+    },
+    {
+      "epoch": 0.04527913558013893,
+      "grad_norm": 1.7014551162719727,
+      "learning_rate": 4.536082474226804e-06,
+      "loss": 2.7356,
+      "step": 22
+    },
+    {
+      "epoch": 0.047337278106508875,
+      "grad_norm": 1.677381157875061,
+      "learning_rate": 4.742268041237113e-06,
+      "loss": 2.7593,
+      "step": 23
+    },
+    {
+      "epoch": 0.04939542063287883,
+      "grad_norm": 1.628554344177246,
+      "learning_rate": 4.948453608247423e-06,
+      "loss": 2.7689,
+      "step": 24
+    },
+    {
+      "epoch": 0.051453563159248775,
+      "grad_norm": 1.4968128204345703,
+      "learning_rate": 5.154639175257732e-06,
+      "loss": 2.6613,
+      "step": 25
+    },
+    {
+      "epoch": 0.05351170568561873,
+      "grad_norm": 1.4734832048416138,
+      "learning_rate": 5.360824742268042e-06,
+      "loss": 2.7095,
+      "step": 26
+    },
+    {
+      "epoch": 0.05556984821198868,
+      "grad_norm": 1.3745571374893188,
+      "learning_rate": 5.567010309278351e-06,
+      "loss": 2.655,
+      "step": 27
+    },
+    {
+      "epoch": 0.05762799073835863,
+      "grad_norm": 1.3381729125976562,
+      "learning_rate": 5.7731958762886594e-06,
+      "loss": 2.55,
+      "step": 28
+    },
+    {
+      "epoch": 0.05968613326472858,
+      "grad_norm": 1.3388073444366455,
+      "learning_rate": 5.979381443298969e-06,
+      "loss": 2.5219,
+      "step": 29
+    },
+    {
+      "epoch": 0.061744275791098535,
+      "grad_norm": 1.317008376121521,
+      "learning_rate": 6.185567010309279e-06,
+      "loss": 2.4491,
+      "step": 30
+    },
+    {
+      "epoch": 0.06380241831746848,
+      "grad_norm": 1.3210794925689697,
+      "learning_rate": 6.391752577319588e-06,
+      "loss": 2.4358,
+      "step": 31
+    },
+    {
+      "epoch": 0.06586056084383844,
+      "grad_norm": 1.182519555091858,
+      "learning_rate": 6.597938144329898e-06,
+      "loss": 2.4514,
+      "step": 32
+    },
+    {
+      "epoch": 0.06791870337020839,
+      "grad_norm": 1.2238099575042725,
+      "learning_rate": 6.804123711340207e-06,
+      "loss": 2.442,
+      "step": 33
+    },
+    {
+      "epoch": 0.06997684589657834,
+      "grad_norm": 1.1793314218521118,
+      "learning_rate": 7.010309278350515e-06,
+      "loss": 2.3864,
+      "step": 34
+    },
+    {
+      "epoch": 0.0720349884229483,
+      "grad_norm": 1.1983020305633545,
+      "learning_rate": 7.216494845360825e-06,
+      "loss": 2.3796,
+      "step": 35
+    },
+    {
+      "epoch": 0.07409313094931824,
+      "grad_norm": 1.2189652919769287,
+      "learning_rate": 7.422680412371135e-06,
+      "loss": 2.4152,
+      "step": 36
+    },
+    {
+      "epoch": 0.07615127347568819,
+      "grad_norm": 1.14923095703125,
+      "learning_rate": 7.628865979381444e-06,
+      "loss": 2.3298,
+      "step": 37
+    },
+    {
+      "epoch": 0.07820941600205815,
+      "grad_norm": 1.147013545036316,
+      "learning_rate": 7.835051546391754e-06,
+      "loss": 2.2488,
+      "step": 38
+    },
+    {
+      "epoch": 0.0802675585284281,
+      "grad_norm": 1.133981466293335,
+      "learning_rate": 8.041237113402063e-06,
+      "loss": 2.1825,
+      "step": 39
+    },
+    {
+      "epoch": 0.08232570105479804,
+      "grad_norm": 1.1686867475509644,
+      "learning_rate": 8.247422680412371e-06,
+      "loss": 2.2282,
+      "step": 40
+    },
+    {
+      "epoch": 0.084383843581168,
+      "grad_norm": 1.131690502166748,
+      "learning_rate": 8.453608247422681e-06,
+      "loss": 2.0962,
+      "step": 41
+    },
+    {
+      "epoch": 0.08644198610753795,
+      "grad_norm": 1.1626195907592773,
+      "learning_rate": 8.65979381443299e-06,
+      "loss": 2.1161,
+      "step": 42
+    },
+    {
+      "epoch": 0.0885001286339079,
+      "grad_norm": 1.1508581638336182,
+      "learning_rate": 8.865979381443299e-06,
+      "loss": 1.9856,
+      "step": 43
+    },
+    {
+      "epoch": 0.09055827116027786,
+      "grad_norm": 1.2286733388900757,
+      "learning_rate": 9.072164948453609e-06,
+      "loss": 2.076,
+      "step": 44
+    },
+    {
+      "epoch": 0.0926164136866478,
+      "grad_norm": 1.82068932056427,
+      "learning_rate": 9.278350515463918e-06,
+      "loss": 1.9995,
+      "step": 45
+    },
+    {
+      "epoch": 0.09467455621301775,
+      "grad_norm": 2.079101324081421,
+      "learning_rate": 9.484536082474226e-06,
+      "loss": 1.9601,
+      "step": 46
+    },
+    {
+      "epoch": 0.0967326987393877,
+      "grad_norm": 1.1209226846694946,
+      "learning_rate": 9.690721649484536e-06,
+      "loss": 1.9346,
+      "step": 47
+    },
+    {
+      "epoch": 0.09879084126575766,
+      "grad_norm": 1.0579711198806763,
+      "learning_rate": 9.896907216494846e-06,
+      "loss": 1.8764,
+      "step": 48
+    },
+    {
+      "epoch": 0.1008489837921276,
+      "grad_norm": 1.0434011220932007,
+      "learning_rate": 1.0103092783505156e-05,
+      "loss": 1.8483,
+      "step": 49
+    },
+    {
+      "epoch": 0.10290712631849755,
+      "grad_norm": 1.0089991092681885,
+      "learning_rate": 1.0309278350515464e-05,
+      "loss": 1.8018,
+      "step": 50
+    },
+    {
+      "epoch": 0.10496526884486751,
+      "grad_norm": 1.0117324590682983,
+      "learning_rate": 1.0515463917525775e-05,
+      "loss": 1.8003,
+      "step": 51
+    },
+    {
+      "epoch": 0.10702341137123746,
+      "grad_norm": 1.0006697177886963,
+      "learning_rate": 1.0721649484536083e-05,
+      "loss": 1.7482,
+      "step": 52
+    },
+    {
+      "epoch": 0.1090815538976074,
+      "grad_norm": 2.1164329051971436,
+      "learning_rate": 1.0927835051546391e-05,
+      "loss": 1.7363,
+      "step": 53
+    },
+    {
+      "epoch": 0.11113969642397736,
+      "grad_norm": 0.9573502540588379,
+      "learning_rate": 1.1134020618556703e-05,
+      "loss": 1.661,
+      "step": 54
+    },
+    {
+      "epoch": 0.11319783895034731,
+      "grad_norm": 1.0059764385223389,
+      "learning_rate": 1.134020618556701e-05,
+      "loss": 1.6979,
+      "step": 55
+    },
+    {
+      "epoch": 0.11525598147671726,
+      "grad_norm": 0.9719656109809875,
+      "learning_rate": 1.1546391752577319e-05,
+      "loss": 1.6318,
+      "step": 56
+    },
+    {
+      "epoch": 0.11731412400308722,
+      "grad_norm": 1.0024539232254028,
+      "learning_rate": 1.175257731958763e-05,
+      "loss": 1.6283,
+      "step": 57
+    },
+    {
+      "epoch": 0.11937226652945716,
+      "grad_norm": 0.9772456288337708,
+      "learning_rate": 1.1958762886597938e-05,
+      "loss": 1.5611,
+      "step": 58
+    },
+    {
+      "epoch": 0.12143040905582711,
+      "grad_norm": 0.9947625994682312,
+      "learning_rate": 1.2164948453608248e-05,
+      "loss": 1.6073,
+      "step": 59
+    },
+    {
+      "epoch": 0.12348855158219707,
+      "grad_norm": 2.112889051437378,
+      "learning_rate": 1.2371134020618558e-05,
+      "loss": 1.6208,
+      "step": 60
+    },
+    {
+      "epoch": 0.12554669410856703,
+      "grad_norm": 1.0515345335006714,
+      "learning_rate": 1.2577319587628866e-05,
+      "loss": 1.569,
+      "step": 61
+    },
+    {
+      "epoch": 0.12760483663493696,
+      "grad_norm": 1.0782145261764526,
+      "learning_rate": 1.2783505154639176e-05,
+      "loss": 1.5097,
+      "step": 62
+    },
+    {
+      "epoch": 0.12966297916130692,
+      "grad_norm": 1.154104232788086,
+      "learning_rate": 1.2989690721649485e-05,
+      "loss": 1.5472,
+      "step": 63
+    },
+    {
+      "epoch": 0.13172112168767688,
+      "grad_norm": 1.1614656448364258,
+      "learning_rate": 1.3195876288659795e-05,
+      "loss": 1.4833,
+      "step": 64
+    },
+    {
+      "epoch": 0.13377926421404682,
+      "grad_norm": 1.1720911264419556,
+      "learning_rate": 1.3402061855670103e-05,
+      "loss": 1.4644,
+      "step": 65
+    },
+    {
+      "epoch": 0.13583740674041678,
+      "grad_norm": 1.8903896808624268,
+      "learning_rate": 1.3608247422680415e-05,
+      "loss": 1.4286,
+      "step": 66
+    },
+    {
+      "epoch": 0.13789554926678674,
+      "grad_norm": 1.2675013542175293,
+      "learning_rate": 1.3814432989690723e-05,
+      "loss": 1.416,
+      "step": 67
+    },
+    {
+      "epoch": 0.13995369179315667,
+      "grad_norm": 1.266434907913208,
+      "learning_rate": 1.402061855670103e-05,
+      "loss": 1.3171,
+      "step": 68
+    },
+    {
+      "epoch": 0.14201183431952663,
+      "grad_norm": 1.3408889770507812,
+      "learning_rate": 1.4226804123711342e-05,
+      "loss": 1.3396,
+      "step": 69
+    },
+    {
+      "epoch": 0.1440699768458966,
+      "grad_norm": 1.3862446546554565,
+      "learning_rate": 1.443298969072165e-05,
+      "loss": 1.2642,
+      "step": 70
+    },
+    {
+      "epoch": 0.14612811937226652,
+      "grad_norm": 2.110553026199341,
+      "learning_rate": 1.4639175257731958e-05,
+      "loss": 1.2593,
+      "step": 71
+    },
+    {
+      "epoch": 0.14818626189863648,
+      "grad_norm": 1.7017499208450317,
+      "learning_rate": 1.484536082474227e-05,
+      "loss": 1.24,
+      "step": 72
+    },
+    {
+      "epoch": 0.15024440442500644,
+      "grad_norm": 1.9851700067520142,
+      "learning_rate": 1.5051546391752578e-05,
+      "loss": 1.2313,
+      "step": 73
+    },
+    {
+      "epoch": 0.15230254695137638,
+      "grad_norm": 2.009608030319214,
+      "learning_rate": 1.5257731958762888e-05,
+      "loss": 1.1281,
+      "step": 74
+    },
+    {
+      "epoch": 0.15436068947774634,
+      "grad_norm": 2.7587485313415527,
+      "learning_rate": 1.5463917525773197e-05,
+      "loss": 1.1248,
+      "step": 75
+    },
+    {
+      "epoch": 0.1564188320041163,
+      "grad_norm": 2.780954599380493,
+      "learning_rate": 1.5670103092783507e-05,
+      "loss": 1.0797,
+      "step": 76
+    },
+    {
+      "epoch": 0.15847697453048623,
+      "grad_norm": 3.1470866203308105,
+      "learning_rate": 1.5876288659793813e-05,
+      "loss": 1.0064,
+      "step": 77
+    },
+    {
+      "epoch": 0.1605351170568562,
+      "grad_norm": 4.653595447540283,
+      "learning_rate": 1.6082474226804127e-05,
+      "loss": 0.9219,
+      "step": 78
+    },
+    {
+      "epoch": 0.16259325958322615,
+      "grad_norm": 4.157363414764404,
+      "learning_rate": 1.6288659793814433e-05,
+      "loss": 0.8709,
+      "step": 79
+    },
+    {
+      "epoch": 0.16465140210959608,
+      "grad_norm": 4.5814924240112305,
+      "learning_rate": 1.6494845360824743e-05,
+      "loss": 0.7693,
+      "step": 80
+    },
+    {
+      "epoch": 0.16670954463596604,
+      "grad_norm": 5.096139907836914,
+      "learning_rate": 1.6701030927835052e-05,
+      "loss": 0.6868,
+      "step": 81
+    },
+    {
+      "epoch": 0.168767687162336,
+      "grad_norm": 4.858880519866943,
+      "learning_rate": 1.6907216494845362e-05,
+      "loss": 0.5971,
+      "step": 82
+    },
+    {
+      "epoch": 0.17082582968870594,
+      "grad_norm": 4.42564582824707,
+      "learning_rate": 1.7113402061855672e-05,
+      "loss": 0.4719,
+      "step": 83
+    },
+    {
+      "epoch": 0.1728839722150759,
+      "grad_norm": 7.720851421356201,
+      "learning_rate": 1.731958762886598e-05,
+      "loss": 0.3943,
+      "step": 84
+    },
+    {
+      "epoch": 0.17494211474144586,
+      "grad_norm": 0.41923192143440247,
+      "learning_rate": 1.752577319587629e-05,
+      "loss": 0.3635,
+      "step": 85
+    },
+    {
+      "epoch": 0.1770002572678158,
+      "grad_norm": 0.2771846354007721,
+      "learning_rate": 1.7731958762886598e-05,
+      "loss": 0.3597,
+      "step": 86
+    },
+    {
+      "epoch": 0.17905839979418575,
+      "grad_norm": 0.24761857092380524,
+      "learning_rate": 1.793814432989691e-05,
+      "loss": 0.3735,
+      "step": 87
+    },
+    {
+      "epoch": 0.1811165423205557,
+      "grad_norm": 0.23277048766613007,
+      "learning_rate": 1.8144329896907217e-05,
+      "loss": 0.3643,
+      "step": 88
+    },
+    {
+      "epoch": 0.18317468484692565,
+      "grad_norm": 0.22931228578090668,
+      "learning_rate": 1.8350515463917527e-05,
+      "loss": 0.3519,
+      "step": 89
+    },
+    {
+      "epoch": 0.1852328273732956,
+      "grad_norm": 0.20750615000724792,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 0.3431,
+      "step": 90
+    },
+    {
+      "epoch": 0.18729096989966554,
+      "grad_norm": 0.2080322951078415,
+      "learning_rate": 1.8762886597938147e-05,
+      "loss": 0.3632,
+      "step": 91
+    },
+    {
+      "epoch": 0.1893491124260355,
+      "grad_norm": 0.20186181366443634,
+      "learning_rate": 1.8969072164948453e-05,
+      "loss": 0.3492,
+      "step": 92
+    },
+    {
+      "epoch": 0.19140725495240546,
+      "grad_norm": 0.19172786176204681,
+      "learning_rate": 1.9175257731958766e-05,
+      "loss": 0.3552,
+      "step": 93
+    },
+    {
+      "epoch": 0.1934653974787754,
+      "grad_norm": 0.1747850626707077,
+      "learning_rate": 1.9381443298969072e-05,
+      "loss": 0.3355,
+      "step": 94
+    },
+    {
+      "epoch": 0.19552354000514535,
+      "grad_norm": 0.196411594748497,
+      "learning_rate": 1.9587628865979382e-05,
+      "loss": 0.3271,
+      "step": 95
+    },
+    {
+      "epoch": 0.1975816825315153,
+      "grad_norm": 0.20063228905200958,
+      "learning_rate": 1.9793814432989692e-05,
+      "loss": 0.3351,
+      "step": 96
+    },
+    {
+      "epoch": 0.19963982505788525,
+      "grad_norm": 0.19240939617156982,
+      "learning_rate": 2e-05,
+      "loss": 0.3266,
+      "step": 97
+    },
+    {
+      "epoch": 0.2016979675842552,
+      "grad_norm": 0.18206572532653809,
+      "learning_rate": 1.997709049255441e-05,
+      "loss": 0.3393,
+      "step": 98
+    },
+    {
+      "epoch": 0.20375611011062517,
+      "grad_norm": 0.20384562015533447,
+      "learning_rate": 1.9954180985108823e-05,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "grad_norm": 0.19944581389427185,
+      "learning_rate": 1.9931271477663232e-05,
+      "loss": 0.3268,
+      "step": 100
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "eval_loss": 0.3456890285015106,
+      "eval_runtime": 2114.0178,
+      "eval_samples_per_second": 3.677,
+      "eval_steps_per_second": 0.92,
+      "step": 100
+    },
+    {
+      "epoch": 0.20787239516336506,
+      "grad_norm": 0.17743557691574097,
+      "learning_rate": 1.990836197021764e-05,
+      "loss": 0.3439,
+      "step": 101
+    },
+    {
+      "epoch": 0.20993053768973502,
+      "grad_norm": 0.18746449053287506,
+      "learning_rate": 1.9885452462772053e-05,
+      "loss": 0.326,
+      "step": 102
+    },
+    {
+      "epoch": 0.21198868021610495,
+      "grad_norm": 0.18555815517902374,
+      "learning_rate": 1.9862542955326462e-05,
+      "loss": 0.3337,
+      "step": 103
+    },
+    {
+      "epoch": 0.2140468227424749,
+      "grad_norm": 0.16591575741767883,
+      "learning_rate": 1.9839633447880874e-05,
+      "loss": 0.3121,
+      "step": 104
+    },
+    {
+      "epoch": 0.21610496526884487,
+      "grad_norm": 0.1621987372636795,
+      "learning_rate": 1.9816723940435283e-05,
+      "loss": 0.3287,
+      "step": 105
+    },
+    {
+      "epoch": 0.2181631077952148,
+      "grad_norm": 0.1614532470703125,
+      "learning_rate": 1.9793814432989692e-05,
+      "loss": 0.3306,
+      "step": 106
+    },
+    {
+      "epoch": 0.22022125032158477,
+      "grad_norm": 0.17993387579917908,
+      "learning_rate": 1.9770904925544104e-05,
+      "loss": 0.3341,
+      "step": 107
+    },
+    {
+      "epoch": 0.22227939284795473,
+      "grad_norm": 0.1550011783838272,
+      "learning_rate": 1.9747995418098513e-05,
+      "loss": 0.3197,
+      "step": 108
+    },
+    {
+      "epoch": 0.22433753537432466,
+      "grad_norm": 0.18471524119377136,
+      "learning_rate": 1.9725085910652922e-05,
+      "loss": 0.3285,
+      "step": 109
+    },
+    {
+      "epoch": 0.22639567790069462,
+      "grad_norm": 0.15604373812675476,
+      "learning_rate": 1.9702176403207334e-05,
+      "loss": 0.3298,
+      "step": 110
+    },
+    {
+      "epoch": 0.22845382042706458,
+      "grad_norm": 0.1682298630475998,
+      "learning_rate": 1.9679266895761743e-05,
+      "loss": 0.3343,
+      "step": 111
+    },
+    {
+      "epoch": 0.2305119629534345,
+      "grad_norm": 0.14933635294437408,
+      "learning_rate": 1.9656357388316152e-05,
+      "loss": 0.3134,
+      "step": 112
+    },
+    {
+      "epoch": 0.23257010547980447,
+      "grad_norm": 0.14892347157001495,
+      "learning_rate": 1.963344788087056e-05,
+      "loss": 0.3154,
+      "step": 113
+    },
+    {
+      "epoch": 0.23462824800617443,
+      "grad_norm": 0.1577889323234558,
+      "learning_rate": 1.9610538373424973e-05,
+      "loss": 0.3122,
+      "step": 114
+    },
+    {
+      "epoch": 0.23668639053254437,
+      "grad_norm": 0.16482344269752502,
+      "learning_rate": 1.9587628865979382e-05,
+      "loss": 0.3193,
+      "step": 115
+    },
+    {
+      "epoch": 0.23874453305891433,
+      "grad_norm": 0.15328913927078247,
+      "learning_rate": 1.956471935853379e-05,
+      "loss": 0.3217,
+      "step": 116
+    },
+    {
+      "epoch": 0.2408026755852843,
+      "grad_norm": 0.16140656173229218,
+      "learning_rate": 1.9541809851088203e-05,
+      "loss": 0.318,
+      "step": 117
+    },
+    {
+      "epoch": 0.24286081811165422,
+      "grad_norm": 0.15448373556137085,
+      "learning_rate": 1.9518900343642612e-05,
+      "loss": 0.3205,
+      "step": 118
+    },
+    {
+      "epoch": 0.24491896063802418,
+      "grad_norm": 0.14716887474060059,
+      "learning_rate": 1.9495990836197025e-05,
+      "loss": 0.3164,
+      "step": 119
+    },
+    {
+      "epoch": 0.24697710316439414,
+      "grad_norm": 0.16582027077674866,
+      "learning_rate": 1.9473081328751433e-05,
+      "loss": 0.3191,
+      "step": 120
+    },
+    {
+      "epoch": 0.24903524569076407,
+      "grad_norm": 0.15213699638843536,
+      "learning_rate": 1.9450171821305842e-05,
+      "loss": 0.304,
+      "step": 121
+    },
+    {
+      "epoch": 0.25109338821713406,
+      "grad_norm": 0.1659238487482071,
+      "learning_rate": 1.9427262313860255e-05,
+      "loss": 0.3184,
+      "step": 122
+    },
+    {
+      "epoch": 0.253151530743504,
+      "grad_norm": 0.15596656501293182,
+      "learning_rate": 1.9404352806414663e-05,
+      "loss": 0.3092,
+      "step": 123
+    },
+    {
+      "epoch": 0.2552096732698739,
+      "grad_norm": 0.15868476033210754,
+      "learning_rate": 1.9381443298969072e-05,
+      "loss": 0.3163,
+      "step": 124
+    },
+    {
+      "epoch": 0.2572678157962439,
+      "grad_norm": 0.15386095643043518,
+      "learning_rate": 1.9358533791523485e-05,
+      "loss": 0.3049,
+      "step": 125
+    },
+    {
+      "epoch": 0.25932595832261385,
+      "grad_norm": 0.15179213881492615,
+      "learning_rate": 1.9335624284077894e-05,
+      "loss": 0.3131,
+      "step": 126
+    },
+    {
+      "epoch": 0.2613841008489838,
+      "grad_norm": 0.1595134735107422,
+      "learning_rate": 1.9312714776632306e-05,
+      "loss": 0.3069,
+      "step": 127
+    },
+    {
+      "epoch": 0.26344224337535377,
+      "grad_norm": 0.16989803314208984,
+      "learning_rate": 1.9289805269186715e-05,
+      "loss": 0.3052,
+      "step": 128
+    },
+    {
+      "epoch": 0.2655003859017237,
+      "grad_norm": 0.14803892374038696,
+      "learning_rate": 1.9266895761741124e-05,
+      "loss": 0.3065,
+      "step": 129
+    },
+    {
+      "epoch": 0.26755852842809363,
+      "grad_norm": 0.16676583886146545,
+      "learning_rate": 1.9243986254295536e-05,
+      "loss": 0.2962,
+      "step": 130
+    },
+    {
+      "epoch": 0.2696166709544636,
+      "grad_norm": 0.15694552659988403,
+      "learning_rate": 1.9221076746849945e-05,
+      "loss": 0.3096,
+      "step": 131
+    },
+    {
+      "epoch": 0.27167481348083355,
+      "grad_norm": 0.17696696519851685,
+      "learning_rate": 1.9198167239404354e-05,
+      "loss": 0.3145,
+      "step": 132
+    },
+    {
+      "epoch": 0.2737329560072035,
+      "grad_norm": 0.17204038798809052,
+      "learning_rate": 1.9175257731958766e-05,
+      "loss": 0.3248,
+      "step": 133
+    },
+    {
+      "epoch": 0.2757910985335735,
+      "grad_norm": 0.15630359947681427,
+      "learning_rate": 1.9152348224513175e-05,
+      "loss": 0.3117,
+      "step": 134
+    },
+    {
+      "epoch": 0.2778492410599434,
+      "grad_norm": 0.15757997334003448,
+      "learning_rate": 1.9129438717067584e-05,
+      "loss": 0.3145,
+      "step": 135
+    },
+    {
+      "epoch": 0.27990738358631334,
+      "grad_norm": 0.16273653507232666,
+      "learning_rate": 1.9106529209621996e-05,
+      "loss": 0.3159,
+      "step": 136
+    },
+    {
+      "epoch": 0.28196552611268333,
+      "grad_norm": 0.16213104128837585,
+      "learning_rate": 1.9083619702176405e-05,
+      "loss": 0.2949,
+      "step": 137
+    },
+    {
+      "epoch": 0.28402366863905326,
+      "grad_norm": 0.15377865731716156,
+      "learning_rate": 1.9060710194730814e-05,
+      "loss": 0.306,
+      "step": 138
+    },
+    {
+      "epoch": 0.2860818111654232,
+      "grad_norm": 0.1545962244272232,
+      "learning_rate": 1.9037800687285223e-05,
+      "loss": 0.2966,
+      "step": 139
+    },
+    {
+      "epoch": 0.2881399536917932,
+      "grad_norm": 0.15516617894172668,
+      "learning_rate": 1.9014891179839635e-05,
+      "loss": 0.3122,
+      "step": 140
+    },
+    {
+      "epoch": 0.2901980962181631,
+      "grad_norm": 0.14734458923339844,
+      "learning_rate": 1.8991981672394044e-05,
+      "loss": 0.3118,
+      "step": 141
+    },
+    {
+      "epoch": 0.29225623874453305,
+      "grad_norm": 0.1644304096698761,
+      "learning_rate": 1.8969072164948453e-05,
+      "loss": 0.3027,
+      "step": 142
+    },
+    {
+      "epoch": 0.29431438127090304,
+      "grad_norm": 0.14632569253444672,
+      "learning_rate": 1.8946162657502865e-05,
+      "loss": 0.3023,
+      "step": 143
+    },
+    {
+      "epoch": 0.29637252379727297,
+      "grad_norm": 0.1573137789964676,
+      "learning_rate": 1.8923253150057274e-05,
+      "loss": 0.3102,
+      "step": 144
+    },
+    {
+      "epoch": 0.2984306663236429,
+      "grad_norm": 0.16423144936561584,
+      "learning_rate": 1.8900343642611686e-05,
+      "loss": 0.3033,
+      "step": 145
+    },
+    {
+      "epoch": 0.3004888088500129,
+      "grad_norm": 0.15420907735824585,
+      "learning_rate": 1.8877434135166095e-05,
+      "loss": 0.3089,
+      "step": 146
+    },
+    {
+      "epoch": 0.3025469513763828,
+      "grad_norm": 0.1579178273677826,
+      "learning_rate": 1.8854524627720504e-05,
+      "loss": 0.3071,
+      "step": 147
+    },
+    {
+      "epoch": 0.30460509390275275,
+      "grad_norm": 0.15866397321224213,
+      "learning_rate": 1.8831615120274916e-05,
+      "loss": 0.3083,
+      "step": 148
+    },
+    {
+      "epoch": 0.30666323642912274,
+      "grad_norm": 0.16651487350463867,
+      "learning_rate": 1.8808705612829325e-05,
+      "loss": 0.3099,
+      "step": 149
+    },
+    {
+      "epoch": 0.3087213789554927,
+      "grad_norm": 0.16281908750534058,
+      "learning_rate": 1.8785796105383734e-05,
+      "loss": 0.3034,
+      "step": 150
+    },
+    {
+      "epoch": 0.3107795214818626,
+      "grad_norm": 0.17449837923049927,
+      "learning_rate": 1.8762886597938147e-05,
+      "loss": 0.3054,
+      "step": 151
+    },
+    {
+      "epoch": 0.3128376640082326,
+      "grad_norm": 0.15403546392917633,
+      "learning_rate": 1.8739977090492555e-05,
+      "loss": 0.297,
+      "step": 152
+    },
+    {
+      "epoch": 0.31489580653460253,
+      "grad_norm": 0.1472466140985489,
+      "learning_rate": 1.8717067583046968e-05,
+      "loss": 0.2973,
+      "step": 153
+    },
+    {
+      "epoch": 0.31695394906097246,
+      "grad_norm": 0.16027937829494476,
+      "learning_rate": 1.8694158075601377e-05,
+      "loss": 0.3054,
+      "step": 154
+    },
+    {
+      "epoch": 0.31901209158734245,
+      "grad_norm": 0.17086225748062134,
+      "learning_rate": 1.8671248568155786e-05,
+      "loss": 0.307,
+      "step": 155
+    },
+    {
+      "epoch": 0.3210702341137124,
+      "grad_norm": 0.15930697321891785,
+      "learning_rate": 1.8648339060710198e-05,
+      "loss": 0.293,
+      "step": 156
+    },
+    {
+      "epoch": 0.3231283766400823,
+      "grad_norm": 0.17086376249790192,
+      "learning_rate": 1.8625429553264607e-05,
+      "loss": 0.293,
+      "step": 157
+    },
+    {
+      "epoch": 0.3251865191664523,
+      "grad_norm": 0.15970875322818756,
+      "learning_rate": 1.8602520045819016e-05,
+      "loss": 0.3083,
+      "step": 158
+    },
+    {
+      "epoch": 0.32724466169282224,
+      "grad_norm": 0.16355909407138824,
+      "learning_rate": 1.8579610538373428e-05,
+      "loss": 0.3139,
+      "step": 159
+    },
+    {
+      "epoch": 0.32930280421919217,
+      "grad_norm": 0.15183711051940918,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 0.2953,
+      "step": 160
+    },
+    {
+      "epoch": 0.33136094674556216,
+      "grad_norm": 0.15123715996742249,
+      "learning_rate": 1.853379152348225e-05,
+      "loss": 0.3025,
+      "step": 161
+    },
+    {
+      "epoch": 0.3334190892719321,
+      "grad_norm": 0.1576143503189087,
+      "learning_rate": 1.8510882016036658e-05,
+      "loss": 0.2904,
+      "step": 162
+    },
+    {
+      "epoch": 0.335477231798302,
+      "grad_norm": 0.1457504779100418,
+      "learning_rate": 1.8487972508591067e-05,
+      "loss": 0.2909,
+      "step": 163
+    },
+    {
+      "epoch": 0.337535374324672,
+      "grad_norm": 0.1557442992925644,
+      "learning_rate": 1.846506300114548e-05,
+      "loss": 0.3027,
+      "step": 164
+    },
+    {
+      "epoch": 0.33959351685104194,
+      "grad_norm": 0.15662318468093872,
+      "learning_rate": 1.8442153493699888e-05,
+      "loss": 0.311,
+      "step": 165
+    },
+    {
+      "epoch": 0.3416516593774119,
+      "grad_norm": 0.16177058219909668,
+      "learning_rate": 1.8419243986254297e-05,
+      "loss": 0.2944,
+      "step": 166
+    },
+    {
+      "epoch": 0.34370980190378186,
+      "grad_norm": 0.16406729817390442,
+      "learning_rate": 1.8396334478808706e-05,
+      "loss": 0.2927,
+      "step": 167
+    },
+    {
+      "epoch": 0.3457679444301518,
+      "grad_norm": 0.16642791032791138,
+      "learning_rate": 1.8373424971363115e-05,
+      "loss": 0.3063,
+      "step": 168
+    },
+    {
+      "epoch": 0.34782608695652173,
+      "grad_norm": 0.1650693714618683,
+      "learning_rate": 1.8350515463917527e-05,
+      "loss": 0.2957,
+      "step": 169
+    },
+    {
+      "epoch": 0.3498842294828917,
+      "grad_norm": 0.15349675714969635,
+      "learning_rate": 1.8327605956471936e-05,
+      "loss": 0.297,
+      "step": 170
+    },
+    {
+      "epoch": 0.35194237200926165,
+      "grad_norm": 0.17770209908485413,
+      "learning_rate": 1.8304696449026348e-05,
+      "loss": 0.3011,
+      "step": 171
+    },
+    {
+      "epoch": 0.3540005145356316,
+      "grad_norm": 0.1647631675004959,
+      "learning_rate": 1.8281786941580757e-05,
+      "loss": 0.2962,
+      "step": 172
+    },
+    {
+      "epoch": 0.35605865706200157,
+      "grad_norm": 0.1603834480047226,
+      "learning_rate": 1.8258877434135166e-05,
+      "loss": 0.2937,
+      "step": 173
+    },
+    {
+      "epoch": 0.3581167995883715,
+      "grad_norm": 0.16780880093574524,
+      "learning_rate": 1.8235967926689578e-05,
+      "loss": 0.2997,
+      "step": 174
+    },
+    {
+      "epoch": 0.36017494211474144,
+      "grad_norm": 0.15976767241954803,
+      "learning_rate": 1.8213058419243987e-05,
+      "loss": 0.3043,
+      "step": 175
+    },
+    {
+      "epoch": 0.3622330846411114,
+      "grad_norm": 0.16236485540866852,
+      "learning_rate": 1.8190148911798396e-05,
+      "loss": 0.3069,
+      "step": 176
+    },
+    {
+      "epoch": 0.36429122716748136,
+      "grad_norm": 0.16391968727111816,
+      "learning_rate": 1.816723940435281e-05,
+      "loss": 0.2923,
+      "step": 177
+    },
+    {
+      "epoch": 0.3663493696938513,
+      "grad_norm": 0.15806889533996582,
+      "learning_rate": 1.8144329896907217e-05,
+      "loss": 0.2872,
+      "step": 178
+    },
+    {
+      "epoch": 0.3684075122202212,
+      "grad_norm": 0.1627352088689804,
+      "learning_rate": 1.812142038946163e-05,
+      "loss": 0.3032,
+      "step": 179
+    },
+    {
+      "epoch": 0.3704656547465912,
+      "grad_norm": 0.15103371441364288,
+      "learning_rate": 1.809851088201604e-05,
+      "loss": 0.2847,
+      "step": 180
+    },
+    {
+      "epoch": 0.37252379727296114,
+      "grad_norm": 0.15178488194942474,
+      "learning_rate": 1.8075601374570447e-05,
+      "loss": 0.3017,
+      "step": 181
+    },
+    {
+      "epoch": 0.3745819397993311,
+      "grad_norm": 0.15493899583816528,
+      "learning_rate": 1.805269186712486e-05,
+      "loss": 0.2901,
+      "step": 182
+    },
+    {
+      "epoch": 0.37664008232570106,
+      "grad_norm": 0.15990686416625977,
+      "learning_rate": 1.802978235967927e-05,
+      "loss": 0.2861,
+      "step": 183
+    },
+    {
+      "epoch": 0.378698224852071,
+      "grad_norm": 0.15824148058891296,
+      "learning_rate": 1.8006872852233677e-05,
+      "loss": 0.2885,
+      "step": 184
+    },
+    {
+      "epoch": 0.38075636737844093,
+      "grad_norm": 0.15690775215625763,
+      "learning_rate": 1.798396334478809e-05,
+      "loss": 0.2814,
+      "step": 185
+    },
+    {
+      "epoch": 0.3828145099048109,
+      "grad_norm": 0.15833796560764313,
+      "learning_rate": 1.79610538373425e-05,
+      "loss": 0.2847,
+      "step": 186
+    },
+    {
+      "epoch": 0.38487265243118085,
+      "grad_norm": 0.16560044884681702,
+      "learning_rate": 1.793814432989691e-05,
+      "loss": 0.3061,
+      "step": 187
+    },
+    {
+      "epoch": 0.3869307949575508,
+      "grad_norm": 0.16240179538726807,
+      "learning_rate": 1.791523482245132e-05,
+      "loss": 0.2943,
+      "step": 188
+    },
+    {
+      "epoch": 0.38898893748392077,
+      "grad_norm": 0.15825721621513367,
+      "learning_rate": 1.789232531500573e-05,
+      "loss": 0.2934,
+      "step": 189
+    },
+    {
+      "epoch": 0.3910470800102907,
+      "grad_norm": 0.16665388643741608,
+      "learning_rate": 1.786941580756014e-05,
+      "loss": 0.291,
+      "step": 190
+    },
+    {
+      "epoch": 0.39310522253666064,
+      "grad_norm": 0.16581200063228607,
+      "learning_rate": 1.784650630011455e-05,
+      "loss": 0.2849,
+      "step": 191
+    },
+    {
+      "epoch": 0.3951633650630306,
+      "grad_norm": 0.1604345291852951,
+      "learning_rate": 1.782359679266896e-05,
+      "loss": 0.3,
+      "step": 192
+    },
+    {
+      "epoch": 0.39722150758940056,
+      "grad_norm": 0.16107915341854095,
+      "learning_rate": 1.7800687285223368e-05,
+      "loss": 0.2847,
+      "step": 193
+    },
+    {
+      "epoch": 0.3992796501157705,
+      "grad_norm": 0.1571730375289917,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 0.2863,
+      "step": 194
+    },
+    {
+      "epoch": 0.4013377926421405,
+      "grad_norm": 0.1656399518251419,
+      "learning_rate": 1.775486827033219e-05,
+      "loss": 0.2878,
+      "step": 195
+    },
+    {
+      "epoch": 0.4033959351685104,
+      "grad_norm": 0.16738460958003998,
+      "learning_rate": 1.7731958762886598e-05,
+      "loss": 0.286,
+      "step": 196
+    },
+    {
+      "epoch": 0.40545407769488034,
+      "grad_norm": 0.16704292595386505,
+      "learning_rate": 1.770904925544101e-05,
+      "loss": 0.2919,
+      "step": 197
+    },
+    {
+      "epoch": 0.40751222022125033,
+      "grad_norm": 0.16215579211711884,
+      "learning_rate": 1.768613974799542e-05,
+      "loss": 0.2874,
+      "step": 198
+    },
+    {
+      "epoch": 0.40957036274762026,
+      "grad_norm": 0.15573479235172272,
+      "learning_rate": 1.7663230240549828e-05,
+      "loss": 0.2904,
+      "step": 199
+    },
+    {
+      "epoch": 0.4116285052739902,
+      "grad_norm": 0.1707623153924942,
+      "learning_rate": 1.764032073310424e-05,
+      "loss": 0.289,
+      "step": 200
+    },
+    {
+      "epoch": 0.4116285052739902,
+      "eval_loss": 0.3214050829410553,
+      "eval_runtime": 2449.7742,
+      "eval_samples_per_second": 3.173,
+      "eval_steps_per_second": 0.794,
+      "step": 200
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 970,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.615055995939963e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee55a11f5a306eb7c39b536fdfe2459bc279e468da50f6adda478c4deffcb812
+size 5688

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/codegemma-7b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/codegemma-7b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4904f256bc0d01bcdb1e0e13e3ffde750efe2e272e6acbcc661481ab3b7afccd
+size 800116456

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c65cc37f7b35b81a9c19fbda436234ae27b3e7ffe674a3420cf814c7f38a722b
+size 406743860

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b36f846e18f654c3cfee105cea70c9e74df15bfcb4fc79f4b82ed01c58b83ae3
+size 14244

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c92157e090687c4e3d6cd57a32a0d42e6f559fd56b8e5771861e40841b6ea7c3
+size 1064

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d964a2c8346d40f95791533eae48730d5f163c2e65fd16333560fd3e661df318
+size 34362915

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:583f2ebd2a1936009b7da991ea255504db68c7a9713a78673d1335a87098966c
+size 4241023

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/tokenizer_config.json

@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<|file_separator|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/trainer_state.json

@@ -0,0 +1,2158 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.6174427579109854,
+  "eval_steps": 100,
+  "global_step": 300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0020581425263699513,
+      "grad_norm": 11.994463920593262,
+      "learning_rate": 2.061855670103093e-07,
+      "loss": 2.91,
+      "step": 1
+    },
+    {
+      "epoch": 0.004116285052739903,
+      "grad_norm": 11.769092559814453,
+      "learning_rate": 4.123711340206186e-07,
+      "loss": 2.8686,
+      "step": 2
+    },
+    {
+      "epoch": 0.0061744275791098535,
+      "grad_norm": 13.05551815032959,
+      "learning_rate": 6.185567010309279e-07,
+      "loss": 3.0286,
+      "step": 3
+    },
+    {
+      "epoch": 0.008232570105479805,
+      "grad_norm": 12.334521293640137,
+      "learning_rate": 8.247422680412372e-07,
+      "loss": 2.904,
+      "step": 4
+    },
+    {
+      "epoch": 0.010290712631849755,
+      "grad_norm": 12.075353622436523,
+      "learning_rate": 1.0309278350515464e-06,
+      "loss": 2.8991,
+      "step": 5
+    },
+    {
+      "epoch": 0.012348855158219707,
+      "grad_norm": 11.86032485961914,
+      "learning_rate": 1.2371134020618557e-06,
+      "loss": 3.0007,
+      "step": 6
+    },
+    {
+      "epoch": 0.014406997684589657,
+      "grad_norm": 10.10457992553711,
+      "learning_rate": 1.4432989690721649e-06,
+      "loss": 2.8493,
+      "step": 7
+    },
+    {
+      "epoch": 0.01646514021095961,
+      "grad_norm": 8.56408405303955,
+      "learning_rate": 1.6494845360824744e-06,
+      "loss": 2.9573,
+      "step": 8
+    },
+    {
+      "epoch": 0.01852328273732956,
+      "grad_norm": 6.307392120361328,
+      "learning_rate": 1.8556701030927837e-06,
+      "loss": 2.9507,
+      "step": 9
+    },
+    {
+      "epoch": 0.02058142526369951,
+      "grad_norm": 4.276430130004883,
+      "learning_rate": 2.061855670103093e-06,
+      "loss": 2.8988,
+      "step": 10
+    },
+    {
+      "epoch": 0.022639567790069464,
+      "grad_norm": 2.5912015438079834,
+      "learning_rate": 2.268041237113402e-06,
+      "loss": 2.9926,
+      "step": 11
+    },
+    {
+      "epoch": 0.024697710316439414,
+      "grad_norm": 2.018446207046509,
+      "learning_rate": 2.4742268041237115e-06,
+      "loss": 2.9874,
+      "step": 12
+    },
+    {
+      "epoch": 0.026755852842809364,
+      "grad_norm": 1.8558588027954102,
+      "learning_rate": 2.680412371134021e-06,
+      "loss": 2.8608,
+      "step": 13
+    },
+    {
+      "epoch": 0.028813995369179314,
+      "grad_norm": 1.9658265113830566,
+      "learning_rate": 2.8865979381443297e-06,
+      "loss": 2.8596,
+      "step": 14
+    },
+    {
+      "epoch": 0.030872137895549268,
+      "grad_norm": 1.872044563293457,
+      "learning_rate": 3.0927835051546395e-06,
+      "loss": 2.8836,
+      "step": 15
+    },
+    {
+      "epoch": 0.03293028042191922,
+      "grad_norm": 1.8884096145629883,
+      "learning_rate": 3.298969072164949e-06,
+      "loss": 2.9383,
+      "step": 16
+    },
+    {
+      "epoch": 0.03498842294828917,
+      "grad_norm": 1.8795744180679321,
+      "learning_rate": 3.5051546391752577e-06,
+      "loss": 2.883,
+      "step": 17
+    },
+    {
+      "epoch": 0.03704656547465912,
+      "grad_norm": 1.783678412437439,
+      "learning_rate": 3.7113402061855674e-06,
+      "loss": 2.8019,
+      "step": 18
+    },
+    {
+      "epoch": 0.039104708001029075,
+      "grad_norm": 1.820617914199829,
+      "learning_rate": 3.917525773195877e-06,
+      "loss": 2.8813,
+      "step": 19
+    },
+    {
+      "epoch": 0.04116285052739902,
+      "grad_norm": 1.8188731670379639,
+      "learning_rate": 4.123711340206186e-06,
+      "loss": 2.8401,
+      "step": 20
+    },
+    {
+      "epoch": 0.043220993053768975,
+      "grad_norm": 1.7305251359939575,
+      "learning_rate": 4.329896907216495e-06,
+      "loss": 2.7478,
+      "step": 21
+    },
+    {
+      "epoch": 0.04527913558013893,
+      "grad_norm": 1.7014551162719727,
+      "learning_rate": 4.536082474226804e-06,
+      "loss": 2.7356,
+      "step": 22
+    },
+    {
+      "epoch": 0.047337278106508875,
+      "grad_norm": 1.677381157875061,
+      "learning_rate": 4.742268041237113e-06,
+      "loss": 2.7593,
+      "step": 23
+    },
+    {
+      "epoch": 0.04939542063287883,
+      "grad_norm": 1.628554344177246,
+      "learning_rate": 4.948453608247423e-06,
+      "loss": 2.7689,
+      "step": 24
+    },
+    {
+      "epoch": 0.051453563159248775,
+      "grad_norm": 1.4968128204345703,
+      "learning_rate": 5.154639175257732e-06,
+      "loss": 2.6613,
+      "step": 25
+    },
+    {
+      "epoch": 0.05351170568561873,
+      "grad_norm": 1.4734832048416138,
+      "learning_rate": 5.360824742268042e-06,
+      "loss": 2.7095,
+      "step": 26
+    },
+    {
+      "epoch": 0.05556984821198868,
+      "grad_norm": 1.3745571374893188,
+      "learning_rate": 5.567010309278351e-06,
+      "loss": 2.655,
+      "step": 27
+    },
+    {
+      "epoch": 0.05762799073835863,
+      "grad_norm": 1.3381729125976562,
+      "learning_rate": 5.7731958762886594e-06,
+      "loss": 2.55,
+      "step": 28
+    },
+    {
+      "epoch": 0.05968613326472858,
+      "grad_norm": 1.3388073444366455,
+      "learning_rate": 5.979381443298969e-06,
+      "loss": 2.5219,
+      "step": 29
+    },
+    {
+      "epoch": 0.061744275791098535,
+      "grad_norm": 1.317008376121521,
+      "learning_rate": 6.185567010309279e-06,
+      "loss": 2.4491,
+      "step": 30
+    },
+    {
+      "epoch": 0.06380241831746848,
+      "grad_norm": 1.3210794925689697,
+      "learning_rate": 6.391752577319588e-06,
+      "loss": 2.4358,
+      "step": 31
+    },
+    {
+      "epoch": 0.06586056084383844,
+      "grad_norm": 1.182519555091858,
+      "learning_rate": 6.597938144329898e-06,
+      "loss": 2.4514,
+      "step": 32
+    },
+    {
+      "epoch": 0.06791870337020839,
+      "grad_norm": 1.2238099575042725,
+      "learning_rate": 6.804123711340207e-06,
+      "loss": 2.442,
+      "step": 33
+    },
+    {
+      "epoch": 0.06997684589657834,
+      "grad_norm": 1.1793314218521118,
+      "learning_rate": 7.010309278350515e-06,
+      "loss": 2.3864,
+      "step": 34
+    },
+    {
+      "epoch": 0.0720349884229483,
+      "grad_norm": 1.1983020305633545,
+      "learning_rate": 7.216494845360825e-06,
+      "loss": 2.3796,
+      "step": 35
+    },
+    {
+      "epoch": 0.07409313094931824,
+      "grad_norm": 1.2189652919769287,
+      "learning_rate": 7.422680412371135e-06,
+      "loss": 2.4152,
+      "step": 36
+    },
+    {
+      "epoch": 0.07615127347568819,
+      "grad_norm": 1.14923095703125,
+      "learning_rate": 7.628865979381444e-06,
+      "loss": 2.3298,
+      "step": 37
+    },
+    {
+      "epoch": 0.07820941600205815,
+      "grad_norm": 1.147013545036316,
+      "learning_rate": 7.835051546391754e-06,
+      "loss": 2.2488,
+      "step": 38
+    },
+    {
+      "epoch": 0.0802675585284281,
+      "grad_norm": 1.133981466293335,
+      "learning_rate": 8.041237113402063e-06,
+      "loss": 2.1825,
+      "step": 39
+    },
+    {
+      "epoch": 0.08232570105479804,
+      "grad_norm": 1.1686867475509644,
+      "learning_rate": 8.247422680412371e-06,
+      "loss": 2.2282,
+      "step": 40
+    },
+    {
+      "epoch": 0.084383843581168,
+      "grad_norm": 1.131690502166748,
+      "learning_rate": 8.453608247422681e-06,
+      "loss": 2.0962,
+      "step": 41
+    },
+    {
+      "epoch": 0.08644198610753795,
+      "grad_norm": 1.1626195907592773,
+      "learning_rate": 8.65979381443299e-06,
+      "loss": 2.1161,
+      "step": 42
+    },
+    {
+      "epoch": 0.0885001286339079,
+      "grad_norm": 1.1508581638336182,
+      "learning_rate": 8.865979381443299e-06,
+      "loss": 1.9856,
+      "step": 43
+    },
+    {
+      "epoch": 0.09055827116027786,
+      "grad_norm": 1.2286733388900757,
+      "learning_rate": 9.072164948453609e-06,
+      "loss": 2.076,
+      "step": 44
+    },
+    {
+      "epoch": 0.0926164136866478,
+      "grad_norm": 1.82068932056427,
+      "learning_rate": 9.278350515463918e-06,
+      "loss": 1.9995,
+      "step": 45
+    },
+    {
+      "epoch": 0.09467455621301775,
+      "grad_norm": 2.079101324081421,
+      "learning_rate": 9.484536082474226e-06,
+      "loss": 1.9601,
+      "step": 46
+    },
+    {
+      "epoch": 0.0967326987393877,
+      "grad_norm": 1.1209226846694946,
+      "learning_rate": 9.690721649484536e-06,
+      "loss": 1.9346,
+      "step": 47
+    },
+    {
+      "epoch": 0.09879084126575766,
+      "grad_norm": 1.0579711198806763,
+      "learning_rate": 9.896907216494846e-06,
+      "loss": 1.8764,
+      "step": 48
+    },
+    {
+      "epoch": 0.1008489837921276,
+      "grad_norm": 1.0434011220932007,
+      "learning_rate": 1.0103092783505156e-05,
+      "loss": 1.8483,
+      "step": 49
+    },
+    {
+      "epoch": 0.10290712631849755,
+      "grad_norm": 1.0089991092681885,
+      "learning_rate": 1.0309278350515464e-05,
+      "loss": 1.8018,
+      "step": 50
+    },
+    {
+      "epoch": 0.10496526884486751,
+      "grad_norm": 1.0117324590682983,
+      "learning_rate": 1.0515463917525775e-05,
+      "loss": 1.8003,
+      "step": 51
+    },
+    {
+      "epoch": 0.10702341137123746,
+      "grad_norm": 1.0006697177886963,
+      "learning_rate": 1.0721649484536083e-05,
+      "loss": 1.7482,
+      "step": 52
+    },
+    {
+      "epoch": 0.1090815538976074,
+      "grad_norm": 2.1164329051971436,
+      "learning_rate": 1.0927835051546391e-05,
+      "loss": 1.7363,
+      "step": 53
+    },
+    {
+      "epoch": 0.11113969642397736,
+      "grad_norm": 0.9573502540588379,
+      "learning_rate": 1.1134020618556703e-05,
+      "loss": 1.661,
+      "step": 54
+    },
+    {
+      "epoch": 0.11319783895034731,
+      "grad_norm": 1.0059764385223389,
+      "learning_rate": 1.134020618556701e-05,
+      "loss": 1.6979,
+      "step": 55
+    },
+    {
+      "epoch": 0.11525598147671726,
+      "grad_norm": 0.9719656109809875,
+      "learning_rate": 1.1546391752577319e-05,
+      "loss": 1.6318,
+      "step": 56
+    },
+    {
+      "epoch": 0.11731412400308722,
+      "grad_norm": 1.0024539232254028,
+      "learning_rate": 1.175257731958763e-05,
+      "loss": 1.6283,
+      "step": 57
+    },
+    {
+      "epoch": 0.11937226652945716,
+      "grad_norm": 0.9772456288337708,
+      "learning_rate": 1.1958762886597938e-05,
+      "loss": 1.5611,
+      "step": 58
+    },
+    {
+      "epoch": 0.12143040905582711,
+      "grad_norm": 0.9947625994682312,
+      "learning_rate": 1.2164948453608248e-05,
+      "loss": 1.6073,
+      "step": 59
+    },
+    {
+      "epoch": 0.12348855158219707,
+      "grad_norm": 2.112889051437378,
+      "learning_rate": 1.2371134020618558e-05,
+      "loss": 1.6208,
+      "step": 60
+    },
+    {
+      "epoch": 0.12554669410856703,
+      "grad_norm": 1.0515345335006714,
+      "learning_rate": 1.2577319587628866e-05,
+      "loss": 1.569,
+      "step": 61
+    },
+    {
+      "epoch": 0.12760483663493696,
+      "grad_norm": 1.0782145261764526,
+      "learning_rate": 1.2783505154639176e-05,
+      "loss": 1.5097,
+      "step": 62
+    },
+    {
+      "epoch": 0.12966297916130692,
+      "grad_norm": 1.154104232788086,
+      "learning_rate": 1.2989690721649485e-05,
+      "loss": 1.5472,
+      "step": 63
+    },
+    {
+      "epoch": 0.13172112168767688,
+      "grad_norm": 1.1614656448364258,
+      "learning_rate": 1.3195876288659795e-05,
+      "loss": 1.4833,
+      "step": 64
+    },
+    {
+      "epoch": 0.13377926421404682,
+      "grad_norm": 1.1720911264419556,
+      "learning_rate": 1.3402061855670103e-05,
+      "loss": 1.4644,
+      "step": 65
+    },
+    {
+      "epoch": 0.13583740674041678,
+      "grad_norm": 1.8903896808624268,
+      "learning_rate": 1.3608247422680415e-05,
+      "loss": 1.4286,
+      "step": 66
+    },
+    {
+      "epoch": 0.13789554926678674,
+      "grad_norm": 1.2675013542175293,
+      "learning_rate": 1.3814432989690723e-05,
+      "loss": 1.416,
+      "step": 67
+    },
+    {
+      "epoch": 0.13995369179315667,
+      "grad_norm": 1.266434907913208,
+      "learning_rate": 1.402061855670103e-05,
+      "loss": 1.3171,
+      "step": 68
+    },
+    {
+      "epoch": 0.14201183431952663,
+      "grad_norm": 1.3408889770507812,
+      "learning_rate": 1.4226804123711342e-05,
+      "loss": 1.3396,
+      "step": 69
+    },
+    {
+      "epoch": 0.1440699768458966,
+      "grad_norm": 1.3862446546554565,
+      "learning_rate": 1.443298969072165e-05,
+      "loss": 1.2642,
+      "step": 70
+    },
+    {
+      "epoch": 0.14612811937226652,
+      "grad_norm": 2.110553026199341,
+      "learning_rate": 1.4639175257731958e-05,
+      "loss": 1.2593,
+      "step": 71
+    },
+    {
+      "epoch": 0.14818626189863648,
+      "grad_norm": 1.7017499208450317,
+      "learning_rate": 1.484536082474227e-05,
+      "loss": 1.24,
+      "step": 72
+    },
+    {
+      "epoch": 0.15024440442500644,
+      "grad_norm": 1.9851700067520142,
+      "learning_rate": 1.5051546391752578e-05,
+      "loss": 1.2313,
+      "step": 73
+    },
+    {
+      "epoch": 0.15230254695137638,
+      "grad_norm": 2.009608030319214,
+      "learning_rate": 1.5257731958762888e-05,
+      "loss": 1.1281,
+      "step": 74
+    },
+    {
+      "epoch": 0.15436068947774634,
+      "grad_norm": 2.7587485313415527,
+      "learning_rate": 1.5463917525773197e-05,
+      "loss": 1.1248,
+      "step": 75
+    },
+    {
+      "epoch": 0.1564188320041163,
+      "grad_norm": 2.780954599380493,
+      "learning_rate": 1.5670103092783507e-05,
+      "loss": 1.0797,
+      "step": 76
+    },
+    {
+      "epoch": 0.15847697453048623,
+      "grad_norm": 3.1470866203308105,
+      "learning_rate": 1.5876288659793813e-05,
+      "loss": 1.0064,
+      "step": 77
+    },
+    {
+      "epoch": 0.1605351170568562,
+      "grad_norm": 4.653595447540283,
+      "learning_rate": 1.6082474226804127e-05,
+      "loss": 0.9219,
+      "step": 78
+    },
+    {
+      "epoch": 0.16259325958322615,
+      "grad_norm": 4.157363414764404,
+      "learning_rate": 1.6288659793814433e-05,
+      "loss": 0.8709,
+      "step": 79
+    },
+    {
+      "epoch": 0.16465140210959608,
+      "grad_norm": 4.5814924240112305,
+      "learning_rate": 1.6494845360824743e-05,
+      "loss": 0.7693,
+      "step": 80
+    },
+    {
+      "epoch": 0.16670954463596604,
+      "grad_norm": 5.096139907836914,
+      "learning_rate": 1.6701030927835052e-05,
+      "loss": 0.6868,
+      "step": 81
+    },
+    {
+      "epoch": 0.168767687162336,
+      "grad_norm": 4.858880519866943,
+      "learning_rate": 1.6907216494845362e-05,
+      "loss": 0.5971,
+      "step": 82
+    },
+    {
+      "epoch": 0.17082582968870594,
+      "grad_norm": 4.42564582824707,
+      "learning_rate": 1.7113402061855672e-05,
+      "loss": 0.4719,
+      "step": 83
+    },
+    {
+      "epoch": 0.1728839722150759,
+      "grad_norm": 7.720851421356201,
+      "learning_rate": 1.731958762886598e-05,
+      "loss": 0.3943,
+      "step": 84
+    },
+    {
+      "epoch": 0.17494211474144586,
+      "grad_norm": 0.41923192143440247,
+      "learning_rate": 1.752577319587629e-05,
+      "loss": 0.3635,
+      "step": 85
+    },
+    {
+      "epoch": 0.1770002572678158,
+      "grad_norm": 0.2771846354007721,
+      "learning_rate": 1.7731958762886598e-05,
+      "loss": 0.3597,
+      "step": 86
+    },
+    {
+      "epoch": 0.17905839979418575,
+      "grad_norm": 0.24761857092380524,
+      "learning_rate": 1.793814432989691e-05,
+      "loss": 0.3735,
+      "step": 87
+    },
+    {
+      "epoch": 0.1811165423205557,
+      "grad_norm": 0.23277048766613007,
+      "learning_rate": 1.8144329896907217e-05,
+      "loss": 0.3643,
+      "step": 88
+    },
+    {
+      "epoch": 0.18317468484692565,
+      "grad_norm": 0.22931228578090668,
+      "learning_rate": 1.8350515463917527e-05,
+      "loss": 0.3519,
+      "step": 89
+    },
+    {
+      "epoch": 0.1852328273732956,
+      "grad_norm": 0.20750615000724792,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 0.3431,
+      "step": 90
+    },
+    {
+      "epoch": 0.18729096989966554,
+      "grad_norm": 0.2080322951078415,
+      "learning_rate": 1.8762886597938147e-05,
+      "loss": 0.3632,
+      "step": 91
+    },
+    {
+      "epoch": 0.1893491124260355,
+      "grad_norm": 0.20186181366443634,
+      "learning_rate": 1.8969072164948453e-05,
+      "loss": 0.3492,
+      "step": 92
+    },
+    {
+      "epoch": 0.19140725495240546,
+      "grad_norm": 0.19172786176204681,
+      "learning_rate": 1.9175257731958766e-05,
+      "loss": 0.3552,
+      "step": 93
+    },
+    {
+      "epoch": 0.1934653974787754,
+      "grad_norm": 0.1747850626707077,
+      "learning_rate": 1.9381443298969072e-05,
+      "loss": 0.3355,
+      "step": 94
+    },
+    {
+      "epoch": 0.19552354000514535,
+      "grad_norm": 0.196411594748497,
+      "learning_rate": 1.9587628865979382e-05,
+      "loss": 0.3271,
+      "step": 95
+    },
+    {
+      "epoch": 0.1975816825315153,
+      "grad_norm": 0.20063228905200958,
+      "learning_rate": 1.9793814432989692e-05,
+      "loss": 0.3351,
+      "step": 96
+    },
+    {
+      "epoch": 0.19963982505788525,
+      "grad_norm": 0.19240939617156982,
+      "learning_rate": 2e-05,
+      "loss": 0.3266,
+      "step": 97
+    },
+    {
+      "epoch": 0.2016979675842552,
+      "grad_norm": 0.18206572532653809,
+      "learning_rate": 1.997709049255441e-05,
+      "loss": 0.3393,
+      "step": 98
+    },
+    {
+      "epoch": 0.20375611011062517,
+      "grad_norm": 0.20384562015533447,
+      "learning_rate": 1.9954180985108823e-05,
+      "loss": 0.3395,
+      "step": 99
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "grad_norm": 0.19944581389427185,
+      "learning_rate": 1.9931271477663232e-05,
+      "loss": 0.3268,
+      "step": 100
+    },
+    {
+      "epoch": 0.2058142526369951,
+      "eval_loss": 0.3456890285015106,
+      "eval_runtime": 2114.0178,
+      "eval_samples_per_second": 3.677,
+      "eval_steps_per_second": 0.92,
+      "step": 100
+    },
+    {
+      "epoch": 0.20787239516336506,
+      "grad_norm": 0.17743557691574097,
+      "learning_rate": 1.990836197021764e-05,
+      "loss": 0.3439,
+      "step": 101
+    },
+    {
+      "epoch": 0.20993053768973502,
+      "grad_norm": 0.18746449053287506,
+      "learning_rate": 1.9885452462772053e-05,
+      "loss": 0.326,
+      "step": 102
+    },
+    {
+      "epoch": 0.21198868021610495,
+      "grad_norm": 0.18555815517902374,
+      "learning_rate": 1.9862542955326462e-05,
+      "loss": 0.3337,
+      "step": 103
+    },
+    {
+      "epoch": 0.2140468227424749,
+      "grad_norm": 0.16591575741767883,
+      "learning_rate": 1.9839633447880874e-05,
+      "loss": 0.3121,
+      "step": 104
+    },
+    {
+      "epoch": 0.21610496526884487,
+      "grad_norm": 0.1621987372636795,
+      "learning_rate": 1.9816723940435283e-05,
+      "loss": 0.3287,
+      "step": 105
+    },
+    {
+      "epoch": 0.2181631077952148,
+      "grad_norm": 0.1614532470703125,
+      "learning_rate": 1.9793814432989692e-05,
+      "loss": 0.3306,
+      "step": 106
+    },
+    {
+      "epoch": 0.22022125032158477,
+      "grad_norm": 0.17993387579917908,
+      "learning_rate": 1.9770904925544104e-05,
+      "loss": 0.3341,
+      "step": 107
+    },
+    {
+      "epoch": 0.22227939284795473,
+      "grad_norm": 0.1550011783838272,
+      "learning_rate": 1.9747995418098513e-05,
+      "loss": 0.3197,
+      "step": 108
+    },
+    {
+      "epoch": 0.22433753537432466,
+      "grad_norm": 0.18471524119377136,
+      "learning_rate": 1.9725085910652922e-05,
+      "loss": 0.3285,
+      "step": 109
+    },
+    {
+      "epoch": 0.22639567790069462,
+      "grad_norm": 0.15604373812675476,
+      "learning_rate": 1.9702176403207334e-05,
+      "loss": 0.3298,
+      "step": 110
+    },
+    {
+      "epoch": 0.22845382042706458,
+      "grad_norm": 0.1682298630475998,
+      "learning_rate": 1.9679266895761743e-05,
+      "loss": 0.3343,
+      "step": 111
+    },
+    {
+      "epoch": 0.2305119629534345,
+      "grad_norm": 0.14933635294437408,
+      "learning_rate": 1.9656357388316152e-05,
+      "loss": 0.3134,
+      "step": 112
+    },
+    {
+      "epoch": 0.23257010547980447,
+      "grad_norm": 0.14892347157001495,
+      "learning_rate": 1.963344788087056e-05,
+      "loss": 0.3154,
+      "step": 113
+    },
+    {
+      "epoch": 0.23462824800617443,
+      "grad_norm": 0.1577889323234558,
+      "learning_rate": 1.9610538373424973e-05,
+      "loss": 0.3122,
+      "step": 114
+    },
+    {
+      "epoch": 0.23668639053254437,
+      "grad_norm": 0.16482344269752502,
+      "learning_rate": 1.9587628865979382e-05,
+      "loss": 0.3193,
+      "step": 115
+    },
+    {
+      "epoch": 0.23874453305891433,
+      "grad_norm": 0.15328913927078247,
+      "learning_rate": 1.956471935853379e-05,
+      "loss": 0.3217,
+      "step": 116
+    },
+    {
+      "epoch": 0.2408026755852843,
+      "grad_norm": 0.16140656173229218,
+      "learning_rate": 1.9541809851088203e-05,
+      "loss": 0.318,
+      "step": 117
+    },
+    {
+      "epoch": 0.24286081811165422,
+      "grad_norm": 0.15448373556137085,
+      "learning_rate": 1.9518900343642612e-05,
+      "loss": 0.3205,
+      "step": 118
+    },
+    {
+      "epoch": 0.24491896063802418,
+      "grad_norm": 0.14716887474060059,
+      "learning_rate": 1.9495990836197025e-05,
+      "loss": 0.3164,
+      "step": 119
+    },
+    {
+      "epoch": 0.24697710316439414,
+      "grad_norm": 0.16582027077674866,
+      "learning_rate": 1.9473081328751433e-05,
+      "loss": 0.3191,
+      "step": 120
+    },
+    {
+      "epoch": 0.24903524569076407,
+      "grad_norm": 0.15213699638843536,
+      "learning_rate": 1.9450171821305842e-05,
+      "loss": 0.304,
+      "step": 121
+    },
+    {
+      "epoch": 0.25109338821713406,
+      "grad_norm": 0.1659238487482071,
+      "learning_rate": 1.9427262313860255e-05,
+      "loss": 0.3184,
+      "step": 122
+    },
+    {
+      "epoch": 0.253151530743504,
+      "grad_norm": 0.15596656501293182,
+      "learning_rate": 1.9404352806414663e-05,
+      "loss": 0.3092,
+      "step": 123
+    },
+    {
+      "epoch": 0.2552096732698739,
+      "grad_norm": 0.15868476033210754,
+      "learning_rate": 1.9381443298969072e-05,
+      "loss": 0.3163,
+      "step": 124
+    },
+    {
+      "epoch": 0.2572678157962439,
+      "grad_norm": 0.15386095643043518,
+      "learning_rate": 1.9358533791523485e-05,
+      "loss": 0.3049,
+      "step": 125
+    },
+    {
+      "epoch": 0.25932595832261385,
+      "grad_norm": 0.15179213881492615,
+      "learning_rate": 1.9335624284077894e-05,
+      "loss": 0.3131,
+      "step": 126
+    },
+    {
+      "epoch": 0.2613841008489838,
+      "grad_norm": 0.1595134735107422,
+      "learning_rate": 1.9312714776632306e-05,
+      "loss": 0.3069,
+      "step": 127
+    },
+    {
+      "epoch": 0.26344224337535377,
+      "grad_norm": 0.16989803314208984,
+      "learning_rate": 1.9289805269186715e-05,
+      "loss": 0.3052,
+      "step": 128
+    },
+    {
+      "epoch": 0.2655003859017237,
+      "grad_norm": 0.14803892374038696,
+      "learning_rate": 1.9266895761741124e-05,
+      "loss": 0.3065,
+      "step": 129
+    },
+    {
+      "epoch": 0.26755852842809363,
+      "grad_norm": 0.16676583886146545,
+      "learning_rate": 1.9243986254295536e-05,
+      "loss": 0.2962,
+      "step": 130
+    },
+    {
+      "epoch": 0.2696166709544636,
+      "grad_norm": 0.15694552659988403,
+      "learning_rate": 1.9221076746849945e-05,
+      "loss": 0.3096,
+      "step": 131
+    },
+    {
+      "epoch": 0.27167481348083355,
+      "grad_norm": 0.17696696519851685,
+      "learning_rate": 1.9198167239404354e-05,
+      "loss": 0.3145,
+      "step": 132
+    },
+    {
+      "epoch": 0.2737329560072035,
+      "grad_norm": 0.17204038798809052,
+      "learning_rate": 1.9175257731958766e-05,
+      "loss": 0.3248,
+      "step": 133
+    },
+    {
+      "epoch": 0.2757910985335735,
+      "grad_norm": 0.15630359947681427,
+      "learning_rate": 1.9152348224513175e-05,
+      "loss": 0.3117,
+      "step": 134
+    },
+    {
+      "epoch": 0.2778492410599434,
+      "grad_norm": 0.15757997334003448,
+      "learning_rate": 1.9129438717067584e-05,
+      "loss": 0.3145,
+      "step": 135
+    },
+    {
+      "epoch": 0.27990738358631334,
+      "grad_norm": 0.16273653507232666,
+      "learning_rate": 1.9106529209621996e-05,
+      "loss": 0.3159,
+      "step": 136
+    },
+    {
+      "epoch": 0.28196552611268333,
+      "grad_norm": 0.16213104128837585,
+      "learning_rate": 1.9083619702176405e-05,
+      "loss": 0.2949,
+      "step": 137
+    },
+    {
+      "epoch": 0.28402366863905326,
+      "grad_norm": 0.15377865731716156,
+      "learning_rate": 1.9060710194730814e-05,
+      "loss": 0.306,
+      "step": 138
+    },
+    {
+      "epoch": 0.2860818111654232,
+      "grad_norm": 0.1545962244272232,
+      "learning_rate": 1.9037800687285223e-05,
+      "loss": 0.2966,
+      "step": 139
+    },
+    {
+      "epoch": 0.2881399536917932,
+      "grad_norm": 0.15516617894172668,
+      "learning_rate": 1.9014891179839635e-05,
+      "loss": 0.3122,
+      "step": 140
+    },
+    {
+      "epoch": 0.2901980962181631,
+      "grad_norm": 0.14734458923339844,
+      "learning_rate": 1.8991981672394044e-05,
+      "loss": 0.3118,
+      "step": 141
+    },
+    {
+      "epoch": 0.29225623874453305,
+      "grad_norm": 0.1644304096698761,
+      "learning_rate": 1.8969072164948453e-05,
+      "loss": 0.3027,
+      "step": 142
+    },
+    {
+      "epoch": 0.29431438127090304,
+      "grad_norm": 0.14632569253444672,
+      "learning_rate": 1.8946162657502865e-05,
+      "loss": 0.3023,
+      "step": 143
+    },
+    {
+      "epoch": 0.29637252379727297,
+      "grad_norm": 0.1573137789964676,
+      "learning_rate": 1.8923253150057274e-05,
+      "loss": 0.3102,
+      "step": 144
+    },
+    {
+      "epoch": 0.2984306663236429,
+      "grad_norm": 0.16423144936561584,
+      "learning_rate": 1.8900343642611686e-05,
+      "loss": 0.3033,
+      "step": 145
+    },
+    {
+      "epoch": 0.3004888088500129,
+      "grad_norm": 0.15420907735824585,
+      "learning_rate": 1.8877434135166095e-05,
+      "loss": 0.3089,
+      "step": 146
+    },
+    {
+      "epoch": 0.3025469513763828,
+      "grad_norm": 0.1579178273677826,
+      "learning_rate": 1.8854524627720504e-05,
+      "loss": 0.3071,
+      "step": 147
+    },
+    {
+      "epoch": 0.30460509390275275,
+      "grad_norm": 0.15866397321224213,
+      "learning_rate": 1.8831615120274916e-05,
+      "loss": 0.3083,
+      "step": 148
+    },
+    {
+      "epoch": 0.30666323642912274,
+      "grad_norm": 0.16651487350463867,
+      "learning_rate": 1.8808705612829325e-05,
+      "loss": 0.3099,
+      "step": 149
+    },
+    {
+      "epoch": 0.3087213789554927,
+      "grad_norm": 0.16281908750534058,
+      "learning_rate": 1.8785796105383734e-05,
+      "loss": 0.3034,
+      "step": 150
+    },
+    {
+      "epoch": 0.3107795214818626,
+      "grad_norm": 0.17449837923049927,
+      "learning_rate": 1.8762886597938147e-05,
+      "loss": 0.3054,
+      "step": 151
+    },
+    {
+      "epoch": 0.3128376640082326,
+      "grad_norm": 0.15403546392917633,
+      "learning_rate": 1.8739977090492555e-05,
+      "loss": 0.297,
+      "step": 152
+    },
+    {
+      "epoch": 0.31489580653460253,
+      "grad_norm": 0.1472466140985489,
+      "learning_rate": 1.8717067583046968e-05,
+      "loss": 0.2973,
+      "step": 153
+    },
+    {
+      "epoch": 0.31695394906097246,
+      "grad_norm": 0.16027937829494476,
+      "learning_rate": 1.8694158075601377e-05,
+      "loss": 0.3054,
+      "step": 154
+    },
+    {
+      "epoch": 0.31901209158734245,
+      "grad_norm": 0.17086225748062134,
+      "learning_rate": 1.8671248568155786e-05,
+      "loss": 0.307,
+      "step": 155
+    },
+    {
+      "epoch": 0.3210702341137124,
+      "grad_norm": 0.15930697321891785,
+      "learning_rate": 1.8648339060710198e-05,
+      "loss": 0.293,
+      "step": 156
+    },
+    {
+      "epoch": 0.3231283766400823,
+      "grad_norm": 0.17086376249790192,
+      "learning_rate": 1.8625429553264607e-05,
+      "loss": 0.293,
+      "step": 157
+    },
+    {
+      "epoch": 0.3251865191664523,
+      "grad_norm": 0.15970875322818756,
+      "learning_rate": 1.8602520045819016e-05,
+      "loss": 0.3083,
+      "step": 158
+    },
+    {
+      "epoch": 0.32724466169282224,
+      "grad_norm": 0.16355909407138824,
+      "learning_rate": 1.8579610538373428e-05,
+      "loss": 0.3139,
+      "step": 159
+    },
+    {
+      "epoch": 0.32930280421919217,
+      "grad_norm": 0.15183711051940918,
+      "learning_rate": 1.8556701030927837e-05,
+      "loss": 0.2953,
+      "step": 160
+    },
+    {
+      "epoch": 0.33136094674556216,
+      "grad_norm": 0.15123715996742249,
+      "learning_rate": 1.853379152348225e-05,
+      "loss": 0.3025,
+      "step": 161
+    },
+    {
+      "epoch": 0.3334190892719321,
+      "grad_norm": 0.1576143503189087,
+      "learning_rate": 1.8510882016036658e-05,
+      "loss": 0.2904,
+      "step": 162
+    },
+    {
+      "epoch": 0.335477231798302,
+      "grad_norm": 0.1457504779100418,
+      "learning_rate": 1.8487972508591067e-05,
+      "loss": 0.2909,
+      "step": 163
+    },
+    {
+      "epoch": 0.337535374324672,
+      "grad_norm": 0.1557442992925644,
+      "learning_rate": 1.846506300114548e-05,
+      "loss": 0.3027,
+      "step": 164
+    },
+    {
+      "epoch": 0.33959351685104194,
+      "grad_norm": 0.15662318468093872,
+      "learning_rate": 1.8442153493699888e-05,
+      "loss": 0.311,
+      "step": 165
+    },
+    {
+      "epoch": 0.3416516593774119,
+      "grad_norm": 0.16177058219909668,
+      "learning_rate": 1.8419243986254297e-05,
+      "loss": 0.2944,
+      "step": 166
+    },
+    {
+      "epoch": 0.34370980190378186,
+      "grad_norm": 0.16406729817390442,
+      "learning_rate": 1.8396334478808706e-05,
+      "loss": 0.2927,
+      "step": 167
+    },
+    {
+      "epoch": 0.3457679444301518,
+      "grad_norm": 0.16642791032791138,
+      "learning_rate": 1.8373424971363115e-05,
+      "loss": 0.3063,
+      "step": 168
+    },
+    {
+      "epoch": 0.34782608695652173,
+      "grad_norm": 0.1650693714618683,
+      "learning_rate": 1.8350515463917527e-05,
+      "loss": 0.2957,
+      "step": 169
+    },
+    {
+      "epoch": 0.3498842294828917,
+      "grad_norm": 0.15349675714969635,
+      "learning_rate": 1.8327605956471936e-05,
+      "loss": 0.297,
+      "step": 170
+    },
+    {
+      "epoch": 0.35194237200926165,
+      "grad_norm": 0.17770209908485413,
+      "learning_rate": 1.8304696449026348e-05,
+      "loss": 0.3011,
+      "step": 171
+    },
+    {
+      "epoch": 0.3540005145356316,
+      "grad_norm": 0.1647631675004959,
+      "learning_rate": 1.8281786941580757e-05,
+      "loss": 0.2962,
+      "step": 172
+    },
+    {
+      "epoch": 0.35605865706200157,
+      "grad_norm": 0.1603834480047226,
+      "learning_rate": 1.8258877434135166e-05,
+      "loss": 0.2937,
+      "step": 173
+    },
+    {
+      "epoch": 0.3581167995883715,
+      "grad_norm": 0.16780880093574524,
+      "learning_rate": 1.8235967926689578e-05,
+      "loss": 0.2997,
+      "step": 174
+    },
+    {
+      "epoch": 0.36017494211474144,
+      "grad_norm": 0.15976767241954803,
+      "learning_rate": 1.8213058419243987e-05,
+      "loss": 0.3043,
+      "step": 175
+    },
+    {
+      "epoch": 0.3622330846411114,
+      "grad_norm": 0.16236485540866852,
+      "learning_rate": 1.8190148911798396e-05,
+      "loss": 0.3069,
+      "step": 176
+    },
+    {
+      "epoch": 0.36429122716748136,
+      "grad_norm": 0.16391968727111816,
+      "learning_rate": 1.816723940435281e-05,
+      "loss": 0.2923,
+      "step": 177
+    },
+    {
+      "epoch": 0.3663493696938513,
+      "grad_norm": 0.15806889533996582,
+      "learning_rate": 1.8144329896907217e-05,
+      "loss": 0.2872,
+      "step": 178
+    },
+    {
+      "epoch": 0.3684075122202212,
+      "grad_norm": 0.1627352088689804,
+      "learning_rate": 1.812142038946163e-05,
+      "loss": 0.3032,
+      "step": 179
+    },
+    {
+      "epoch": 0.3704656547465912,
+      "grad_norm": 0.15103371441364288,
+      "learning_rate": 1.809851088201604e-05,
+      "loss": 0.2847,
+      "step": 180
+    },
+    {
+      "epoch": 0.37252379727296114,
+      "grad_norm": 0.15178488194942474,
+      "learning_rate": 1.8075601374570447e-05,
+      "loss": 0.3017,
+      "step": 181
+    },
+    {
+      "epoch": 0.3745819397993311,
+      "grad_norm": 0.15493899583816528,
+      "learning_rate": 1.805269186712486e-05,
+      "loss": 0.2901,
+      "step": 182
+    },
+    {
+      "epoch": 0.37664008232570106,
+      "grad_norm": 0.15990686416625977,
+      "learning_rate": 1.802978235967927e-05,
+      "loss": 0.2861,
+      "step": 183
+    },
+    {
+      "epoch": 0.378698224852071,
+      "grad_norm": 0.15824148058891296,
+      "learning_rate": 1.8006872852233677e-05,
+      "loss": 0.2885,
+      "step": 184
+    },
+    {
+      "epoch": 0.38075636737844093,
+      "grad_norm": 0.15690775215625763,
+      "learning_rate": 1.798396334478809e-05,
+      "loss": 0.2814,
+      "step": 185
+    },
+    {
+      "epoch": 0.3828145099048109,
+      "grad_norm": 0.15833796560764313,
+      "learning_rate": 1.79610538373425e-05,
+      "loss": 0.2847,
+      "step": 186
+    },
+    {
+      "epoch": 0.38487265243118085,
+      "grad_norm": 0.16560044884681702,
+      "learning_rate": 1.793814432989691e-05,
+      "loss": 0.3061,
+      "step": 187
+    },
+    {
+      "epoch": 0.3869307949575508,
+      "grad_norm": 0.16240179538726807,
+      "learning_rate": 1.791523482245132e-05,
+      "loss": 0.2943,
+      "step": 188
+    },
+    {
+      "epoch": 0.38898893748392077,
+      "grad_norm": 0.15825721621513367,
+      "learning_rate": 1.789232531500573e-05,
+      "loss": 0.2934,
+      "step": 189
+    },
+    {
+      "epoch": 0.3910470800102907,
+      "grad_norm": 0.16665388643741608,
+      "learning_rate": 1.786941580756014e-05,
+      "loss": 0.291,
+      "step": 190
+    },
+    {
+      "epoch": 0.39310522253666064,
+      "grad_norm": 0.16581200063228607,
+      "learning_rate": 1.784650630011455e-05,
+      "loss": 0.2849,
+      "step": 191
+    },
+    {
+      "epoch": 0.3951633650630306,
+      "grad_norm": 0.1604345291852951,
+      "learning_rate": 1.782359679266896e-05,
+      "loss": 0.3,
+      "step": 192
+    },
+    {
+      "epoch": 0.39722150758940056,
+      "grad_norm": 0.16107915341854095,
+      "learning_rate": 1.7800687285223368e-05,
+      "loss": 0.2847,
+      "step": 193
+    },
+    {
+      "epoch": 0.3992796501157705,
+      "grad_norm": 0.1571730375289917,
+      "learning_rate": 1.7777777777777777e-05,
+      "loss": 0.2863,
+      "step": 194
+    },
+    {
+      "epoch": 0.4013377926421405,
+      "grad_norm": 0.1656399518251419,
+      "learning_rate": 1.775486827033219e-05,
+      "loss": 0.2878,
+      "step": 195
+    },
+    {
+      "epoch": 0.4033959351685104,
+      "grad_norm": 0.16738460958003998,
+      "learning_rate": 1.7731958762886598e-05,
+      "loss": 0.286,
+      "step": 196
+    },
+    {
+      "epoch": 0.40545407769488034,
+      "grad_norm": 0.16704292595386505,
+      "learning_rate": 1.770904925544101e-05,
+      "loss": 0.2919,
+      "step": 197
+    },
+    {
+      "epoch": 0.40751222022125033,
+      "grad_norm": 0.16215579211711884,
+      "learning_rate": 1.768613974799542e-05,
+      "loss": 0.2874,
+      "step": 198
+    },
+    {
+      "epoch": 0.40957036274762026,
+      "grad_norm": 0.15573479235172272,
+      "learning_rate": 1.7663230240549828e-05,
+      "loss": 0.2904,
+      "step": 199
+    },
+    {
+      "epoch": 0.4116285052739902,
+      "grad_norm": 0.1707623153924942,
+      "learning_rate": 1.764032073310424e-05,
+      "loss": 0.289,
+      "step": 200
+    },
+    {
+      "epoch": 0.4116285052739902,
+      "eval_loss": 0.3214050829410553,
+      "eval_runtime": 2449.7742,
+      "eval_samples_per_second": 3.173,
+      "eval_steps_per_second": 0.794,
+      "step": 200
+    },
+    {
+      "epoch": 0.4136866478003602,
+      "grad_norm": 0.1699172556400299,
+      "learning_rate": 1.761741122565865e-05,
+      "loss": 0.2852,
+      "step": 201
+    },
+    {
+      "epoch": 0.4157447903267301,
+      "grad_norm": 0.19150058925151825,
+      "learning_rate": 1.7594501718213058e-05,
+      "loss": 0.29,
+      "step": 202
+    },
+    {
+      "epoch": 0.41780293285310005,
+      "grad_norm": 0.15794627368450165,
+      "learning_rate": 1.757159221076747e-05,
+      "loss": 0.2746,
+      "step": 203
+    },
+    {
+      "epoch": 0.41986107537947004,
+      "grad_norm": 0.17305190861225128,
+      "learning_rate": 1.754868270332188e-05,
+      "loss": 0.3003,
+      "step": 204
+    },
+    {
+      "epoch": 0.42191921790583997,
+      "grad_norm": 0.16257523000240326,
+      "learning_rate": 1.752577319587629e-05,
+      "loss": 0.2789,
+      "step": 205
+    },
+    {
+      "epoch": 0.4239773604322099,
+      "grad_norm": 0.17273619771003723,
+      "learning_rate": 1.75028636884307e-05,
+      "loss": 0.2917,
+      "step": 206
+    },
+    {
+      "epoch": 0.4260355029585799,
+      "grad_norm": 0.17502790689468384,
+      "learning_rate": 1.747995418098511e-05,
+      "loss": 0.2992,
+      "step": 207
+    },
+    {
+      "epoch": 0.4280936454849498,
+      "grad_norm": 0.16464050114154816,
+      "learning_rate": 1.745704467353952e-05,
+      "loss": 0.2873,
+      "step": 208
+    },
+    {
+      "epoch": 0.43015178801131976,
+      "grad_norm": 0.1681668758392334,
+      "learning_rate": 1.743413516609393e-05,
+      "loss": 0.2991,
+      "step": 209
+    },
+    {
+      "epoch": 0.43220993053768975,
+      "grad_norm": 0.16957956552505493,
+      "learning_rate": 1.741122565864834e-05,
+      "loss": 0.2868,
+      "step": 210
+    },
+    {
+      "epoch": 0.4342680730640597,
+      "grad_norm": 0.15875883400440216,
+      "learning_rate": 1.738831615120275e-05,
+      "loss": 0.2946,
+      "step": 211
+    },
+    {
+      "epoch": 0.4363262155904296,
+      "grad_norm": 0.18127889931201935,
+      "learning_rate": 1.736540664375716e-05,
+      "loss": 0.2835,
+      "step": 212
+    },
+    {
+      "epoch": 0.4383843581167996,
+      "grad_norm": 0.17822811007499695,
+      "learning_rate": 1.7342497136311573e-05,
+      "loss": 0.2944,
+      "step": 213
+    },
+    {
+      "epoch": 0.44044250064316953,
+      "grad_norm": 0.17555806040763855,
+      "learning_rate": 1.731958762886598e-05,
+      "loss": 0.3001,
+      "step": 214
+    },
+    {
+      "epoch": 0.44250064316953946,
+      "grad_norm": 0.18709121644496918,
+      "learning_rate": 1.729667812142039e-05,
+      "loss": 0.282,
+      "step": 215
+    },
+    {
+      "epoch": 0.44455878569590945,
+      "grad_norm": 0.16322475671768188,
+      "learning_rate": 1.7273768613974803e-05,
+      "loss": 0.2883,
+      "step": 216
+    },
+    {
+      "epoch": 0.4466169282222794,
+      "grad_norm": 0.1677054911851883,
+      "learning_rate": 1.7250859106529212e-05,
+      "loss": 0.28,
+      "step": 217
+    },
+    {
+      "epoch": 0.4486750707486493,
+      "grad_norm": 0.15764063596725464,
+      "learning_rate": 1.722794959908362e-05,
+      "loss": 0.2768,
+      "step": 218
+    },
+    {
+      "epoch": 0.4507332132750193,
+      "grad_norm": 0.16166841983795166,
+      "learning_rate": 1.7205040091638033e-05,
+      "loss": 0.2868,
+      "step": 219
+    },
+    {
+      "epoch": 0.45279135580138924,
+      "grad_norm": 0.1799350380897522,
+      "learning_rate": 1.7182130584192442e-05,
+      "loss": 0.2891,
+      "step": 220
+    },
+    {
+      "epoch": 0.45484949832775917,
+      "grad_norm": 0.18119174242019653,
+      "learning_rate": 1.715922107674685e-05,
+      "loss": 0.2841,
+      "step": 221
+    },
+    {
+      "epoch": 0.45690764085412916,
+      "grad_norm": 0.17725548148155212,
+      "learning_rate": 1.713631156930126e-05,
+      "loss": 0.3038,
+      "step": 222
+    },
+    {
+      "epoch": 0.4589657833804991,
+      "grad_norm": 0.1628233790397644,
+      "learning_rate": 1.7113402061855672e-05,
+      "loss": 0.2868,
+      "step": 223
+    },
+    {
+      "epoch": 0.461023925906869,
+      "grad_norm": 0.1745166927576065,
+      "learning_rate": 1.709049255441008e-05,
+      "loss": 0.3033,
+      "step": 224
+    },
+    {
+      "epoch": 0.463082068433239,
+      "grad_norm": 0.17708267271518707,
+      "learning_rate": 1.706758304696449e-05,
+      "loss": 0.2842,
+      "step": 225
+    },
+    {
+      "epoch": 0.46514021095960895,
+      "grad_norm": 0.1738453358411789,
+      "learning_rate": 1.7044673539518902e-05,
+      "loss": 0.3005,
+      "step": 226
+    },
+    {
+      "epoch": 0.4671983534859789,
+      "grad_norm": 0.1706874966621399,
+      "learning_rate": 1.702176403207331e-05,
+      "loss": 0.2924,
+      "step": 227
+    },
+    {
+      "epoch": 0.46925649601234887,
+      "grad_norm": 0.1697423756122589,
+      "learning_rate": 1.699885452462772e-05,
+      "loss": 0.2783,
+      "step": 228
+    },
+    {
+      "epoch": 0.4713146385387188,
+      "grad_norm": 0.1783403754234314,
+      "learning_rate": 1.6975945017182132e-05,
+      "loss": 0.2924,
+      "step": 229
+    },
+    {
+      "epoch": 0.47337278106508873,
+      "grad_norm": 0.17431536316871643,
+      "learning_rate": 1.695303550973654e-05,
+      "loss": 0.2792,
+      "step": 230
+    },
+    {
+      "epoch": 0.4754309235914587,
+      "grad_norm": 0.164026141166687,
+      "learning_rate": 1.6930126002290953e-05,
+      "loss": 0.2825,
+      "step": 231
+    },
+    {
+      "epoch": 0.47748906611782865,
+      "grad_norm": 0.16449657082557678,
+      "learning_rate": 1.6907216494845362e-05,
+      "loss": 0.2831,
+      "step": 232
+    },
+    {
+      "epoch": 0.4795472086441986,
+      "grad_norm": 0.1812741607427597,
+      "learning_rate": 1.688430698739977e-05,
+      "loss": 0.2849,
+      "step": 233
+    },
+    {
+      "epoch": 0.4816053511705686,
+      "grad_norm": 0.18431834876537323,
+      "learning_rate": 1.6861397479954183e-05,
+      "loss": 0.2802,
+      "step": 234
+    },
+    {
+      "epoch": 0.4836634936969385,
+      "grad_norm": 0.18349015712738037,
+      "learning_rate": 1.6838487972508592e-05,
+      "loss": 0.2804,
+      "step": 235
+    },
+    {
+      "epoch": 0.48572163622330844,
+      "grad_norm": 0.1769968420267105,
+      "learning_rate": 1.6815578465063e-05,
+      "loss": 0.2777,
+      "step": 236
+    },
+    {
+      "epoch": 0.4877797787496784,
+      "grad_norm": 0.17207500338554382,
+      "learning_rate": 1.6792668957617413e-05,
+      "loss": 0.2883,
+      "step": 237
+    },
+    {
+      "epoch": 0.48983792127604836,
+      "grad_norm": 0.1729692667722702,
+      "learning_rate": 1.6769759450171822e-05,
+      "loss": 0.2784,
+      "step": 238
+    },
+    {
+      "epoch": 0.4918960638024183,
+      "grad_norm": 0.17234881222248077,
+      "learning_rate": 1.6746849942726235e-05,
+      "loss": 0.2816,
+      "step": 239
+    },
+    {
+      "epoch": 0.4939542063287883,
+      "grad_norm": 0.17132551968097687,
+      "learning_rate": 1.6723940435280644e-05,
+      "loss": 0.2812,
+      "step": 240
+    },
+    {
+      "epoch": 0.4960123488551582,
+      "grad_norm": 0.1752254068851471,
+      "learning_rate": 1.6701030927835052e-05,
+      "loss": 0.2799,
+      "step": 241
+    },
+    {
+      "epoch": 0.49807049138152815,
+      "grad_norm": 0.1768665313720703,
+      "learning_rate": 1.6678121420389465e-05,
+      "loss": 0.2966,
+      "step": 242
+    },
+    {
+      "epoch": 0.5001286339078981,
+      "grad_norm": 0.18139514327049255,
+      "learning_rate": 1.6655211912943874e-05,
+      "loss": 0.2816,
+      "step": 243
+    },
+    {
+      "epoch": 0.5021867764342681,
+      "grad_norm": 0.17312943935394287,
+      "learning_rate": 1.6632302405498283e-05,
+      "loss": 0.2845,
+      "step": 244
+    },
+    {
+      "epoch": 0.5042449189606381,
+      "grad_norm": 0.17966389656066895,
+      "learning_rate": 1.6609392898052695e-05,
+      "loss": 0.2864,
+      "step": 245
+    },
+    {
+      "epoch": 0.506303061487008,
+      "grad_norm": 0.16653811931610107,
+      "learning_rate": 1.6586483390607104e-05,
+      "loss": 0.2759,
+      "step": 246
+    },
+    {
+      "epoch": 0.5083612040133779,
+      "grad_norm": 0.1634613424539566,
+      "learning_rate": 1.6563573883161516e-05,
+      "loss": 0.2728,
+      "step": 247
+    },
+    {
+      "epoch": 0.5104193465397479,
+      "grad_norm": 0.17358507215976715,
+      "learning_rate": 1.654066437571592e-05,
+      "loss": 0.2706,
+      "step": 248
+    },
+    {
+      "epoch": 0.5124774890661178,
+      "grad_norm": 0.17524316906929016,
+      "learning_rate": 1.6517754868270334e-05,
+      "loss": 0.2805,
+      "step": 249
+    },
+    {
+      "epoch": 0.5145356315924878,
+      "grad_norm": 0.18134094774723053,
+      "learning_rate": 1.6494845360824743e-05,
+      "loss": 0.2909,
+      "step": 250
+    },
+    {
+      "epoch": 0.5165937741188578,
+      "grad_norm": 0.17795510590076447,
+      "learning_rate": 1.647193585337915e-05,
+      "loss": 0.2889,
+      "step": 251
+    },
+    {
+      "epoch": 0.5186519166452277,
+      "grad_norm": 0.16782547533512115,
+      "learning_rate": 1.6449026345933564e-05,
+      "loss": 0.2842,
+      "step": 252
+    },
+    {
+      "epoch": 0.5207100591715976,
+      "grad_norm": 0.17360062897205353,
+      "learning_rate": 1.6426116838487973e-05,
+      "loss": 0.2763,
+      "step": 253
+    },
+    {
+      "epoch": 0.5227682016979676,
+      "grad_norm": 0.17241406440734863,
+      "learning_rate": 1.6403207331042385e-05,
+      "loss": 0.2753,
+      "step": 254
+    },
+    {
+      "epoch": 0.5248263442243375,
+      "grad_norm": 0.1709229201078415,
+      "learning_rate": 1.6380297823596794e-05,
+      "loss": 0.2732,
+      "step": 255
+    },
+    {
+      "epoch": 0.5268844867507075,
+      "grad_norm": 0.1807374209165573,
+      "learning_rate": 1.6357388316151203e-05,
+      "loss": 0.2856,
+      "step": 256
+    },
+    {
+      "epoch": 0.5289426292770775,
+      "grad_norm": 0.1749904304742813,
+      "learning_rate": 1.6334478808705615e-05,
+      "loss": 0.285,
+      "step": 257
+    },
+    {
+      "epoch": 0.5310007718034474,
+      "grad_norm": 0.16673170030117035,
+      "learning_rate": 1.6311569301260024e-05,
+      "loss": 0.2825,
+      "step": 258
+    },
+    {
+      "epoch": 0.5330589143298173,
+      "grad_norm": 0.17239685356616974,
+      "learning_rate": 1.6288659793814433e-05,
+      "loss": 0.2845,
+      "step": 259
+    },
+    {
+      "epoch": 0.5351170568561873,
+      "grad_norm": 0.1831504851579666,
+      "learning_rate": 1.6265750286368845e-05,
+      "loss": 0.2859,
+      "step": 260
+    },
+    {
+      "epoch": 0.5371751993825572,
+      "grad_norm": 0.18507827818393707,
+      "learning_rate": 1.6242840778923254e-05,
+      "loss": 0.293,
+      "step": 261
+    },
+    {
+      "epoch": 0.5392333419089272,
+      "grad_norm": 0.16738134622573853,
+      "learning_rate": 1.6219931271477663e-05,
+      "loss": 0.2853,
+      "step": 262
+    },
+    {
+      "epoch": 0.5412914844352972,
+      "grad_norm": 0.1701226830482483,
+      "learning_rate": 1.6197021764032075e-05,
+      "loss": 0.2763,
+      "step": 263
+    },
+    {
+      "epoch": 0.5433496269616671,
+      "grad_norm": 0.18195705115795135,
+      "learning_rate": 1.6174112256586484e-05,
+      "loss": 0.2797,
+      "step": 264
+    },
+    {
+      "epoch": 0.545407769488037,
+      "grad_norm": 0.1832309514284134,
+      "learning_rate": 1.6151202749140896e-05,
+      "loss": 0.2885,
+      "step": 265
+    },
+    {
+      "epoch": 0.547465912014407,
+      "grad_norm": 0.1773810088634491,
+      "learning_rate": 1.6128293241695305e-05,
+      "loss": 0.2682,
+      "step": 266
+    },
+    {
+      "epoch": 0.5495240545407769,
+      "grad_norm": 0.16989603638648987,
+      "learning_rate": 1.6105383734249714e-05,
+      "loss": 0.2821,
+      "step": 267
+    },
+    {
+      "epoch": 0.551582197067147,
+      "grad_norm": 0.17835170030593872,
+      "learning_rate": 1.6082474226804127e-05,
+      "loss": 0.2774,
+      "step": 268
+    },
+    {
+      "epoch": 0.5536403395935169,
+      "grad_norm": 0.1777082234621048,
+      "learning_rate": 1.6059564719358535e-05,
+      "loss": 0.2726,
+      "step": 269
+    },
+    {
+      "epoch": 0.5556984821198868,
+      "grad_norm": 0.18766450881958008,
+      "learning_rate": 1.6036655211912944e-05,
+      "loss": 0.2879,
+      "step": 270
+    },
+    {
+      "epoch": 0.5577566246462567,
+      "grad_norm": 0.1868186593055725,
+      "learning_rate": 1.6013745704467357e-05,
+      "loss": 0.2808,
+      "step": 271
+    },
+    {
+      "epoch": 0.5598147671726267,
+      "grad_norm": 0.16695882380008698,
+      "learning_rate": 1.5990836197021766e-05,
+      "loss": 0.2668,
+      "step": 272
+    },
+    {
+      "epoch": 0.5618729096989966,
+      "grad_norm": 0.17224495112895966,
+      "learning_rate": 1.5967926689576178e-05,
+      "loss": 0.2682,
+      "step": 273
+    },
+    {
+      "epoch": 0.5639310522253667,
+      "grad_norm": 0.20116423070430756,
+      "learning_rate": 1.5945017182130587e-05,
+      "loss": 0.276,
+      "step": 274
+    },
+    {
+      "epoch": 0.5659891947517366,
+      "grad_norm": 0.19478343427181244,
+      "learning_rate": 1.5922107674684996e-05,
+      "loss": 0.2854,
+      "step": 275
+    },
+    {
+      "epoch": 0.5680473372781065,
+      "grad_norm": 0.20242950320243835,
+      "learning_rate": 1.5899198167239405e-05,
+      "loss": 0.2854,
+      "step": 276
+    },
+    {
+      "epoch": 0.5701054798044765,
+      "grad_norm": 0.19146093726158142,
+      "learning_rate": 1.5876288659793813e-05,
+      "loss": 0.2817,
+      "step": 277
+    },
+    {
+      "epoch": 0.5721636223308464,
+      "grad_norm": 0.1804896742105484,
+      "learning_rate": 1.5853379152348226e-05,
+      "loss": 0.2714,
+      "step": 278
+    },
+    {
+      "epoch": 0.5742217648572163,
+      "grad_norm": 0.19315646588802338,
+      "learning_rate": 1.5830469644902635e-05,
+      "loss": 0.2703,
+      "step": 279
+    },
+    {
+      "epoch": 0.5762799073835864,
+      "grad_norm": 0.1910266876220703,
+      "learning_rate": 1.5807560137457047e-05,
+      "loss": 0.2728,
+      "step": 280
+    },
+    {
+      "epoch": 0.5783380499099563,
+      "grad_norm": 0.20330773293972015,
+      "learning_rate": 1.5784650630011456e-05,
+      "loss": 0.2717,
+      "step": 281
+    },
+    {
+      "epoch": 0.5803961924363262,
+      "grad_norm": 0.19080683588981628,
+      "learning_rate": 1.5761741122565865e-05,
+      "loss": 0.2679,
+      "step": 282
+    },
+    {
+      "epoch": 0.5824543349626962,
+      "grad_norm": 0.18052135407924652,
+      "learning_rate": 1.5738831615120277e-05,
+      "loss": 0.2815,
+      "step": 283
+    },
+    {
+      "epoch": 0.5845124774890661,
+      "grad_norm": 0.1998361051082611,
+      "learning_rate": 1.5715922107674686e-05,
+      "loss": 0.2888,
+      "step": 284
+    },
+    {
+      "epoch": 0.586570620015436,
+      "grad_norm": 0.1978764683008194,
+      "learning_rate": 1.5693012600229095e-05,
+      "loss": 0.2926,
+      "step": 285
+    },
+    {
+      "epoch": 0.5886287625418061,
+      "grad_norm": 0.17189203202724457,
+      "learning_rate": 1.5670103092783507e-05,
+      "loss": 0.2674,
+      "step": 286
+    },
+    {
+      "epoch": 0.590686905068176,
+      "grad_norm": 0.1937166303396225,
+      "learning_rate": 1.5647193585337916e-05,
+      "loss": 0.2838,
+      "step": 287
+    },
+    {
+      "epoch": 0.5927450475945459,
+      "grad_norm": 0.18978627026081085,
+      "learning_rate": 1.5624284077892328e-05,
+      "loss": 0.273,
+      "step": 288
+    },
+    {
+      "epoch": 0.5948031901209159,
+      "grad_norm": 0.17718705534934998,
+      "learning_rate": 1.5601374570446737e-05,
+      "loss": 0.2842,
+      "step": 289
+    },
+    {
+      "epoch": 0.5968613326472858,
+      "grad_norm": 0.1912536770105362,
+      "learning_rate": 1.5578465063001146e-05,
+      "loss": 0.2736,
+      "step": 290
+    },
+    {
+      "epoch": 0.5989194751736557,
+      "grad_norm": 0.18104907870292664,
+      "learning_rate": 1.555555555555556e-05,
+      "loss": 0.274,
+      "step": 291
+    },
+    {
+      "epoch": 0.6009776177000258,
+      "grad_norm": 0.1620381772518158,
+      "learning_rate": 1.5532646048109967e-05,
+      "loss": 0.2663,
+      "step": 292
+    },
+    {
+      "epoch": 0.6030357602263957,
+      "grad_norm": 0.17973916232585907,
+      "learning_rate": 1.5509736540664376e-05,
+      "loss": 0.2791,
+      "step": 293
+    },
+    {
+      "epoch": 0.6050939027527656,
+      "grad_norm": 0.16821186244487762,
+      "learning_rate": 1.548682703321879e-05,
+      "loss": 0.2787,
+      "step": 294
+    },
+    {
+      "epoch": 0.6071520452791356,
+      "grad_norm": 0.18426693975925446,
+      "learning_rate": 1.5463917525773197e-05,
+      "loss": 0.2886,
+      "step": 295
+    },
+    {
+      "epoch": 0.6092101878055055,
+      "grad_norm": 0.19796033203601837,
+      "learning_rate": 1.5441008018327606e-05,
+      "loss": 0.268,
+      "step": 296
+    },
+    {
+      "epoch": 0.6112683303318754,
+      "grad_norm": 0.1971343755722046,
+      "learning_rate": 1.541809851088202e-05,
+      "loss": 0.2761,
+      "step": 297
+    },
+    {
+      "epoch": 0.6133264728582455,
+      "grad_norm": 0.17458567023277283,
+      "learning_rate": 1.5395189003436427e-05,
+      "loss": 0.2831,
+      "step": 298
+    },
+    {
+      "epoch": 0.6153846153846154,
+      "grad_norm": 0.17610400915145874,
+      "learning_rate": 1.537227949599084e-05,
+      "loss": 0.2691,
+      "step": 299
+    },
+    {
+      "epoch": 0.6174427579109854,
+      "grad_norm": 0.1929042488336563,
+      "learning_rate": 1.534936998854525e-05,
+      "loss": 0.2847,
+      "step": 300
+    },
+    {
+      "epoch": 0.6174427579109854,
+      "eval_loss": 0.2959522604942322,
+      "eval_runtime": 2428.6339,
+      "eval_samples_per_second": 3.201,
+      "eval_steps_per_second": 0.8,
+      "step": 300
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 970,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 9.916186406277612e+17,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee55a11f5a306eb7c39b536fdfe2459bc279e468da50f6adda478c4deffcb812
+size 5688

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: unsloth/codegemma-7b-it
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.0

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "unsloth/codegemma-7b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "q_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c28d1150755988379f75a873814aff4bb698c6bfe61017321740de287540011d
+size 800116456

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8eaaabccd9eef6be0dbad53b8764bdb248322edd314f59dc0c08eeaa8b6aaea0
+size 406743860

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6d4fbb4eb2a12b17b73babfa38b90bdafc73a682a0af60498a1e002b1fd5b3a
+size 14244

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4268342c3fe77dd38c7543ef5b2cdacbd82e8a3bb5c5cae6fb040e599526f0e
+size 1064

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d964a2c8346d40f95791533eae48730d5f163c2e65fd16333560fd3e661df318
+size 34362915

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:583f2ebd2a1936009b7da991ea255504db68c7a9713a78673d1335a87098966c
+size 4241023

codegemma_instruct_cot_ft_lora_r64_alpha64/checkpoint-400/tokenizer_config.json

@@ -0,0 +1,2011 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<eos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<bos>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "5": {
+      "content": "<2mass>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "6": {
+      "content": "[@BOS@]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "7": {
+      "content": "<unused0>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "8": {
+      "content": "<unused1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "9": {
+      "content": "<unused2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "10": {
+      "content": "<unused3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "11": {
+      "content": "<unused4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "12": {
+      "content": "<unused5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "13": {
+      "content": "<unused6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "14": {
+      "content": "<unused7>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "15": {
+      "content": "<unused8>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "16": {
+      "content": "<unused9>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "17": {
+      "content": "<unused10>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "18": {
+      "content": "<unused11>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "19": {
+      "content": "<unused12>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "20": {
+      "content": "<unused13>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "21": {
+      "content": "<unused14>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "22": {
+      "content": "<unused15>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "23": {
+      "content": "<unused16>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "24": {
+      "content": "<unused17>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "25": {
+      "content": "<unused18>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "26": {
+      "content": "<unused19>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "27": {
+      "content": "<unused20>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "28": {
+      "content": "<unused21>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "29": {
+      "content": "<unused22>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30": {
+      "content": "<unused23>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "31": {
+      "content": "<unused24>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "32": {
+      "content": "<unused25>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "33": {
+      "content": "<unused26>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "34": {
+      "content": "<unused27>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "35": {
+      "content": "<unused28>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "36": {
+      "content": "<unused29>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "37": {
+      "content": "<unused30>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "38": {
+      "content": "<unused31>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "39": {
+      "content": "<unused32>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "40": {
+      "content": "<unused33>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "41": {
+      "content": "<unused34>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "42": {
+      "content": "<unused35>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "43": {
+      "content": "<unused36>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "44": {
+      "content": "<unused37>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "45": {
+      "content": "<unused38>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "46": {
+      "content": "<unused39>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "47": {
+      "content": "<unused40>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "48": {
+      "content": "<unused41>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "49": {
+      "content": "<unused42>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50": {
+      "content": "<unused43>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "51": {
+      "content": "<unused44>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "52": {
+      "content": "<unused45>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "53": {
+      "content": "<unused46>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "54": {
+      "content": "<unused47>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55": {
+      "content": "<unused48>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "56": {
+      "content": "<unused49>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "57": {
+      "content": "<unused50>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "58": {
+      "content": "<unused51>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "59": {
+      "content": "<unused52>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "60": {
+      "content": "<unused53>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "61": {
+      "content": "<unused54>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "62": {
+      "content": "<unused55>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "63": {
+      "content": "<unused56>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "64": {
+      "content": "<unused57>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "65": {
+      "content": "<unused58>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "66": {
+      "content": "<unused59>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "67": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "68": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "69": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "70": {
+      "content": "<|file_separator|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "71": {
+      "content": "<unused64>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "72": {
+      "content": "<unused65>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "73": {
+      "content": "<unused66>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "74": {
+      "content": "<unused67>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "75": {
+      "content": "<unused68>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "76": {
+      "content": "<unused69>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "77": {
+      "content": "<unused70>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "78": {
+      "content": "<unused71>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "79": {
+      "content": "<unused72>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "80": {
+      "content": "<unused73>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "81": {
+      "content": "<unused74>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "82": {
+      "content": "<unused75>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "83": {
+      "content": "<unused76>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "84": {
+      "content": "<unused77>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "85": {
+      "content": "<unused78>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "86": {
+      "content": "<unused79>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "87": {
+      "content": "<unused80>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "88": {
+      "content": "<unused81>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "89": {
+      "content": "<unused82>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "90": {
+      "content": "<unused83>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "91": {
+      "content": "<unused84>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "92": {
+      "content": "<unused85>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "93": {
+      "content": "<unused86>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "94": {
+      "content": "<unused87>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "95": {
+      "content": "<unused88>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "96": {
+      "content": "<unused89>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "97": {
+      "content": "<unused90>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "98": {
+      "content": "<unused91>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "99": {
+      "content": "<unused92>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "100": {
+      "content": "<unused93>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "101": {
+      "content": "<unused94>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "102": {
+      "content": "<unused95>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "103": {
+      "content": "<unused96>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "104": {
+      "content": "<unused97>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "105": {
+      "content": "<unused98>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "106": {
+      "content": "<start_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "107": {
+      "content": "<end_of_turn>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "108": {
+      "content": "\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "109": {
+      "content": "\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "110": {
+      "content": "\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "111": {
+      "content": "\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "112": {
+      "content": "\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "113": {
+      "content": "\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "114": {
+      "content": "\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "115": {
+      "content": "\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "116": {
+      "content": "\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "117": {
+      "content": "\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "118": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "119": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "120": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "121": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "122": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "123": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "124": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "125": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "126": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "127": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "128": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "129": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "130": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "131": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "132": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "133": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "134": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "135": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "136": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "137": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "138": {
+      "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "139": {
+      "content": "▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "140": {
+      "content": "▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "141": {
+      "content": "▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "142": {
+      "content": "▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "143": {
+      "content": "▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "144": {
+      "content": "▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "145": {
+      "content": "▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "146": {
+      "content": "▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "147": {
+      "content": "▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "148": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "149": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "150": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "152": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "153": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "154": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "155": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "156": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "157": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "158": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "159": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "160": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "161": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "162": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "163": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "164": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "165": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "166": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "167": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "168": {
+      "content": "▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "169": {
+      "content": "<table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "170": {
+      "content": "<caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "171": {
+      "content": "<thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "172": {
+      "content": "<tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "173": {
+      "content": "<tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "174": {
+      "content": "<tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "175": {
+      "content": "<th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "176": {
+      "content": "<td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "177": {
+      "content": "</table>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "178": {
+      "content": "</caption>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "179": {
+      "content": "</thead>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "180": {
+      "content": "</tbody>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "181": {
+      "content": "</tfoot>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "182": {
+      "content": "</tr>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "183": {
+      "content": "</th>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "184": {
+      "content": "</td>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "185": {
+      "content": "<h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "186": {
+      "content": "<h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "187": {
+      "content": "<h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "188": {
+      "content": "<h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "189": {
+      "content": "<h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "190": {
+      "content": "<h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "191": {
+      "content": "<blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "192": {
+      "content": "</h1>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "193": {
+      "content": "</h2>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "194": {
+      "content": "</h3>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "195": {
+      "content": "</h4>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "196": {
+      "content": "</h5>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "197": {
+      "content": "</h6>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "198": {
+      "content": "</blockquote>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "199": {
+      "content": "<strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "200": {
+      "content": "<em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "201": {
+      "content": "<b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "202": {
+      "content": "<i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "203": {
+      "content": "<u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "204": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "205": {
+      "content": "<sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "206": {
+      "content": "<sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "207": {
+      "content": "<code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "208": {
+      "content": "</strong>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "209": {
+      "content": "</em>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "210": {
+      "content": "</b>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "211": {
+      "content": "</i>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "212": {
+      "content": "</u>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "213": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "214": {
+      "content": "</sub>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "215": {
+      "content": "</sup>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "216": {
+      "content": "</code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255968": {
+      "content": "[toxicity=0]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255969": {
+      "content": "\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255970": {
+      "content": "\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255971": {
+      "content": "\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255972": {
+      "content": "\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255973": {
+      "content": "\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255974": {
+      "content": "\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255975": {
+      "content": "\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255976": {
+      "content": "\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255977": {
+      "content": "\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255978": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255979": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255980": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255981": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255982": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255983": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255984": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255985": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255986": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255987": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255988": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255989": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255990": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255991": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255992": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255993": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255994": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255995": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255996": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255997": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255998": {
+      "content": "\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "255999": {
+      "content": "<unused99>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "bos_token": "<bos>",
+  "chat_template": "{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<eos>",
+  "extra_special_tokens": {},
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}