Update README.md

README.md

@@ -1,5 +1,5 @@
 ---
-license: apache-2.0
+license: mit
 base_model: mistralai/Mistral-7B-v0.1
 tags:
 - generated_from_trainer
@@ -8,27 +8,35 @@ model-index:
   results: []
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
-# supercot-lora
+# mistral-v0.1-supercot-lora
 
-This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
+This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the [supercot](https://huggingface.co/datasets/kaiokendev/SuperCOT-dataset) dataset.
 It achieves the following results on the evaluation set:
 - Loss: 0.9790
 
 ## Model description
 
-More information needed
+SuperCOT is a LoRA trained with the aim of making Mistral follow prompts for Langchain better, by infusing chain-of-thought datasets, code explanations and instructions, snippets, logical deductions and Alpaca GPT-4 prompts. It uses a mixture of the following datasets:
+
+https://huggingface.co/datasets/QingyiSi/Alpaca-CoT
+- Chain of thought QED
+- Chain of thought Aqua
+- CodeAlpaca
+  
+https://huggingface.co/datasets/neulab/conala
+- Code snippets
+  
+https://huggingface.co/datasets/yahma/alpaca-cleaned
+- Alpaca GPT4
 
 ## Intended uses & limitations
 
-More information needed
+The model will show biases similar to those exhibited by the base model. It is not intended for supplying factual information or advice in any form.
 
 ## Training and evaluation data
 
-More information needed
+[kaiokendev/SuperCOT-dataset](https://huggingface.co/datasets/kaiokendev/SuperCOT-dataset)
 
 ## Training procedure
 
@@ -108,3 +116,37 @@ The following hyperparameters were used during training:
 - Pytorch 2.0.1+cu118
 - Datasets 2.14.5
 - Tokenizers 0.14.0
+
+### Citations
+
+Alpaca COT datasets
+```
+@misc{alpaca-cot,
+  author = {Qingyi Si, Zheng Lin },
+  school = {Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China},
+  title = {Alpaca-CoT: An Instruction Fine-Tuning Platform with Instruction Data Collection and Unified Large Language Models Interface},
+  year = {2023},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/PhoebusSi/alpaca-CoT}},
+}
+```
+Stanford Alpaca
+```
+@misc{alpaca,
+  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
+  title = {Stanford Alpaca: An Instruction-following LLaMA model},
+  year = {2023},
+  publisher = {GitHub},
+  journal = {GitHub repository},
+  howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
+}
+```
+Google FLAN
+```
+@inproceedings{weifinetuned,
+  title={Finetuned Language Models are Zero-Shot Learners},
+  author={Wei, Jason and Bosma, Maarten and Zhao, Vincent and Guu, Kelvin and Yu, Adams Wei and Lester, Brian and Du, Nan and Dai, Andrew M and Le, Quoc V},
+  booktitle={International Conference on Learning Representations}
+}
+```