Update README.md

README.md

@@ -2,16 +2,23 @@
 license: cc-by-sa-4.0
 datasets:
 - bigcode/the-stack-dedup
+tags:
+- code
 ---
 
 
 # replit-code-v1-3b 
 
-`replit-code-v1-3b` is a 2.7B model. It is trained on the Stack Dedup v1.2 dataset.
+`replit-code-v1-3b` is a 2.7B Causal Language Model focused on Code Completion. The model has been trained on a subset of the Stack Dedup v1.2 dataset.
 
+The training mixture includes 20 different languages, listed here in descending order of number of tokens: 
+<br/>
+`Markdown`, `Java`, `JavaScript`, `Python`, `TypeScript`, `PHP`, `SQL`, `JSX`, `reStructuredText`, `Rust`, `C`, `CSS`, `Go`, `C++`, `HTML`, `Vue`, `Ruby`, `Jupyter Notebook`, `R`, `Shell`
 
+In total, the training dataset contains 175B tokens, which were repeated over 3 epochs -- in total, `replit-code-v1-3b` has been trained on 525B tokens (~195 tokens per parameter).
 
-## Model
+
+## How to use the model
 
 
 ```python
@@ -101,4 +108,4 @@ Coming soon.
 Coming soon.
 
 ## Model Hash
-5bc28ce32c6f9aec935ead7b60ea1c46
+5bc28ce32c6f9aec935ead7b60ea1c46