pretrain core 2

README.md

@@ -255,20 +255,14 @@ Saving checkpoint to '../out/pretrain-core-1/final/lit_model.pth'
 ----------------------------------------
 ```
 
-Backup `wandb`:
-
 ```bash
 mv wandb wandb-pretrain-core-1
 ```
 
-Copy config:
-
 ```bash
 cp ../config-1.json ../out/pretrain-core-1/final/config.json
 ```
 
-Chat with model:
-
 ```bash
 CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt chat ../out/pretrain-core-1/final
 ```
@@ -280,3 +274,19 @@ litgpt convert_pretrained_checkpoint ../out/pretrain-core-1/final ../out/pretrai
 ```bash
 CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt pretrain --config pretrain_core_model_2.yaml
 ```
+
+```bash
+cp ../config-2.json ../out/pretrain-core-2/final/config.json
+```
+
+```bash
+mv wandb wandb-pretrain-core-2
+```
+
+```bash
+CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt chat ../out/pretrain-core-2/final
+```
+
+```bash
+litgpt convert_pretrained_checkpoint ../out/pretrain-core-2/final ../out/pretrain-core-2/checkpoint
+```