dfurman
/

Llama-2-13B-Instruct-v0.2

Text Generation

Model card Files Files and versions

dfurman commited on Nov 17, 2023

Commit

169626f

·

1 Parent(s): 9444756

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -80,7 +80,7 @@ tokenizer = AutoTokenizer.from_pretrained(
 model = AutoModelForCausalLM.from_pretrained(
     config.base_model_name_or_path,
-    torch_dtype=torch.float16,
     device_map="auto",
     trust_remote_code=True,
 )

 model = AutoModelForCausalLM.from_pretrained(
     config.base_model_name_or_path,
+    torch_dtype=torch.bfloat16,
     device_map="auto",
     trust_remote_code=True,
 )