CodyBontecou
/

llada-8b-instruct-duplicate

Text Generation

Model card Files Files and versions Community

CodyBontecou commited on Mar 8

Commit

c709ed6

·

1 Parent(s): bb6d908

device_map

Files changed (1) hide show

handler.py +1 -1

handler.py CHANGED Viewed

@@ -11,7 +11,7 @@ class EndpointHandler:
             low_cpu_mem_usage=True,
             use_flash_attn=False,
             trust_remote_code=True,
-            device_map=split_model(),
         ).eval()
         self.tokenizer = AutoTokenizer.from_pretrained(

             low_cpu_mem_usage=True,
             use_flash_attn=False,
             trust_remote_code=True,
+            device_map="auto",
         ).eval()
         self.tokenizer = AutoTokenizer.from_pretrained(