mlc-ai
/

gpt2-medium-q0f16-MLC

Model card Files Files and versions

ruihanglai commited on Apr 15, 2024

Commit

0a9d82b

·

verified ·

1 Parent(s): 35c9547

Update mlc-chat-config.json

Files changed (1) hide show

mlc-chat-config.json +2 -1

mlc-chat-config.json CHANGED Viewed

@@ -11,7 +11,8 @@
     "context_window_size": 1024,
     "prefill_chunk_size": 1024,
     "scale_attn_by_inverse_layer_idx": false,
-    "tensor_parallel_shards": 1
   },
   "vocab_size": 50257,
   "context_window_size": 1024,

     "context_window_size": 1024,
     "prefill_chunk_size": 1024,
     "scale_attn_by_inverse_layer_idx": false,
+    "tensor_parallel_shards": 1,
+    "max_batch_size": 80
   },
   "vocab_size": 50257,
   "context_window_size": 1024,