ibm-granite
/

granite-4.0-tiny-base-preview

Text Generation

granitemoehybrid

Model card Files Files and versions

ranarag commited on May 1

Commit

090f3e3

·

verified ·

1 Parent(s): adf4329

changing max_position_embeddings

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -65,7 +65,7 @@
   "mamba_n_groups": 1,
   "mamba_n_heads": 48,
   "mamba_proj_bias": false,
-  "max_position_embeddings": 4096,
   "model_type": "granitemoehybrid",
   "normalization_function": "rmsnorm",
   "num_attention_heads": 12,

   "mamba_n_groups": 1,
   "mamba_n_heads": 48,
   "mamba_proj_bias": false,
+  "max_position_embeddings": 131072,
   "model_type": "granitemoehybrid",
   "normalization_function": "rmsnorm",
   "num_attention_heads": 12,