recursal
/

QRWKV6-32B-Instruct-Preview-v0.1

Text Generation

Model card Files Files and versions

SmerkyG commited on Sep 16

Commit

864bad4

·

verified ·

1 Parent(s): 1161365

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -20,7 +20,7 @@
   "model_type": "rwkv6qwen2",
   "num_attention_heads": 40,
   "num_hidden_layers": 64,
-  "lora_rank_tokenshift": 160,
   "lora_rank_decay": 128,
   "num_key_value_heads": 8,
   "rms_norm_eps": 1e-05,

   "model_type": "rwkv6qwen2",
   "num_attention_heads": 40,
   "num_hidden_layers": 64,
+  "lora_rank_tokenshift": 64,
   "lora_rank_decay": 128,
   "num_key_value_heads": 8,
   "rms_norm_eps": 1e-05,