eliebak HF Staff commited on
Commit
9156e79
·
verified ·
1 Parent(s): f712975

add rope_type:yarn

Browse files

I think this should be rope_type:"yarn" and not rope_type:"original" from what i see in the sglang modeling code + the use of beta_fast and beta_slow

Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -26,7 +26,7 @@
26
  "global_attn_every_n": 1,
27
  "model_type": "git",
28
  "torch_dtype": "bfloat16",
29
- "rope_type": "original",
30
  "original_max_position_embeddings": 8192,
31
  "scaling_factor": 16.0,
32
  "extrapolation_factor": 1.0,
 
26
  "global_attn_every_n": 1,
27
  "model_type": "git",
28
  "torch_dtype": "bfloat16",
29
+ "rope_type": "yarn",
30
  "original_max_position_embeddings": 8192,
31
  "scaling_factor": 16.0,
32
  "extrapolation_factor": 1.0,