Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -325,7 +325,8 @@ torchrun --nproc_per_node {number of gpus} \
|
|
| 325 |
--target_modules q_proj k_proj v_proj o_proj \
|
| 326 |
--start_layer 8 \
|
| 327 |
--head_multi True \
|
| 328 |
-
--head_type simple
|
|
|
|
| 329 |
```
|
| 330 |
|
| 331 |
Our rerankers are initialized from [google/gemma-2b](https://huggingface.co/google/gemma-2b) (for llm-based reranker) and [openbmb/MiniCPM-2B-dpo-bf16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) (for llm-based layerwise reranker), and we train it on a mixture of multilingual datasets:
|
|
|
|
| 325 |
--target_modules q_proj k_proj v_proj o_proj \
|
| 326 |
--start_layer 8 \
|
| 327 |
--head_multi True \
|
| 328 |
+
--head_type simple \
|
| 329 |
+
--lora_extra_parameters linear_head
|
| 330 |
```
|
| 331 |
|
| 332 |
Our rerankers are initialized from [google/gemma-2b](https://huggingface.co/google/gemma-2b) (for llm-based reranker) and [openbmb/MiniCPM-2B-dpo-bf16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) (for llm-based layerwise reranker), and we train it on a mixture of multilingual datasets:
|