|
|
--- |
|
|
language: |
|
|
- ko |
|
|
- en |
|
|
base_model: |
|
|
- meta-llama/Llama-3.2-1B |
|
|
--- |
|
|
> @ 2024.10.07 Model [torchtorchkimtorch/Llama-3.2-Korean-GGACHI-1B-Instruct-v1](https://huggingface.co/torchtorchkimtorch/Llama-3.2-Korean-GGACHI-1B-Instruct-v1) Released! |
|
|
|
|
|
> @ 2024.10.18 Performance for KOBEST of Llama-3.2-Korean-GGACHI-1B-Instruct-v1 has been updated! |
|
|
|
|
|
|
|
|
|
|
|
# **Llama-3.2-Korean-GGACHI-1B-Instruct-v1** # |
|
|
 |
|
|
## 모델 설명 (Model Description) |
|
|
|
|
|
GGACHI-1B-Instruct-v1는 Llama-3.2-1B-Instruct 모델을 기반으로 하는 한국어 태스크 수행에 최적화된 instruction-tuned 언어 모델입니다. 230,000개 이상의 고품질 한국어 데이터셋을 사용하여 fine-tuning되었습니다. |
|
|
|
|
|
GGACHI-1B-Instruct-v1 is an instruction-tuned language model optimized for Korean language tasks, based on the Llama-3.2-1B-Instruct model. It has been fine-tuned using over 230,000 high-quality Korean language datasets. |
|
|
|
|
|
## 모델 성능 (Model Performance) |
|
|
|
|
|
|
|
|
#### - 0 shot #### |
|
|
<table style="width:100%; text-align:center; border-collapse:collapse;"> |
|
|
<thead> |
|
|
<tr> |
|
|
<th style="border:1px solid black;">Task</th> |
|
|
<th style="border:1px solid black;">Model</th> |
|
|
<th style="border:1px solid black;">Accuracy</th> |
|
|
</tr> |
|
|
</thead> |
|
|
<tbody> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_boolq</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;"><strong>0.502</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.502</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_copa</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.504</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.521</strong></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.358</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.380</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.476</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.594</strong></td> |
|
|
</tr> |
|
|
</tbody> |
|
|
</table> |
|
|
|
|
|
#### - 5 shot #### |
|
|
<table style="width:100%; text-align:center; border-collapse:collapse;"> |
|
|
<thead> |
|
|
<tr> |
|
|
<th style="border:1px solid black;">Task</th> |
|
|
<th style="border:1px solid black;">Model</th> |
|
|
<th style="border:1px solid black;">Accuracy</th> |
|
|
</tr> |
|
|
</thead> |
|
|
<tbody> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_boolq</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;"><strong>0.571</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;">0.565</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_copa</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.526</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.549</strong></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.364</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.398</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.725</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.795</strong></td> |
|
|
</tr> |
|
|
</tbody> |
|
|
</table> |
|
|
|
|
|
#### - 10 shot #### |
|
|
<table style="width:100%; text-align:center; border-collapse:collapse;"> |
|
|
<thead> |
|
|
<tr> |
|
|
<th style="border:1px solid black;">Task</th> |
|
|
<th style="border:1px solid black;">Model</th> |
|
|
<th style="border:1px solid black;">Accuracy</th> |
|
|
</tr> |
|
|
</thead> |
|
|
<tbody> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_boolq</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;"><strong>0.593</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;">0.571</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_copa</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.525</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.549</strong></td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_hellaswag</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.356</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.394</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td rowspan="2" style="border:1px solid black;">kobest_sentineg</td> |
|
|
<td style="border:1px solid black;">Llama-3.2-1B-Instruct</td> |
|
|
<td style="border:1px solid black;">0.768</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td style="border:1px solid black;"><strong>GGACHI</strong></td> |
|
|
<td style="border:1px solid black;"><strong>0.821</strong></td> |
|
|
</tr> |
|
|
</tbody> |
|
|
</table> |
|
|
|
|
|
## Contact |
|
|
- **김민혁(Minhyuk Kim)** |
|
|
Mail: [email protected] |
|
|
LinkedIn : https://www.linkedin.com/in/mhkim0929/ |
