junnei
/

Phi-4-multimodal-instruct-ko-asr

Automatic Speech Recognition

text-generation

Model card Files Files and versions Community

junnei commited on Mar 12

Commit

dbf7b1f

·

verified ·

1 Parent(s): c65944b

Update README.md

Files changed (1) hide show

README.md +8 -7

README.md CHANGED Viewed

@@ -81,11 +81,12 @@ Script is retrieved from [here](https://gist.github.com/seastar105/d1d8983b27611
 Compared to [Phi-4-mm-inst-zeroth-kor](https://huggingface.co/seastar105/Phi-4-mm-inst-zeroth-kor) and  [Phi-4-multimodal-finetune-ko-speech](https://huggingface.co/daekeun-ml/Phi-4-multimodal-finetune-ko-speech), ASR is significantly improved.
-| Model                | zeroth-test | fleurs-ko2en | fleurs-ko2en-cot | fleurs-en2ko | fleurs-en2ko-cot |
-|----------------------|-------------|--------------|------------------|--------------|------------------|
-| original             |  198.32     | 5.63         | 2.42             | 6.86         | 4.17             |
-| daekeun-ml/Phi-4-multimodal-finetune-ko-speech|  3.80       | 7.03         | 7.04             | 12.50        | 9.54             |
-| seastar105/Phi-4-mm-inst-zeroth-kor |  7.02       | 7.07         | 9.19             | 13.08        | 9.35             |
-| ASR finetune (this model)|  1.31       | 7.46         | 6.24             | 12.15        | 8.91             |
-| + AST finetune with (AST)[https://huggingface.co/datasets/junnei/covost2]|  3.88       | 8.07         | 10.09            | 18.82        | 15.41             |

 Compared to [Phi-4-mm-inst-zeroth-kor](https://huggingface.co/seastar105/Phi-4-mm-inst-zeroth-kor) and  [Phi-4-multimodal-finetune-ko-speech](https://huggingface.co/daekeun-ml/Phi-4-multimodal-finetune-ko-speech), ASR is significantly improved.
+| Model                | zeroth-CER  | zeroth-WER | fleurs-ko2en | fleurs-ko2en-cot | fleurs-en2ko | fleurs-en2ko-cot |
+|----------------------|-------------|------------|--------------|------------------|--------------|------------------|
+| original             |  198.32     |    -      | 5.63         | 2.42             | 6.86         | 4.17             |
+| daekeun-ml/Phi-4-multimodal-finetune-ko-speech|  1.61       |     3.54     | 7.67         | 8.38             | 12.31        | 9.69             |
+| seastar105/Phi-4-mm-inst-zeroth-kor |  7.02       |     -      |7.07         | 9.19             | 13.08        | 9.35             |
+| **ASR finetune (this model)**|  **1.31**       |      -     |7.46         | 6.24             | 12.15        | 8.91             |
+| + 1 epoch finetune with [Covost-Ko](https://huggingface.co/datasets/junnei/covost2-ko)|  3.88       |   -       | **8.07**         | **10.09**            | **18.82**        | **15.41**             |
+| [**AST finetuned model**](https://huggingface.co/junnei/Phi-4-multimodal-instruct-ko-speech/tree/main)|  **1.77**       | **2.99**         | **8.01**             | **9.09**        | **17.09**             | **11.82** |