Mehdi Challakh
commited on
added latest evals
Browse files
README.md
CHANGED
|
@@ -101,13 +101,15 @@ print(answer)
|
|
| 101 |
|
| 102 |
## Performance
|
| 103 |
|
| 104 |
-
|
| 105 |
-
|
|
| 106 |
-
|
|
| 107 |
-
|
|
| 108 |
-
| ROUGE-
|
| 109 |
-
| ROUGE-
|
| 110 |
-
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
### Example
|
| 113 |
|
|
|
|
| 101 |
|
| 102 |
## Performance
|
| 103 |
|
| 104 |
+
| Metric | stable-code-instruct-3b | stable-cypher-instruct-3b |
|
| 105 |
+
| ------------ | ----------------------- | ------------------------- |
|
| 106 |
+
| BLEU-4 | 19.07 | 88.63 |
|
| 107 |
+
| ROUGE-1 | 39.49 | 95.09 |
|
| 108 |
+
| ROUGE-2 | 24.82 | 90.71 |
|
| 109 |
+
| ROUGE-L | 29.63 | 91.51 |
|
| 110 |
+
| Jaro-Winkler | 52.21 | 95.69 |
|
| 111 |
+
| Jaccard | 25.55 | 90.78 |
|
| 112 |
+
| Pass@1 | 0.0 | 51.8 |
|
| 113 |
|
| 114 |
### Example
|
| 115 |
|