Hugging Face's logo

Xenova
/
mbart-large-50-many-to-many-mmt

Translation
Transformers.js
ONNX
mbart
text2text-generation
Model card Files
xet
Community
2
whitphx HF Staff commited on
Commit
ee64621
·
verified ·
1 Parent(s): 76db8be

Add/update the quantized ONNX model files and README.md for Transformers.js v3

Browse files

## Applied Quantizations

### ✅ Based on `decoder_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_bnb4.onnx` (added)

### ✅ Based on `decoder_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_bnb4.onnx` (added)

### ✅ Based on `encoder_model.onnx` *with* slimming

↳ ✅ `int8`: `encoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `encoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `encoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `encoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `encoder_model_bnb4.onnx` (added)

### ✅ Based on `encoder_model.onnx` *with* slimming

↳ ✅ `int8`: `encoder_model_int8.onnx` (added)
↳ ✅ `uint8`: `encoder_model_uint8.onnx` (added)
↳ ✅ `q4`: `encoder_model_q4.onnx` (added)
↳ ✅ `q4f16`: `encoder_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `encoder_model_bnb4.onnx` (added)

### ✅ Based on `decoder_with_past_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_with_past_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_with_past_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_with_past_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_with_past_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_with_past_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_with_past_model_bnb4.onnx` (added)

### ✅ Based on `decoder_with_past_model.onnx` *with* slimming

↳ ✅ `fp16`: `decoder_with_past_model_fp16.onnx` (added)
↳ ✅ `int8`: `decoder_with_past_model_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_with_past_model_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_with_past_model_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_with_past_model_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_with_past_model_bnb4.onnx` (added)

### ❌ Based on `decoder_model_merged.onnx` *with* slimming

```
0%| | 0/1 [00:00<?, ?it/s]
Processing /tmp/tmpod0fxf6l/decoder_model_merged.onnx: 0%| | 0/1 [00:00<?, ?it/s]

0%| | 0/6 [00:00<?, ?it/s]

- Quantizing to fp16: 0%| | 0/6 [00:00<?, ?it/s]/home/ubuntu/src/tjsmigration/transformers.js/scripts/float16.py:73: UserWarning: the float32 number 5.960464477539063e-08 will be truncated to 1e-07
warnings.warn(
/home/ubuntu/src/tjsmigration/transformers.js/scripts/float16.py:92: UserWarning: the float32 number -5.960464477539063e-08 will be truncated to -1e-07
warnings.warn(
/home/ubuntu/src/tjsmigration/transformers.js/scripts/float16.py:85: UserWarning: the float32 number -3.4028234663852886e+38 will be truncated to -10000.0
warnings.warn(

- Quantizing to fp16: 0%| | 0/6 [00:33<?, ?it/s]

Processing /tmp/tmpod0fxf6l/decoder_model_merged.onnx: 0%| | 0/1 [00:33<?, ?it/s]
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/quantize.py", line 377, in <module>
main()
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/quantize.py", line 374, in main
quantize(input_folder, output_folder, quantization_args)
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/quantize.py", line 309, in quantize
quantize_fp16(
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/quantize.py", line 223, in quantize_fp16
check_and_save_model(model_fp16, save_path)
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/utils.py", line 29, in check_and_save_model
strict_check_model(model)
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/utils.py", line 21, in strict_check_model
raise e
File "/home/ubuntu/src/tjsmigration/transformers.js/scripts/utils.py", line 16, in strict_check_model
onnx.checker.check_model(model_or_path, full_check=True)
File "/home/ubuntu/.cache/uv/archive-v0/iAncxVR1WPOl_8LkA6LpD/lib/python3.12/site-packages/onnx/checker.py", line 179, in check_model
C.check_model(
onnx.onnx_cpp2py_export.shape_inference.InferenceError: [ShapeInferenceError] Inference error(s): (op_type:If, node name: optimum::if): [ShapeInferenceError] Inference error(s): (op_type:Add, node name: /model/decoder/embed_positions/Add): [ShapeInferenceError] Inferred shape and existing shape differ in rank: (1) vs (0)
```

### ✅ Based on `decoder_model_merged.onnx` *without* slimming

↳ ✅ `fp16`: `decoder_model_merged_fp16.onnx` (replaced because it was invalid)
↳ ✅ `int8`: `decoder_model_merged_int8.onnx` (added)
↳ ✅ `uint8`: `decoder_model_merged_uint8.onnx` (added)
↳ ✅ `q4`: `decoder_model_merged_q4.onnx` (added)
↳ ✅ `q4f16`: `decoder_model_merged_q4f16.onnx` (added)
↳ ✅ `bnb4`: `decoder_model_merged_bnb4.onnx` (added)

Files changed (23) hide show
  1. onnx/decoder_model_bnb4.onnx +3 -0
  2. onnx/decoder_model_fp16.onnx +3 -0
  3. onnx/decoder_model_int8.onnx +3 -0
  4. onnx/decoder_model_merged_bnb4.onnx +3 -0
  5. onnx/decoder_model_merged_fp16.onnx +2 -2
  6. onnx/decoder_model_merged_int8.onnx +3 -0
  7. onnx/decoder_model_merged_q4.onnx +3 -0
  8. onnx/decoder_model_merged_q4f16.onnx +3 -0
  9. onnx/decoder_model_merged_uint8.onnx +3 -0
  10. onnx/decoder_model_q4.onnx +3 -0
  11. onnx/decoder_model_q4f16.onnx +3 -0
  12. onnx/decoder_model_uint8.onnx +3 -0
  13. onnx/decoder_with_past_model_bnb4.onnx +3 -0
  14. onnx/decoder_with_past_model_fp16.onnx +3 -0
  15. onnx/decoder_with_past_model_int8.onnx +3 -0
  16. onnx/decoder_with_past_model_q4.onnx +3 -0
  17. onnx/decoder_with_past_model_q4f16.onnx +3 -0
  18. onnx/decoder_with_past_model_uint8.onnx +3 -0
  19. onnx/encoder_model_bnb4.onnx +3 -0
  20. onnx/encoder_model_int8.onnx +3 -0
  21. onnx/encoder_model_q4.onnx +3 -0
  22. onnx/encoder_model_q4f16.onnx +3 -0
  23. onnx/encoder_model_uint8.onnx +3 -0
onnx/decoder_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37cce581620ed057b4977092a71c5fcce9abe04c2fd6725b0fd8e8e34a1bfbe0
3
+ size 1143154191
onnx/decoder_model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1335deade4152ae61bd602d0f578ad3db3c1af338fb1c4dd33a2d11cb5053e54
3
+ size 917858315
onnx/decoder_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b98e098e16726ab183e077c89fbcee30906fd0e62f9ef630bff91e5380acaeb
3
+ size 1484265286
onnx/decoder_model_merged_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12a50442ca6888b723820dfc5feddbb8ea1d1b144e15a3e3fa0e49021a2fd73c
3
+ size 1144741364
onnx/decoder_model_merged_fp16.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ba1d678e0d2d269d11ab52f33ed34d6750c330f08831292491dbdbc522ccff18
3
- size 919271293
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:effec0a0c6404c6022e96f6bb0284e51ba3cd22c0050da6960133890c97d0214
3
+ size 919279521
onnx/decoder_model_merged_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac97050c62f63eed714f25a994536798baa3cdf4dde706f55486b35120c6885f
3
+ size 462065376
onnx/decoder_model_merged_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64ef6a720b555d09ef0acd908a1973ad2f86bd498c9c313913006327103f69da
3
+ size 1157322440
onnx/decoder_model_merged_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b511be7839e37c898b166ceb927e9feb0e3389d49ecaf287400b1e52cef3889b
3
+ size 629887060
onnx/decoder_model_merged_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5732925cfd0bc387f710166587bde4be34733fe3766a726d4c400bb81e661467
3
+ size 462065432
onnx/decoder_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:874a768ddfc4175601df1d4ebd81eb8785d4949f500929615315108fcc47be34
3
+ size 1155736131
onnx/decoder_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f2f49c4686c03f91d4a28270df71e60f2e673d6a5aefe3026c9d3f52a8da394
3
+ size 628468938
onnx/decoder_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ccaad840b8f1f385e16c32745df5c9586a2b3cb853b284beb4bbc6cb6f00734
3
+ size 1484265342
onnx/decoder_with_past_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88eb6a4e85b73dd01dd4431e3d92b96f0a114f4eef6ca19375ddb0e2690973ac
3
+ size 1128766813
onnx/decoder_with_past_model_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ccb1d0d723b05537e12db3efaf45a648c1e4e93eae862756bdd28760ed5f5848
3
+ size 867357409
onnx/decoder_with_past_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6167c854ecc93bd8ff7c4d6a4cc80bb3b4278fa9c04dc6230f98a0a9535ab5b6
3
+ size 1458848057
onnx/decoder_with_past_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1cc7a08624c9ccd862a26cbc14730672d6db9bc5e28344746a1c94c560b0cbd7
3
+ size 1139776081
onnx/decoder_with_past_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46ffe1dd6836088b9136ad057493b3e664caef2a1f93d064890d11c6118eaa30
3
+ size 614140400
onnx/decoder_with_past_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fefbc3b2d5d3224bf31d46b60ee3bec5a74884df599c079018f94379f81e59d7
3
+ size 1458848097
onnx/encoder_model_bnb4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f38b43cf06ed36a8e37c53c27a4b454e80b377221d385cc77d49666c878ba0
3
+ size 1114250147
onnx/encoder_model_int8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b2b4b42cf28f08e16490f693d90c6def09bb488ce82304e049db5e1d21a79a8
3
+ size 409055216
onnx/encoder_model_q4.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce907480924a4a5c52d49f7403c747296428f9fb7524e916aed9a14e36ba3b9a
3
+ size 1123686743
onnx/encoder_model_q4f16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60fbed29d7239ac33fe51750f1682634e42b432fe1a3b8d3bfd5e4983344e24e
3
+ size 599710411
onnx/encoder_model_uint8.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a57d7d63cea888c05c55da60f247bf52cd4b476513f779f53d2ea7d1d0502338
3
+ size 409055255