Commit
·
aa281eb
1
Parent(s):
bbfb757
Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ tags:
|
|
| 16 |
- neural-compressor
|
| 17 |
---
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
| 22 |
|
|
@@ -25,19 +25,34 @@ This int8 ONNX model is generated by [neural-compressor](https://github.com/inte
|
|
| 25 |
python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
|
| 26 |
```
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
|
| 31 |
-
|
| 32 |
-
|
|
| 33 |
-
|
|
|
|
|
| 34 |
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
Download the model and script by cloning the repository:
|
| 39 |
```shell
|
| 40 |
git clone https://huggingface.co/Intel/gpt-j-6B-int8-static
|
| 41 |
```
|
| 42 |
|
| 43 |
-
Then you can do inference based on the model and script 'evaluation.ipynb'.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
- neural-compressor
|
| 17 |
---
|
| 18 |
|
| 19 |
+
## Model Details: INT8 GPT-J 6B
|
| 20 |
|
| 21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
| 22 |
|
|
|
|
| 25 |
python -m transformers.onnx --model=EleutherAI/gpt-j-6B onnx_gptj/ --framework pt --opset 13 --feature=causal-lm-with-past
|
| 26 |
```
|
| 27 |
|
| 28 |
+
| Model Detail | Description |
|
| 29 |
+
| ----------- | ----------- |
|
| 30 |
+
| Model Authors - Company | Intel |
|
| 31 |
+
| Date | April 10, 2022 |
|
| 32 |
+
| Version | 1 |
|
| 33 |
+
| Type | Text Generation |
|
| 34 |
+
| Paper or Other Resources | - |
|
| 35 |
+
| License | Apache 2.0 |
|
| 36 |
+
| Questions or Comments | [Community Tab](https://huggingface.co/Intel/gpt-j-6B-int8-static/discussions)|
|
| 37 |
|
| 38 |
+
| Intended Use | Description |
|
| 39 |
+
| ----------- | ----------- |
|
| 40 |
+
| Primary intended uses | You can use the raw model for text generation inference |
|
| 41 |
+
| Primary intended users | Anyone doing text generation inference |
|
| 42 |
+
| Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task. The model should not be used to intentionally create hostile or alienating environments for people.|
|
| 43 |
|
| 44 |
|
| 45 |
+
### How to use
|
| 46 |
|
| 47 |
Download the model and script by cloning the repository:
|
| 48 |
```shell
|
| 49 |
git clone https://huggingface.co/Intel/gpt-j-6B-int8-static
|
| 50 |
```
|
| 51 |
|
| 52 |
+
Then you can do inference based on the model and script 'evaluation.ipynb'.
|
| 53 |
+
|
| 54 |
+
## Metrics (Model Performance):
|
| 55 |
+
| Model | Model Size (GB) | Lambada Acc |
|
| 56 |
+
|---|:---:|:---:|
|
| 57 |
+
| FP32 |23|0.7954|
|
| 58 |
+
| INT8 |6|0.7944|
|