File size: 2,180 Bytes
19e4244 cef553a 9a69816 cef553a 2b6da15 596ae62 2b6da15 596ae62 3dc6644 4f44a02 3dc6644 4f44a02 4dbc985 8390b09 4f44a02 ed21eff 4f44a02 0047dd8 4f44a02 ed21eff 4f44a02 596ae62 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: apache-2.0
---
### DISTILBERT RUNNING ON [DEEPSPARSE](https://github.com/neuralmagic/deepsparse) GOES BRHMMMMMMMM. 🚀🚀🚀
This model is 👇
███████╗ ██████╗ █████╗ ██████╗ ███████╗ ███████╗
██╔════╝ ██╔══██╗ ██╔══██╗ ██╔══██╗ ██╔════╝ ██╔════╝
███████╗ ██████╔╝ ███████║ ██████╔╝ ███████╗ █████╗
╚════██║ ██╔═══╝ ██╔══██║ ██╔══██╗ ╚════██║█ █╔══╝
███████║ ██║ ██║ ██║ ██║ ██ ║███████║ ███████╗
╚══════╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝ ╚═ ╝╚══════╝ ╚══════╝
### LOOKS LIKE THIS 👇
### Inference endpoints, outside of outliers (4ms) is avg. latency on 2 vCPUs:
### Handler for access to inference endpoints
```python
class EndpointHandler:
def __init__(self, path=""):
self.pipeline = Pipeline.create(task="text-classification", model_path=path)
def __call__(self, data: Dict[str, Any]) -> Dict[str, str]:
"""
Args:
data (:obj:): prediction input text
"""
inputs = data.pop("inputs", data)
start = perf_counter()
prediction = self.pipeline(inputs)
end = perf_counter()
latency = end - start
return {
"labels": prediction.labels,
"scores": prediction.scores,
"latency (secs.)": latency
}
```
̷͈̍
̵̳͒R̶̙̓i̸̟͘c̴̻̆k̸̑͜ÿ̷̳́
̸̪̚
̷͖̀ |