File size: 369 Bytes
60cf301 171ad16 |
1 2 3 4 5 6 7 8 9 |
---
tags:
- coreml
---
Example commmand to run:
```
python -m src.coreml_example --model-path-mf ./Llama-3.2-1B-4bits-MF.mlmodelc --model-path-head ./Llama-3.2-1B-HEAD-6Bits.mlmodelc/ --model-path-emb ./Llama-3.2-1B-EMB-16Bits.mlmodelc --prompt "Write a poem joke about Harry Potter" --max-tokens 400 --temp 1.2
```
You need to have set HF_TOKEN or provide on execution |