Update README.md
Browse files
README.md
CHANGED
|
@@ -1,11 +1,14 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
-
Coming soon! just learned about thebloke's quant issues, will update later.
|
| 5 |
|
|
|
|
| 6 |
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
-
|
| 9 |
|
| 10 |
|
| 11 |
|
|
@@ -22,6 +25,6 @@ doesn't work, the file might be named something else so I had success with
|
|
| 22 |
If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
|
| 23 |
|
| 24 |
|
| 25 |
-
Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems.
|
| 26 |
-
You just run it, open the chat interface in a browser, and interact.
|
| 27 |
-
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
| 4 |
|
| 5 |
+
This is a llamafile for [Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1).
|
| 6 |
|
| 7 |
+
These were converted and quantized from source safetensors using llama.cpp on April 3, 2024.
|
| 8 |
+
This matters because there are several GGUF files on HF which were created before llama.cpp's support for MoE quantization was fully debugged,
|
| 9 |
+
even though it looked like it was producing working files at the time.
|
| 10 |
|
| 11 |
+
I'll be uploading the quantized .gguf sources I created as well if anyone wants them as a reference or for further work.
|
| 12 |
|
| 13 |
|
| 14 |
|
|
|
|
| 25 |
If that fails too, just navigate to `/proc/sys/fs/binfmt_msc` and see what files look like `WSLInterop` and echo a -1 to whatever it's called by changing that part of the recommended command.
|
| 26 |
|
| 27 |
|
| 28 |
+
Llamafiles are a standalone executable that run an LLM server locally on a variety of operating systems including FreeBSD, Windows, Windows via WSL, Linux, and Mac.
|
| 29 |
+
The same file works everywhere. You just download, run it, open the chat interface in a browser, and interact. Options can be passed in to expose the api etc.
|
| 30 |
+
See their docs for details.
|