Wow, amazing response time
I am impressed at how responsive you are to make the models available. Thank you!
I agree, again thank you so much to both of you at Unsloth, you are such amazing parts of this open source community.
It's not up yet, guys. Just be patient. 🤞😋
Hey guys we're still working on uploading them! Stay tuned!
the one featured by lmstudio is too large for my 16gb m1 pro.
but my PC 4070 12gb can chat on it just fine... patiently waiting for the upload to dl it!
@testosterones Throw this in your terminal. This tells Mac OS to make 14GB of unified memory accessible to you - you'll be able to run the 12.11 GB MXFP4.
sudo sysctl iogpu.wired_limit_mb=14336
It's uploaded now!! The FP4 version. Please update whichever inference engine youre using!
Dynamic GGUFs with different sizes will come later!! Thanks to llama.cpp if they update it.
CC: @AlexPradas @Joseph717171 @sandlercaleb @testosterones @adhish @TobDeBer @alt909 @owao @Metricon @drjabaka @Tom-Neverwinter
It's uploaded now!! The FP4 version. Please update whichever inference engine youre using!
Dynamic GGUFs with different sizes will come later!! Thanks to llama.cpp if they update it.
CC: @AlexPradas @Joseph717171 @sandlercaleb @testosterones @adhish @TobDeBer @alt909 @owao @Metricon @drjabaka @Tom-Neverwinter
@shimmyshimmer That a boy, Mikey!!! You and Dan fucking rock! 🚀 🚀