view post Post 3694 Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋GGUFs: unsloth/DeepSeek-V3.1-GGUFThe 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.Guide: https://docs.unsloth.ai/basics/deepseek-v3.1 See translation ❤️ 14 14 🔥 5 5 🚀 2 2 + Reply
view article Article Accelerate ND-Parallel: A Guide to Efficient Multi-GPU Training By siro1 and 4 others • 17 days ago • 52
huihui-ai/Huihui-gpt-oss-20b-BF16-abliterated Text Generation • 21B • Updated 17 days ago • 46.6k • 131