MoritzMo123/kfac-svd_uniform_llama-3-8b_0.6
Text Generation • 5B • Updated • 6
LEMS & KFAC-SVD Compression
This collection hosts the compressed models evaluated in our paper: "Layer-wise Error Modeling Search (LEMS) and KFAC-SVD".
Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.7
Text Generation • 6B • Updated • 6Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.8
Text Generation • 7B • Updated • 8Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-3-8b_0.9
Text Generation • 7B • Updated • 4Note Llama-3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.6
Text Generation • 5B • Updated • 3Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.7
Text Generation • 6B • Updated • 4Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.8
Text Generation • 7B • Updated • 3Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-3-8b_0.9
Text Generation • 7B • Updated • 4Note Llama-3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.6
Text Generation • 5B • Updated • 3Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.7
Text Generation • 6B • Updated • 2Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.8
Text Generation • 7B • Updated • 3Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_Qwen3-8B_0.9
Text Generation • 7B • Updated • 3Note Qwen3-8B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.6
Text Generation • 5B • Updated • 52Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.7
Text Generation • 6B • Updated • 19Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.8
Text Generation • 7B • Updated • 6Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_Qwen3-8B_0.9
Text Generation • 7B • Updated • 3Note Qwen3-8B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.6
Text Generation • 4B • Updated • 4Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.7
Text Generation • 5B • Updated • 3Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.8
Text Generation • 6B • Updated • 4Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_mistral-7b_0.9
Text Generation • 7B • Updated • 3Note Mistral-7B compressed with KFAC-SVD and UNIFORM rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.6
Text Generation • 4B • Updated • 4Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.7
Text Generation • 5B • Updated • 7Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (70% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.8
Text Generation • 6B • Updated • 3Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_mistral-7b_0.9
Text Generation • 7B • Updated • 2Note Mistral-7B compressed with KFAC-SVD and LEMS rank allocation (90% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.6
Text Generation • 4B • Updated • 3 • 1Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_uniform_llama-2-7b_0.8
Text Generation • 5B • Updated • 3Note Llama-2-7B compressed with KFAC-SVD and UNIFORM rank allocation (80% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.6
Text Generation • 4B • Updated • 3 • 1Note Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (60% of total linear parameters remaining), without fine-tuning.
MoritzMo123/kfac-svd_lems_llama-2-7b_0.8
Text Generation • 5B • Updated • 9Note Llama-2-7B compressed with KFAC-SVD and LEMS rank allocation (80% of total linear parameters remaining), without fine-tuning.