ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
Paper
•
2403.03853
•
Published
•
66
lshort-transformers
Papers useful when writing the paper: "The Not So Short Transfromers"
-
-
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Paper • 2401.15024 • Published • 73 -
Your Transformer is Secretly Linear
Paper • 2405.12250 • Published • 157 -
Yi: Open Foundation Models by 01.AI
Paper • 2403.04652 • Published • 65 -
Arcee's MergeKit: A Toolkit for Merging Large Language Models
Paper • 2403.13257 • Published • 21 -
The Unreasonable Ineffectiveness of the Deeper Layers
Paper • 2403.17887 • Published • 82 -
Weight subcloning: direct initialization of transformers using larger pretrained ones
Paper • 2312.09299 • Published • 18 -
Evolutionary Optimization of Model Merging Recipes
Paper • 2403.13187 • Published • 58 -
Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 15 -
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
Paper • 2312.15685 • Published • 16