Low-Rank Adapters Meet Neural Architecture Search for LLM Compression
Paper • 2501.16372 • Published • 12
inference optimization
-
-
TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
Paper • 2501.16937 • Published • 8 -
Matryoshka Quantization
Paper • 2502.06786 • Published • 32 -
Identifying Sensitive Weights via Post-quantization Integral
Paper • 2503.01901 • Published • 8 -
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
Paper • 2503.01840 • Published • 10 -
SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models
Paper • 2503.07605 • Published • 66 -
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
Paper • 2503.07067 • Published • 33 -
Efficient Distillation of Classifier-Free Guidance using Adapters
Paper • 2503.07274 • Published • 5 -
RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectories
Paper • 2503.07699 • Published • 5 -
QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge
Paper • 2503.16709 • Published -
xKV: Cross-Layer SVD for KV-Cache Compression
Paper • 2503.18893 • Published • 5 -
LogQuant: Log-Distributed 2-Bit Quantization of KV Cache with Superior Accuracy Preservation
Paper • 2503.19950 • Published • 12 -
Optimal Stepsize for Diffusion Sampling
Paper • 2503.21774 • Published • 13 -
AdaptiVocab: Enhancing LLM Efficiency in Focused Domains through Lightweight Vocabulary Adaptation
Paper • 2503.19693 • Published • 76 -
Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
Paper • 2503.24377 • Published • 18 -
ShortV: Efficient Multimodal Large Language Models by Freezing Visual Tokens in Ineffective Layers
Paper • 2504.00502 • Published • 26 -
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking
Paper • 2504.03947 • Published • 4 -
HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference
Paper • 2504.05897 • Published • 21 -
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence
Paper • 2503.20533 • Published • 12 -
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
Paper • 2504.07891 • Published • 5 -
PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters
Paper • 2504.08791 • Published • 141 -
Adaptive Computation Pruning for the Forgetting Transformer
Paper • 2504.06949 • Published • 3 -
70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float
Paper • 2504.11651 • Published • 31 -
Learning Adaptive Parallel Reasoning with Language Models
Paper • 2504.15466 • Published • 44 -
MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
Paper • 2504.16083 • Published • 8 -
Taming the Titans: A Survey of Efficient LLM Inference Serving
Paper • 2504.19720 • Published • 12 -
A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency
Paper • 2505.01658 • Published • 41 -
ReplaceMe: Network Simplification via Depth Pruning and Transformer Block Linearization
Paper • 2505.02819 • Published • 26 -
RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
Paper • 2505.03005 • Published • 35 -
An Empirical Study of Qwen3 Quantization
Paper • 2505.02214 • Published • 26 -
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
Paper • 2505.12781 • Published • 2 -
QVGen: Pushing the Limit of Quantized Video Generative Models
Paper • 2505.11497 • Published • 4 -
Scaling Law for Quantization-Aware Training
Paper • 2505.14302 • Published • 79 -
WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
Paper • 2505.19427 • Published • 11 -
KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction
Paper • 2505.23416 • Published • 13 -
CLaSp: In-Context Layer Skip for Self-Speculative Decoding
Paper • 2505.24196 • Published • 12 -
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
Paper • 2506.00413 • Published • 10 -
DLP: Dynamic Layerwise Pruning in Large Language Models
Paper • 2505.23807 • Published • 4 -
POSS: Position Specialist Generates Better Draft for Speculative Decoding
Paper • 2506.03566 • Published • 6 -
Inference-Time Hyper-Scaling with KV Cache Compression
Paper • 2506.05345 • Published • 31 -
PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models
Paper • 2506.16054 • Published • 60 -
SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
Paper • 2506.18349 • Published • 13 -
Scaling Speculative Decoding with Lookahead Reasoning
Paper • 2506.19830 • Published • 13 -
LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs
Paper • 2506.21862 • Published • 36 -
VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
Paper • 2506.22694 • Published • 3 -
ZeCO: Zero Communication Overhead Sequence Parallelism for Linear Attention
Paper • 2507.01004 • Published • 10 -
OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decoding
Paper • 2507.02659 • Published • 17 -
any4: Learned 4-bit Numeric Representation for LLMs
Paper • 2507.04610 • Published • 8 -
Beyond Context Limits: Subconscious Threads for Long-Horizon Reasoning
Paper • 2507.16784 • Published • 125 -
LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization
Paper • 2507.15758 • Published • 35 -
The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
Paper • 2507.18553 • Published • 42 -
Representation Shift: Unifying Token Compression with FlashAttention
Paper • 2508.00367 • Published • 16 -
LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
Paper • 2508.02215 • Published • 12 -
MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMs
Paper • 2508.05257 • Published • 13 -
XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
Paper • 2508.10395 • Published • 42 -
Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
Paper • 2508.09834 • Published • 53 -
Metis: Training Large Language Models with Advanced Low-Bit Quantization
Paper • 2509.00404 • Published • 7 -
Set Block Decoding is a Language Model Inference Accelerator
Paper • 2509.04185 • Published • 54 -
Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs
Paper • 2509.11177 • Published • 3 -
OBS-Diff: Accurate Pruning For Diffusion Models in One-Shot
Paper • 2510.06751 • Published • 22 -
BitNet Distillation
Paper • 2510.13998 • Published • 62 -
Direct Multi-Token Decoding
Paper • 2510.11958 • Published • 9 -
Glance: Accelerating Diffusion Models with 1 Sample
Paper • 2512.02899 • Published • 30 -
ECO: Quantized Training without Full-Precision Master Weights
Paper • 2601.22101 • Published • 6