391 papers
Inference Optimization
0/391Speculative decoding, quantization at inference, and runtime tricks.
Progress0 of 391
2026
76- MayKernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernelscode2605.04956Tsinghua UniversityMay 6, 2026~132 min
- MayPiD: Fast and High-Resolution Latent Decoding with Pixel Diffusiondiffusion2605.23902NVIDIAMay 22, 2026~112 min
- MayUniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsificationinference-optimization2605.06221TencentMay 7, 2026~123 min
- MayFull Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Stepsinference-optimization2605.16928RTP-LLMMay 16, 2026~132 min
- MayOSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantizationinference-optimization2605.17757TogetherMay 18, 2026~114 min
- MayEnhancing Train-Free Infinite-Frame Generation for Consistent Long Videosinference-optimization2605.18233alibaba-incMay 18, 2026~99 min
- MayDECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devicesmoe2605.10933Tsinghua NLP GroupMay 11, 2026~105 min
- MayBEAM: Binary Expert Activation Masking for Dynamic Routing in MoEmoe2605.14438alibaba-incMay 14, 2026~112 min
- MayLLMs Improving LLMs: Agentic Discovery for Test-Time Scalingreasoning2605.08083GoogleMay 8, 2026~135 min
- MayRetrieval from Within: An Intrinsic Capability of Attention-Based Modelsretrieval2605.05806NVIDIAMay 7, 2026~119 min
- MayMeasuring Maximum Activations in Open Large Language Modelsserving2605.15572BAIDUMay 15, 2026~113 min
- MaySANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformervision2605.15178NVIDIAMay 14, 2026~113 min
- MayLiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMsvision2605.17260DeepmindMay 17, 2026~106 min
- MayFlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matchingvision2605.20910KAIST AIMay 20, 2026~133 min
- AprUniversal YOCO for Efficient Depth Scalingarchitecture2604.01220Microsoft ResearchApr 1, 2026score 9~105 min
- AprNemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningarchitecture2604.12374NVIDIAApr 14, 2026score 10~121 min
- AprAttention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigationcontext-optimization2604.10098LongCatApr 11, 2026score 9~127 min
- AprTriAttention: Efficient Long Reasoning with Trigonometric KV Compressioninference-optimization2604.04921NVIDIAApr 6, 2026~116 min
- AprDMax: Aggressive Parallel Decoding for dLLMsinference-optimization2604.08302National University of SingaporeApr 9, 2026~108 min
- AprAccelerating Speculative Decoding with Block Diffusion Draft Treesinference-optimization2604.12989Apr 14, 2026~107 min
- AprStochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharinginference-optimization2604.22782AppleApr 3, 2026~101 min
- AprQwen3.5-Omni Technical Reportmultimodal2604.15804Apr 17, 2026~123 min
- AprNemotron 3 Nano Omni: Efficient and Open Multimodal Intelligencemultimodal2604.24954NVIDIAApr 27, 2026~116 min
- AprMiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interactionmultimodal2604.27393OpenBMBApr 30, 2026~120 min
- AprAccelerating RL Post-Training Rollouts via System-Integrated Speculative Decodingrl-training2604.26779NVIDIAApr 29, 2026~133 min
- AprTrain-to-Test (T2) Scaling Laws: Integrating Pretraining and Test-Time Computescaling-laws2604.01411University of Wisconsin-MadisonApr 1, 2026score 9~122 min
- MarProact-VL: A Proactive VideoLLM for Real-Time AI Companionsagents2603.03447Microsoft ResearchMar 3, 2026score 9~94 min
- MarKernel-Smith: A Unified Recipe for Evolutionary Kernel Optimizationcode2603.28342NVIDIAMar 30, 2026~139 min
- MarOn-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformersdiffusion2603.28762Snap ResearchMar 30, 2026score 6~104 min
- MarREAD MOREinference-optimization2603.05451Together AIMar 5, 2026~113 min
- MarIndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuseinference-optimization2603.12201DeepSeekMar 12, 2026~129 min
- MarUnified Spatio-Temporal Token Scoring for Efficient Video VLMsinference-optimization2603.18004Ai2Mar 18, 2026score 9~114 min
- MarTAPS: Task Aware Proposal Distributions for Speculative Samplinginference-optimization2603.27027Image and Video Understanding LabMar 27, 2026score 9~125 min
- MarUnderstand and Accelerate Memory Processing Pipeline for Large Language Model Inferenceinference-optimization2603.29002Mar 30, 2026score 9~128 min
- MarSparse-BitNet: 1.58-bit LLMs are Naturally Friendly to Semi-Structured Sparsitylow-precision2603.05168Microsoft ResearchMar 5, 2026score 9~79 min
- MarMASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Modelsmultimodal2603.04800alibaba-incMar 5, 2026score 9~125 min
- MarDreamLite: A Lightweight On-Device Unified Model for Image Generation and Editingmultimodal2603.28713ByteDanceMar 30, 2026score 6~111 min
- MarHelios: Real Real-Time Long Video Generation Modelvision2603.04379NVIDIAMar 4, 2026score 9~109 min
- MarHiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoisingvision2603.08703Tencent HunyuanMar 9, 2026score 9~125 min
- FebVLS: Steering Pretrained Robot Policies via Vision-Language Modelsalignment2602.03973Ai2Feb 3, 2026score 2~100 min
- FebHySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharingarchitecture2602.03560Xiaomi MiMoFeb 3, 2026score 9~105 min
- FebParallel Track Transformers: Enabling Fast GPU Inference with Reduced Synchronizationarchitecture2602.07306AppleFeb 7, 2026~107 min
- FebRecurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoningarchitecture2602.07845Ai2Feb 8, 2026score 6~107 min
- FebMiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context Modelingarchitecture2602.11761Feb 12, 2026~104 min
- FebDr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generationscode2602.05885Feb 5, 2026~111 min
- FebArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillationdiffusion2602.09014Fudan UniversityFeb 9, 2026score 8~99 min
- FebSLA2: Sparse-Linear Attention with Learnable Routing and QATdiffusion2602.12675Feb 13, 2026~115 min
- FebFrom Scale to Speed: Adaptive Test-Time Scaling for Image Editingdiffusion2603.00141alibaba-incFeb 24, 2026score 6~106 min
- FebFASA: Frequency-aware Sparse Attentioninference-optimization2602.03152Feb 3, 2026~117 min
- FebLycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decodinginference-optimization2602.04541Feb 4, 2026~104 min
- FebDFlash: Block Diffusion for Flash Speculative Decodinginference-optimization2602.06036Feb 5, 2026~116 min
- FebDoes Your Reasoning Model Implicitly Know When to Stop Thinking?inference-optimization2602.08354ByteDanceFeb 9, 2026score 7~98 min
- FebSpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuninginference-optimization2602.13515Feb 13, 2026~108 min
- FebSPEED-Bench: A Unified and Diverse Benchmark for Speculative Decodinginference-optimization2604.09557NVIDIAFeb 10, 2026score 9~89 min
- FebSemantic Search At LinkedInllm-systems2602.07309LinkedInFeb 7, 2026~139 min
- FebWhen to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoningllm-systems2602.10560ByteDance SeedFeb 11, 2026score 8~103 min
- FebK-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Modelllm-systems2602.19128Feb 22, 2026~109 min
- FebUntied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunkingllm-systems2602.21196TogetherFeb 24, 2026score 9~105 min
- FebTest-Time Training with KV Binding Is Secretly Linear Attentionllm-systems2602.21204NVIDIAFeb 24, 2026score 9~131 min
- FebOmniMoE: An Efficient MoE by Orchestrating Atomic Experts at Scalemoe2602.05711Feb 5, 2026~115 min
- FebStep 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parametersmoe2602.10604StepFunFeb 11, 2026score 8~135 min
- FebDualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inferenceserving2602.21548DeepSeekFeb 25, 2026score 9~109 min
- FebLLaDA2.1: Speeding Up Text Diffusion via Token Editingtraining-methods2602.08676Feb 9, 2026~106 min
- FebCUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generationtraining-methods2602.24286ByteDance SeedFeb 27, 2026score 9~110 min
- FebSpanning the Visual Analogy Space with a Weight Basis of LoRAsvision2602.15727NVIDIAFeb 17, 2026score 2~110 min
- JanFast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planningagents2601.09708NVIDIAJan 14, 2026score 3~113 min
- JanFantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigationagents2601.13976Alibaba AMAP CV LabJan 20, 2026score 3~126 min
- JanFlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloningarchitecture2601.11141FlashLabsJan 16, 2026score 2~112 min
- JanQwen3-TTS Technical Reportarchitecture2601.15621Qwen / Alibaba CloudJan 22, 2026score 2~120 min
- JanScaling Embeddings Outperforms Scaling Experts in Language Modelsarchitecture2601.21204Meituan LongCatJan 29, 2026score 8~115 min
- JanConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocationarchitecture2601.21420ByteDance SeedJan 29, 2026score 3~103 min
- JanTransition Matching Distillation for Fast Video Generationinference-optimization2601.09881NVIDIAJan 14, 2026score 9~115 min
- JanFast KVzip: Efficient and Accurate LLM Inference with Gated KV Evictioninference-optimization2601.17668NAVER AI LabJan 25, 2026score 9~116 min
- JanEfficient Autoregressive Video Diffusion with Dummy Headinference-optimization2601.20499Microsoft ResearchJan 28, 2026score 7~106 min
- JanFP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learninglow-precision2601.18150NVIDIAJan 26, 2026score 9~102 min
- JanFlowAct-R1: Towards Interactive Humanoid Video Generationvision2601.10103ByteDanceJan 15, 2026score 3~101 min
2025
126- DecKernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Metaagents2512.23236Dec 29, 2025~128 min
- DecFast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingarchitecture2512.11130NVIDIADec 11, 2025score 4~108 min
- DecDynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Spacearchitecture2512.24617ByteDance SeedDec 31, 2025score 9~113 min
- DecImage Diffusion Preview with Consistency Solverdiffusion2512.13592DeepmindDec 15, 2025score 7~99 min
- DecKnot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animationdiffusion2512.21734TongyiLabDec 25, 2025score 9~120 min
- DecFour Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalinginference-optimization2512.02010Dec 1, 2025~116 min
- DecFast and Accurate Causal Parallel Decoding using Jacobi Forcinginference-optimization2512.14681Dec 16, 2025score 9~104 min
- DecDEER: Draft with Diffusion, Verify with Autoregressive Modelsinference-optimization2512.15176Dec 17, 2025score 10~125 min
- DecTurboDiffusion: Accelerating Video Diffusion Models by 100-200 Timesinference-optimization2512.16093University of California, BerkeleyDec 18, 2025score 10~99 min
- DecKascade: A Practical Sparse Attention Method for Long-Context LLM Inferenceinference-optimization2512.16391Dec 18, 2025~109 min
- DecReasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMsinference-optimization2512.17206alibaba-incDec 19, 2025score 9~117 min
- DecUnderstanding and Harnessing Sparsity in Unified Multimodal Modelsmoe2512.02351ByteDance SeedDec 2, 2025score 9~121 min
- DecSonicMoE: Accelerating MoE with IO and Tile-aware Optimizationsmoe2512.14080Dec 16, 2025~105 min
- DecSeedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Modelmultimodal2512.13507ByteDance SeedDec 15, 2025score 3~94 min
- DecCUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learningtraining-methods2512.02551NVIDIADec 2, 2025~96 min
- DecEfficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedtraining-methods2512.14067NVIDIADec 16, 2025score 9~107 min
- DecEnd-to-End Test-Time Training for Long Contexttraining-methods2512.23675Dec 29, 2025~120 min
- DecDivide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understandingvision2512.04000Microsoft ResearchDec 3, 2025score 8~110 min
- DecQuantile Rendering: Efficiently Embedding High-dimensional Feature on 3D Gaussian Splattingvision2512.20927NVIDIADec 24, 2025score 3~128 min
- NovWUSH: Near-Optimal Adaptive Transforms for LLM Quantizationalignment2512.00956 IST Austria Distributed Algorithms and Systems LabNov 30, 2025score 9~117 min
- NovTiDAR: Think in Diffusion, Talk in Autoregressionarchitecture2511.08923NVIDIANov 12, 2025score 9~120 min
- NovNemotron-Flash: Towards Latency-Optimal Hybrid Small Language Modelsarchitecture2511.18890NVIDIANov 24, 2025score 9~106 min
- NovMixLM: High-Throughput and Effective LLM Ranking via Text-Embedding Mix-Interactionarchitecture2512.07846Nov 25, 2025~117 min
- NovAccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimizationcode2511.15915Stanford UniversityNov 19, 2025~115 min
- NovKLASS: KL-Guided Fast Inference in Masked Diffusion Modelsinference-optimization2511.05664KAIST AINov 7, 2025score 8~114 min
- NovCDLM: Consistency Diffusion Language Models For Faster Samplinginference-optimization2511.19269Together AINov 24, 2025~102 min
- NovInferix: A Block-Diffusion based Next-Generation Inference Engine for World Simulationinference-optimization2511.20714DAMO AcademyNov 25, 2025score 9~100 min
- NovNemotron Elastic: Towards Efficient Many-in-One Reasoning LLMstraining-methods2511.16664NVIDIANov 20, 2025score 9~127 min
- NovSSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Spacetraining-methods2511.20102Nov 25, 2025score 9~129 min
- OctArtificial Hippocampus Networks for Efficient Long-Context Modelingarchitecture2510.07318ByteDance SeedOct 8, 2025score 9~123 min
- OctMoGA: Mixture-of-Groups Attention for End-to-End Long Video Generationarchitecture2510.18692ByteDanceOct 21, 2025score 9~116 min
- OctEvery Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoningarchitecture2510.19338Ant GroupOct 22, 2025score 10~141 min
- OctParallel Loop Transformer for Efficient Test-Time Computation Scalingarchitecture2510.24824ByteDance SeedOct 28, 2025score 9~123 min
- OctKimi Linear: An Expressive, Efficient Attention Architecturearchitecture2510.26692Moonshot / KimiOct 30, 2025~98 min
- OctLongCodeZip: Compress Long Context for Code Language Modelscode2510.00446Oct 1, 2025~119 min
- OctBoundary-Guided Policy Optimization for Memory-efficient RL of Diffusion Large Language Modelsinference-optimization2510.11683Z.aiOct 13, 2025score 9~120 min
- OctINT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatsinference-optimization2510.25602NVIDIAOct 29, 2025score 10~113 min
- OctWhen to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensemblingllm-systems2510.15346KAIST AIOct 17, 2025score 5~120 min
- OctTracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningreasoning2510.10494Microsoft ResearchOct 12, 2025score 9~120 min
- OctVerifier-free Test-Time Sampling for Vision Language Action Modelsrl-training2510.05681KAIST AIOct 7, 2025score 9~106 min
- OctAny-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depthsafety2510.18081ByteDance SeedOct 20, 2025score 8~133 min
- OctScaling Laws Meet Model Architecture: Toward Inference-Efficient LLMsscaling-laws2510.18245Oct 21, 2025~103 min
- OctScaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Searchserving2510.22101LinkedInOct 25, 2025~111 min
- OctRDMA Point-to-Point Communication for LLM Systemsserving2510.27656Oct 31, 2025~117 min
- OctScalable In-context Ranking with Generative Modelstraining-methods2510.05396DeepmindOct 6, 2025score 8~107 min
- OctBitNet Distillationtraining-methods2510.13998Microsoft ResearchOct 15, 2025score 9~127 min
- OctMemory Retrieval and Consolidation in Large Language Models through Function Tokensuncategorized2510.08203ByteDance SeedOct 9, 2025score 7~115 min
- OctThe End of Manual Decoding: Towards Truly End-to-End Language Modelsuncategorized2510.26697Oct 30, 2025~126 min
- SepAstra: A Multi-Agent System for GPU Kernel Performance Optimizationagents2509.07506Sep 9, 2025~102 min
- SepSLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attentionarchitecture2509.24006Tsinghua UniversitySep 28, 2025score 10~110 min
- SepInfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptationarchitecture2509.24663OpenBMBSep 29, 2025score 9~116 min
- SepSANA-Video: Efficient Video Generation with Block Linear Diffusion Transformerarchitecture2509.24695NVIDIASep 29, 2025score 8~105 min
- SepDC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Spacediffusion2509.25180NVIDIASep 29, 2025score 8~99 min
- SepSet Block Decoding is a Language Model Inference Acceleratorinference-optimization2509.04185Sep 4, 2025score 9~102 min
- SepBridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationinference-optimization2509.23202NVIDIASep 27, 2025score 9~99 min
- SepThe Pitfalls of KV Cache Compressioninference-optimization2510.00231Sep 30, 2025~125 min
- SepLongLive: Real-time Interactive Long Video Generationllm-systems2509.22622NVIDIASep 26, 2025score 6~111 min
- SepFast-dLLM v2: Efficient Block-Diffusion LLMllm-systems2509.26328Sep 30, 2025~114 min
- SepMetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactionmultimodal2509.18095Sep 22, 2025score 9~117 min
- SepDC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencodervision2509.25182NVIDIASep 29, 2025score 6~105 min
- AugSeed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecode2508.02193Aug 4, 2025score 9~115 min
- AugLess Is More: Training-Free Sparse Attention with Global Locality for Efficient Reasoninginference-optimization2508.07101Aug 9, 2025score 9~111 min
- AugInternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencymultimodal2508.18265Aug 25, 2025score 10~107 min
- AugNVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelreasoning2508.14444NVIDIAAug 20, 2025score 10~119 min
- AugTPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inferenceserving2508.15881DeepSeekAug 21, 2025score 9~117 min
- AugTaming the Chaos: Coordinated Autoscaling for Heterogeneous and Disaggregated LLM Inferenceserving2508.19559Aug 27, 2025score 9~115 min
- AugTreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modelingtraining-methods2508.17445ByteDance SeedAug 24, 2025score 9~105 min
- AugREINA: Regularized Entropy Information-Based Loss for Efficient Simultaneous Speech Translationuncategorized2508.04946Roblox CorporationAug 7, 2025score 2~114 min
- JulDecoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generationarchitecture2507.06607Microsoft ResearchJul 9, 2025score 9~109 min
- JulLocality-aware Parallel Decoding for Efficient Autoregressive Image Generationinference-optimization2507.01957Jul 2, 2025score 9~111 min
- JulThe Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmllm-systems2507.18553 IST Austria Distributed Algorithms and Systems LabJul 24, 2025score 9~149 min
- JulDoes More Inference-Time Compute Really Help Robustness?safety2507.15974DeepSeekJul 21, 2025~111 min
- JulHelix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decodingserving2507.07120Jul 7, 2025~95 min
- JulCUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningtraining-methods2507.14111Jul 18, 2025score 10~138 min
- JulARC-Hunyuan-Video-7B: Structured Video Comprehension of Real-World Shortstraining-methods2507.20939TencentJul 28, 2025score 9~125 min
- JunLog-Linear Attentionarchitecture2506.04761Jun 5, 2025~122 min
- JunMultiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationarchitecture2506.09991Jun 11, 2025score 9~114 min
- JunRectified Sparse Attentioninference-optimization2506.04108Jun 4, 2025score 9~102 min
- JunInference-Time Hyper-Scaling with KV Cache Compressioninference-optimization2506.05345NVIDIAJun 5, 2025score 9~102 min
- JunBeyond the Buzz: A Pragmatic Take on Inference Disaggregationserving2506.05508Jun 5, 2025~100 min
- JunSelf Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusiontraining-methods2506.08009Jun 9, 2025score 9~120 min
- JunSparseLoRA: Accelerating LLM Fine-Tuning with Contextual Sparsitytraining-methods2506.16500Jun 19, 2025score 9~103 min
- JunSeedVR2: One-Step Video Restoration via Diffusion Adversarial Post-Trainingvision2506.05301ByteDance SeedJun 5, 2025score 8~104 min
- JunCyberV: Cybernetics for Test-time Scaling in Video Understandingvision2506.07971ByteDanceJun 9, 2025score 8~125 min
- JunHunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Conditionvision2506.17201Tencent HunyuanJun 20, 2025score 2~116 min
- MayQwenLong-CPRS: Towards $\infty$-LLMs with Dynamic Context Optimizationcontext-optimization2505.18092May 23, 2025score 9~115 min
- MayVSA: Faster Video Diffusion with Trainable Sparse Attentiondiffusion2505.13389May 19, 2025score 9~105 min
- MayEfficientLLM: Efficiency in Large Language Modelsevaluation2505.13840May 20, 2025score 10~104 min
- MaySageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Traininginference-optimization2505.11594May 16, 2025score 10~110 min
- MaySparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationinference-optimization2505.18875May 24, 2025score 9~93 min
- MaySageAttention2++: A More Efficient Implementation of SageAttention2inference-optimization2505.21136May 27, 2025score 9~93 min
- MayFast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodinginference-optimization2505.22618May 28, 2025score 10~107 min
- MayParallel Scaling Law for Language Modelsscaling-laws2505.10475May 15, 2025~87 min
- MayPrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applicationsserving2505.07203May 12, 2025~105 min
- MayRADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scaletraining-methods2505.03005May 5, 2025score 9~133 min
- AprMulti-Token Attentionarchitecture2504.00927Apr 1, 2025~100 min
- AprBitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMsarchitecture2504.18415Apr 25, 2025score 9~90 min
- AprHogwild! Inference: Parallel LLM Generation via Concurrent Attentioninference-optimization2504.06261Apr 8, 2025score 9~116 min
- AprSleep-time Compute: Beyond Inference Scaling at Test-timeinference-optimization2504.13171Apr 17, 2025~127 min
- AprBitNet b1.58 2B4T Technical Reportllm-systems2504.12285Microsoft ResearchApr 16, 2025score 9~106 min
- AprSmolVLM: Redefining small and efficient multimodal modelsmultimodal2504.05299Hugging Face Smol ModelsApr 7, 2025~116 min
- AprLearning Adaptive Parallel Reasoning with Language Modelsreasoning2504.15466Apr 21, 2025~108 min
- AprMegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelismserving2504.02263Apr 3, 2025~102 min
- AprMSCCL++: Rethinking GPU Communication Abstractions for AI Inferenceserving2504.09014Apr 11, 2025~111 min
- AprMinitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruningtraining-methods2504.11409Apr 15, 2025score 9~124 min
- MarEAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testinference-optimization2503.01840Mar 3, 2025score 10~123 min
- MarQuantization for OpenAI's Whisper Models: A Comparative Analysisinference-optimization2503.09905OpenAIMar 12, 2025score 3~112 min
- FebNative Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentionarchitecture2502.11089DeepSeekFeb 16, 2025score 9~127 min
- FebMoBA: Mixture of Block Attention for Long-Context LLMsarchitecture2502.13189Moonshot AIFeb 18, 2025score 9~112 min
- FebKernelBench: Can LLMs Write Efficient GPU Kernels?evaluation2502.10517Feb 14, 2025~112 min
- FebCan 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scalinginference-optimization2502.06703Feb 10, 2025score 9~110 min
- FebTransMLA: Multi-Head Latent Attention Is All You Needinference-optimization2502.07864DeepSeekFeb 11, 2025score 10~101 min
- FebInfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPUinference-optimization2502.08910Feb 13, 2025score 10~130 min
- FebLServe: Efficient Long-sequence LLM Serving with Unified Sparse Attentioninference-optimization2502.14866Feb 20, 2025score 9~120 min
- FebChain of Draft: Thinking Faster by Writing Lessprompting2502.18600Zoom AIFeb 25, 2025score 8~88 min
- FebMaximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit Flipssafety2502.07408NVIDIAFeb 11, 2025~127 min
- FebMatryoshka Quantizationtraining-methods2502.06786Feb 10, 2025score 9~112 min
- JanEvolving Deeper LLM Thinkingagents2501.09891DeepMindJan 17, 2025~119 min
- JanTensor Product Attention Is All You Needarchitecture2501.06425math-aiJan 11, 2025score 9~113 min
- JanSigma: Differential Rescaling of Query, Key and Value for Efficient Language Modelsarchitecture2501.13629Jan 23, 2025score 9~111 min
- JanQwen2.5-1M Technical Reportcontext-optimization2501.15383Qwen / Alibaba CloudJan 26, 2025~114 min
- JanTrading Inference-Time Compute for Adversarial Robustnessreasoning2501.18841OpenAIJan 31, 2025score 10~141 min
- JanReward-Guided Speculative Decoding for Efficient LLM Reasoningreasoning2501.19324SalesforceJan 31, 2025score 9~129 min
- Jans1: Simple test-time scalingreasoning2501.19393OpenAIJan 31, 2025~112 min
- JanFlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingserving2501.01005Jan 2, 2025~136 min
- JanTransformer-Squared: Self-adaptive LLMstraining-methods2501.06252Jan 9, 2025score 9~115 min
2024
95- DecFLEX ATTENTION: A PROGRAMMING MODEL FOR GENERATING OPTIMIZED ATTENTION KERNELSarchitecture2412.05496Dec 7, 2024~116 min
- DecNVILA: Efficient Frontier Visual Language Modelsllm-systems2412.04468Dec 5, 2024~117 min
- DecEfficiently Serving LLM Reasoning Programs with Certaindexserving2412.20993Dec 30, 2024score 9~106 min
- DecParallelized Autoregressive Visual Generationvision2412.15119Dec 19, 2024score 9~103 min
- NovBitNet a4.8: 4-bit Activations for 1-bit LLMsarchitecture2411.04965Nov 7, 2024score 9~92 min
- NovHymba: A Hybrid-head Architecture for Small Language Modelsarchitecture2411.13676NVIDIANov 20, 2024score 9~102 min
- NovContext Parallelism for Scalable Million-Token Inferenceinference-optimization2411.01783Nov 4, 2024~130 min
- Nov"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantizationinference-optimization2411.02355DropboxNov 4, 2024score 10~118 min
- NovSmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformersinference-optimization2411.10510RobloxNov 15, 2024score 9~102 min
- NovSageAttention2 Technical Report: Accurate 4 Bit Attention for Plug-and-play Inference Accelerationinference-optimization2411.10958Nov 17, 2024score 9~109 min
- NovStar Attention: Efficient LLM Inference over Long Sequencesinference-optimization2411.17116Nov 26, 2024score 9~124 min
- NovMARCONI: PREFIX CACHING FOR THE ERA OF HYBRID LLMSserving2411.19379Nov 28, 2024~99 min
- NovCut Your Losses in Large-Vocabulary Language Modelstraining-methods2411.09009AppleNov 13, 2024score 9~114 min
- OctSelective Attention Improves Transformerarchitecture2410.02703Oct 3, 2024score 9~123 min
- OctAddition is All You Need for Energy-efficient Language Modelsinference-optimization2410.00907Oct 1, 2024score 9~109 min
- OctSageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationinference-optimization2410.02367Oct 3, 2024score 9~112 min
- OctPrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantizationinference-optimization2410.05265Oct 7, 2024score 9~117 min
- OctDuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headsinference-optimization2410.10819Oct 14, 2024score 9~125 min
- OctSeerAttention: Learning Intrinsic Sparse Attention in Your LLMsinference-optimization2410.13276Oct 17, 2024score 9~107 min
- OctA Survey of Small Language Modelsllm-systems2410.20011Oct 25, 2024~119 min
- OctEoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximationllm-systems2410.21271NVIDIAOct 28, 2024score 6~113 min
- OctInference Scaling for Long-Context Retrieval Augmented Generationreasoning2410.04343Oct 6, 2024~124 min
- OctA Comparative Study on Reasoning Patterns of OpenAI's o1 Modelreasoning2410.13639Oct 17, 2024score 9~110 min
- OctSwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformationserving2410.03960SnowflakeOct 4, 2024score 9~128 min
- OctOrca: A Distributed Serving System for Transformer-Based Generative Modelsserving2410.17840Oct 23, 2024~108 min
- OctREAD MOREserving2410.20399Together AIOct 27, 2024~108 min
- OctShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferenceserving2410.21465ByteDance SeedOct 28, 2024score 9~114 min
- OctLiger Kernel: Efficient Triton Kernels for LLM Trainingtraining-methods2410.10989Oct 14, 2024~124 min
- SepRetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalinference-optimization2409.10516Sep 16, 2024score 8~105 min
- SepMaskLLM: Learnable Semi-Structured Sparsity for Large Language Modelsllm-systems2409.17481Sep 26, 2024score 9~118 min
- SepVPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelslow-precision2409.17066Sep 25, 2024score 9~107 min
- AugThe Mamba in the Llama: Distilling and Accelerating Hybrid Modelsarchitecture2408.15237Aug 27, 2024~116 min
- AugFocusLLM: Precise Understanding of Long Context by Dynamic Condensingcontext-optimization2408.11745Aug 21, 2024score 9~135 min
- AugScaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parametersinference-optimization2408.03314Aug 6, 2024score 10~122 min
- AugWriting in the Margins: Better Inference Pattern for Long Context Retrievalinference-optimization2408.14906Aug 27, 2024score 8~108 min
- AugNanoFlow: Towards Optimal Large Language Model Serving Throughputserving2408.12757Aug 22, 2024~108 min
- AugLLM Pruning and Distillation in Practice: The Minitron Approachtraining-methods2408.11796Aug 21, 2024~110 min
- JulGoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compressionarchitecture2407.12077Jul 16, 2024score 9~124 min
- JulMInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioninference-optimization2407.02490Qwen / Alibaba CloudJul 2, 2024score 9~128 min
- JulInference Performance Optimization for Large Language Models on CPUsinference-optimization2407.07304Jul 10, 2024score 9~98 min
- JulFlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioninference-optimization2407.08608DropboxJul 11, 2024~111 min
- JulHuman-inspired Episodic Memory for Infinite Context LLMsinference-optimization2407.09450Jul 12, 2024score 9~127 min
- JulLazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inferenceinference-optimization2407.14057AppleJul 19, 2024score 9~100 min
- JulAn Efficient Inference Framework for Early-exit Large Language Modelsinference-optimization2407.20272Jul 25, 2024~103 min
- JulLarge Language Monkeys: Scaling Inference Compute with Repeated Samplingscaling-laws2407.21787Jul 31, 2024~130 min
- JulLiNR: Model Based Neural Retrieval on GPUs at LinkedInserving2407.13218Jul 18, 2024~132 min
- JunBlock Transformer: Global-to-Local Language Modeling for Fast Inferencearchitecture2406.02657Jun 4, 2024score 9~100 min
- JunSamba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingarchitecture2406.07522Jun 11, 2024score 9~115 min
- JunA Simple and Effective $L_2$ Norm-Based Strategy for KV Cache Compressioninference-optimization2406.11430Jun 17, 2024score 8~93 min
- JunInfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementinference-optimization2406.19707Jun 28, 2024~107 min
- JunT-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgeinference-optimization2407.00088Jun 25, 2024score 8~124 min
- JunLlumnix: Dynamic Scheduling for Large Language Model Servingserving2406.03243Jun 5, 2024~113 min
- JunPowerInfer-2: Fast Large Language Model Inference on a Smartphoneserving2406.06282Jun 10, 2024score 8~108 min
- JunMooncake: A KVCache-centric Disaggregated Architecture for LLM Servingserving2407.00079Jun 24, 2024~127 min
- MayLayer-Condensed KV Cache for Efficient Inference of Large Language Modelsarchitecture2405.10637May 17, 2024score 9~96 min
- MayReducing Transformer Key-Value Cache Size with Cross-Layer Attentionarchitecture2405.12981May 21, 2024score 9~103 min
- MayClover: Regressive Lightweight Speculative Decoding with Sequential Knowledgeinference-optimization2405.00263May 1, 2024score 9~105 min
- MayDistributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inferenceinference-optimization2405.14105May 23, 2024score 9~89 min
- MayNearest Neighbor Speculative Decoding for LLM Generation and Attributionllm-systems2405.19325May 29, 2024score 9~103 min
- MayParrot: Efficient Serving of LLM-based Applications with Semantic Variableserving2405.19888May 30, 2024score 9~98 min
- AprMixture-of-Depths: Dynamically allocating compute in transformer-based language modelsarchitecture2404.02258Apr 2, 2024score 9~101 min
- AprLeave No Context Behind: Efficient Infinite Context Transformers with Infini-attentionarchitecture2404.07143Apr 10, 2024score 9~113 min
- AprLayerSkip: Enabling Early Exit Inference and Self-Speculative Decodinginference-optimization2404.16710Apr 25, 2024score 9~107 min
- AprKangaroo: Lossless Self-Speculative Decoding via Double Early Exitinginference-optimization2404.18911HUAWEI Noah's Ark LabApr 29, 2024score 9~106 min
- AprRecurrentGemma: Moving Past Transformers for Efficient Open Language Modelsllm-systems2404.07839Apr 11, 2024score 10~108 min
- AprJetMoE: Reaching Llama2 Performance with 0.1M Dollarsmoe2404.07413Apr 11, 2024score 9~79 min
- AprBetter & Faster Large Language Models via Multi-token Predictiontraining-methods2404.19737Apr 30, 2024~103 min
- MarLLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncontext-optimization2403.12968Microsoft ResearchMar 19, 2024score 8~113 min
- MarQuaRot: Outlier-Free 4-Bit Inference in Rotated LLMsinference-optimization2404.00456Mar 30, 2024~106 min
- MarRecurrent Drafter for Fast Speculative Decoding in Large Language Modelsllm-systems2403.09919AppleMar 14, 2024~125 min
- MarThe Unreasonable Ineffectiveness of the Deeper Layerspretraining2403.17887Mar 26, 2024score 9~100 min
- MarTaming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serveserving2403.02310Mar 4, 2024~86 min
- FebMobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casesarchitecture2402.14905Meta AI / FAIRFeb 22, 2024score 10~111 min
- FebSimple linear attention language models balance the recall-throughput tradeoffarchitecture2402.18668Together AIFeb 28, 2024score 9~112 min
- FebGriffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelsarchitecture2402.19427Google ResearchFeb 29, 2024score 9~123 min
- FebBreak the Sequential Dependency of LLM Inference Using LOOKAHEAD DECODINGinference-optimization2402.02057Feb 3, 2024~122 min
- FebSpeculative Streaming: Fast LLM Inference without Auxiliary Modelsinference-optimization2402.11131AppleFeb 16, 2024score 9~100 min
- FebOuroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decodinginference-optimization2402.13720Feb 21, 2024score 9~105 min
- FebLiRank: Industrial Large Scale Ranking Models at LinkedInllm-systems2402.06859Feb 10, 2024score 3~133 min
- FebSelf-Discover: Large Language Models Self-Compose Reasoning Structuresprompting2402.03620DeepMindFeb 6, 2024score 9~88 min
- FebChain-of-Thought Reasoning without Promptingreasoning2402.10200Feb 15, 2024~113 min
- FebHydragen: High-Throughput LLM Inference with Shared Prefixesserving2402.05099Feb 7, 2024score 9~105 min
- FebChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitionserving2402.15220Feb 23, 2024score 10~108 min
- FebTraining-Free Long-Context Scaling of Large Language Modelstraining-methods2402.17463Qwen / Alibaba CloudFeb 27, 2024score 9~121 min
- JanSliceGPT: Compress Large Language Models by Deleting Rows and Columnsarchitecture2401.15024Jan 26, 2024score 8~122 min
- JanTransformers are Multi-State RNNsinference-optimization2401.06104Jan 11, 2024score 9~93 min
- JanAPAR: LLMs Can Do Auto-Parallel Auto-Regressive Decodinginference-optimization2401.06761Zhipu / GLMJan 12, 2024~109 min
- JanMedusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headsinference-optimization2401.10774Jan 19, 2024score 10~108 min
- JanEAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintyinference-optimization2401.15077Jan 26, 2024~133 min
- JanSoaring from 4K to 400K: Extending LLM's Context with Activation Beaconllm-systems2401.03462Jan 7, 2024score 9~109 min
- JanInference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloadsllm-systems2401.11181Jan 20, 2024~115 min
- JanDistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingserving2401.09670Jan 18, 2024~102 min
- JanFP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Designserving2401.14112Jan 25, 2024score 9~96 min
- JanLightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Modelstraining-methods2401.04658Jan 9, 2024score 9~118 min
- JanTuning Language Models by Proxytraining-methods2401.08565Jan 16, 2024~104 min
2023
56- DecSGLang: Efficient Execution of Structured Language Model Programsagents2312.07104Dec 12, 2023~101 min
- DecCodestral Mambaarchitecture2312.00752MistralDec 1, 2023~109 min
- DecSparQ Attention: Bandwidth-Efficient LLM Inferenceinference-optimization2312.04985Dec 8, 2023score 9~132 min
- DecCascade Speculative Drafting for Even Faster LLM Inferenceinference-optimization2312.11462Dec 18, 2023score 9~101 min
- DecLLM in a Flash: Efficient Large Language Model Inference with Limited Memoryinference-optimization2312.11514AppleDec 12, 2023score 10~117 min
- DecZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskslow-precision2312.08583Dec 14, 2023score 9~123 min
- DecSwitchHead: Accelerating Transformers with Mixture-of-Experts Attentionmoe2312.07987Dec 13, 2023score 9~109 min
- DecBeyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawsscaling-laws2401.00448Dec 31, 2023score 9~99 min
- NovFlashDecoding++: Faster Large Language Model Inference on GPUsinference-optimization2311.01282Nov 2, 2023score 9~129 min
- NovPrompt Cache: Modular Attention Reuse for Low-Latency Inferenceinference-optimization2311.04934Nov 7, 2023score 9~118 min
- NovFlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Coresinference-optimization2311.05908Together AINov 10, 2023score 9~128 min
- NovFast Chain-of-Thought: A Glance of Future from Parallel Decoding Leads to Answers Fasterinference-optimization2311.08263Nov 14, 2023score 9~102 min
- NovRelax: Composable Abstractions for End-to-End Dynamic Machine Learningserving2311.02103Nov 1, 2023~107 min
- NovS-LORA: SERVING THOUSANDS OF CONCURRENT LORA ADAPTERSserving2311.03285Nov 6, 2023~106 min
- OctControlled Decoding from Language Modelsalignment2310.17022Oct 25, 2023score 9~110 min
- OctLLM-FP4: 4-Bit Floating-Point Quantized Transformersinference-optimization2310.16836Oct 25, 2023score 9~104 min
- OctREAD MOREinference-optimization2310.17157Together AIOct 26, 2023score 9~117 min
- OctRing Attention with Blockwise Transformers for Near-Infinite Contextllm-systems2310.01889Oct 3, 2023~101 min
- OctMicroscaling Data Formats for Deep Learninglow-precision2310.10537Oct 16, 2023score 9~124 min
- OctQMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelsmoe2310.16795Oct 25, 2023score 9~111 min
- OctMistral 7Bpretraining2310.06825MistralOct 10, 2023~97 min
- OctPUNICA: MULTI-TENANT LORA SERVINGserving2310.18547Oct 28, 2023~100 min
- SepDoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelsinference-optimization2309.03883Sep 7, 2023score 9~113 min
- SepEfficient Post-training Quantization with FP8 Formatsinference-optimization2309.14592Sep 26, 2023score 9~126 min
- SepEFFICIENT STREAMING LANGUAGE MODELS WITH ATTENTION SINKSinference-optimization2309.17453Sep 29, 2023~109 min
- SepEfficient Memory Management for Large Language Model Serving with PagedAttentionserving2309.06180Sep 12, 2023score 10~122 min
- AugFLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Searchinference-optimization2308.03290Aug 7, 2023score 8~119 min
- AugAccelerating LLM Inference with Staged Speculative Decodinginference-optimization2308.04623Aug 8, 2023~104 min
- AugOmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelsinference-optimization2308.13137Aug 25, 2023score 9~100 min
- AugLM-Infinite: Simple On-the-Fly Length Generalization for Large Language Modelsinference-optimization2308.16137Aug 30, 2023score 9~105 min
- AugSARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillsllm-systems2308.16369Aug 31, 2023~105 min
- JulRetentive Network: A Successor to Transformer for Large Language Modelsarchitecture2307.08621Jul 17, 2023score 9~130 min
- JulIn-context Autoencoder for Context Compression in a Large Language Modelcontext-optimization2307.06945Jul 13, 2023score 9~107 min
- JulSkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inferenceinference-optimization2307.02628Jul 5, 2023score 9~96 min
- JulFlashAttention-2: Faster Attention with Better Parallelism and Work Partitioninginference-optimization2307.08691Jul 17, 2023~110 min
- JulSkeleton-of-Thought: Large Language Models Can Do Parallel Decodingllm-systems2307.15337Jul 28, 2023score 9~96 min
- JulSelf-consistency for open-ended generationsreasoning2307.06857Jul 11, 2023score 8~118 min
- JunFaster Causal Attention Over Large Sequences Through Sparse Flash Attentionarchitecture2306.01160Jun 1, 2023score 9~101 min
- JunAWQ: Activation-Aware Weight Quantization for On-Device LLM Compression and Accelerationinference-optimization2306.00978Microsoft ResearchJun 1, 2023~110 min
- JunSqueezeLLM: Dense-and-Sparse Quantizationinference-optimization2306.07629Jun 13, 2023~109 min
- JunRead Moreinference-optimization2306.17806EleutherAIJun 30, 2023~115 min
- JunAugmenting Language Models with Long-Term Memoryllm-systems2306.07174Jun 12, 2023score 9~107 min
- JunBinary and Ternary Natural Language Generationlow-precision2306.01841Jun 2, 2023score 9~115 min
- JunH_2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelsserving2306.14048Jun 24, 2023score 9~87 min
- JunOn-Policy Distillation of Language Models: Learning from Self-Generated Mistakestraining-methods2306.13649DeepMindJun 23, 2023score 10~116 min
- MayRead Morearchitecture2305.13048EleutherAIMay 22, 2023score 9~106 min
- MayGQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointsarchitecture2305.13245May 22, 2023~87 min
- MayScaling Catalog Attribute Extraction with Multi-modal LLMsllm-systems2305.05176InstacartMay 9, 2023score 8~97 min
- MayCheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIsserving2305.02440May 3, 2023score 9~111 min
- MayFast Distributed Inference Serving for Large Language Modelsserving2305.05920May 10, 2023~112 min
- MaySpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verificationserving2305.09781May 16, 2023~113 min
- MayQLoRA: Efficient Finetuning of Quantized LLMstraining-methods2305.14314AI21May 23, 2023score 9~103 min
- MayFine-Tuning Language Models with Just Forward Passestraining-methods2305.17333May 27, 2023score 9~112 min
- AprRethinking the Role of Token Retrieval in Multi-Vector Retrievalretrieval2304.01982Apr 4, 2023~101 min
- MarREAD MOREinference-optimization2303.06865Together AIMar 13, 2023~115 min
- MarRequirement Adherence: Boosting Data Labeling Quality Using LLMsreasoning2303.17651UberMar 30, 2023~96 min
2022
9- NovSmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelsinference-optimization2211.10438Nov 18, 2022~96 min
- NovFast Inference from Transformers via Speculative Decodinginference-optimization2211.17192Nov 30, 2022~104 min
- NovEFFICIENTLY SCALING TRANSFORMER INFERENCEserving2211.05102Nov 9, 2022~147 min
- AugLLM.int8(): 8-bit Matrix Multiplication for Transformers at Scaleinference-optimization2208.07339Aug 15, 2022~109 min
- JunZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformersinference-optimization2206.01861Jun 4, 2022~92 min
- JunTutel: Adaptive Mixture-of-Experts at Scalemoe2206.03382Jun 7, 2022~114 min
- JunDeepSpeed-Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scaleserving2207.00032Jun 30, 2022~127 min
- MayOptimizing Test-Time Query Representations for Dense Retrievalinference-optimization2205.12680May 25, 2022~134 min
- MayREAD MOREinference-optimization2205.14135Together AIMay 27, 2022~127 min
2021
12020
7- OctDenoising Diffusion Implicit Modelsdiffusion2010.02502Oct 6, 2020~116 min
- OctFastFormers: Highly Efficient Transformer Models for Natural Language Understandinginference-optimization2010.133825 citesOct 26, 2020~122 min
- JunDeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameterstraining-methods2006.05525Jun 9, 2020~126 min
- AprLexically Constrained Neural Machine Translation with Levenshtein Transformerinference-optimization2004.12681Apr 27, 2020~95 min
- AprColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTretrieval2004.12832Apr 27, 2020~118 min
- FebBERT-of-Theseus: Compressing BERT by Progressive Module Replacingtraining-methods2002.02925Feb 7, 2020~113 min
- JanReformer: The Efficient Transformerarchitecture2001.04451323 citesJan 13, 2020~104 min
2019
6- NovGeneralization through Memorization: Nearest Neighbor Language Modelsretrieval1911.00172Nov 1, 2019~114 min
- OctQ8BERT: Quantized 8Bit BERTinference-optimization1910.06188Oct 14, 2019~91 min
- OctLudwig v0.3 Introduces Hyperparameter Optimization, Transformers and TensorFlow 2 supportpretraining1910.01108UberOct 2, 2019~108 min
- SepQ-BERT: Hessian Based Ultra Low Precision Quantization of BERTinference-optimization1909.05840Sep 12, 2019~112 min
- AprGenerating Long Sequences with Sparse Transformersarchitecture1904.10509Apr 23, 2019~125 min
- AprHAWQ: Hessian Aware Quantization of Neural Networkslow-precision1905.03696Apr 29, 2019~102 min
2018
3- JunQuantizing deep convolutional networks for efficient inference: A whitepaperlow-precision1806.08342Jun 21, 2018~102 min
- MarYear in Review: 2019 Highlights from the Uber Engineering Blogtraining-methods1803.03635UberMar 9, 2018~110 min
- FebTVM: An Automated End-to-End Optimizing Compiler for Deep Learninginference-optimization1802.04799Feb 12, 2018~120 min
2017
3- DecQuantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencelow-precision1712.05877200 citesDec 15, 2017~113 min
- DecTensorFlow-Serving: Flexible, High-Performance ML Servingserving1712.06139Dec 17, 2017~101 min
- OctTo prune, or not to prune: exploring the efficacy of pruning for model compressioninference-optimization1710.01878Oct 5, 2017~104 min
2016
7- DecClipper: A Low-Latency Online Prediction Serving Systemserving1612.03079Dec 9, 2016~118 min
- SepGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationarchitecture1609.08144Sep 26, 2016~114 min
- AugPruning Filters for Efficient ConvNetsarchitecture1608.08710Aug 31, 2016~105 min
- JulOn the efficient representation and execution of deep acoustic modelsinference-optimization1607.046836 citesJul 15, 2016~107 min
- FebEIE: Efficient Inference Engine on Compressed Deep Neural Networkarchitecture1602.01528Feb 4, 2016~119 min
- FebBinarized Neural Networkslow-precision1602.02830Feb 9, 2016~111 min
- FebSqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model sizevision1602.07360Feb 24, 2016~121 min