Long context
50 papers in this thread, across 16 domains.
Progress0 of 48
- 2026Randomized YaRN Improves Length Generalization for Long-Context Reasoningno summary yetcs cl2606.23687NYU0 citesJun 22, 2026cs cl
- 2026UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic SparsificationInference Optimization2605.06221TencentMay 7, 2026~123 minInference Optimization
- 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K ContextMultimodal2605.13831ByteDance SeedMay 13, 2026~98 minMultimodal
- 2026EndPrompt: Efficient Long-Context Extension via Terminal AnchoringTraining Methods2605.14589BAIDUMay 14, 2026~127 minTraining Methods
- 2026LoGeR: Long-Context Geometric Reconstruction with Hybrid MemoryVision2603.03269DeepmindMar 3, 2026score 9~132 minVision
- 2026MiniCPM-SALA: Hybridizing Sparse and Linear Attention for Efficient Long-Context ModelingArchitecture2602.11761Feb 12, 2026~104 minArchitecture
- 2026LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse DecodingInference Optimization2602.04541Feb 4, 2026~104 minInference Optimization
- 2026When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context ReasoningLLM Systems2602.10560ByteDance SeedFeb 11, 2026score 8~103 minLLM Systems
- 2026Hybrid Linear Attention Done Right: Efficient Distillation and Effective Architectures for Extremely Long ContextsArchitecture2601.22156OpenBMBJan 29, 2026score 9~127 minArchitecture
- 2025Kascade: A Practical Sparse Attention Method for Long-Context LLM InferenceInference Optimization2512.16391Dec 18, 2025~109 minInference Optimization
- 2025QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory ManagementTraining Methods2512.12967Dec 15, 2025~110 minTraining Methods
- 2025Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode SupervisionTraining Methods2512.15489NVIDIADec 17, 2025score 8~103 minTraining Methods
- 2025End-to-End Test-Time Training for Long ContextTraining Methods2512.23675Dec 29, 2025~120 minTraining Methods
- 2025Artificial Hippocampus Networks for Efficient Long-Context ModelingArchitecture2510.07318ByteDance SeedOct 8, 2025score 9~123 minArchitecture
- 2025Every Attention Matters: An Efficient Hybrid Architecture for Long-Context ReasoningArchitecture2510.19338Ant GroupOct 22, 2025score 10~141 minArchitecture
- 2025LongCodeZip: Compress Long Context for Code Language ModelsCode2510.00446Oct 1, 2025~119 minCode
- 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long ContextsReasoning2510.19363Microsoft ResearchOct 22, 2025score 9~114 minReasoning
- 2025DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context ParallelismTraining Methods2510.10620Oct 12, 2025~99 minTraining Methods
- 2025Efficient Long-context Language Model Training by Core Attention DisaggregationTraining Methods2510.18121Oct 20, 2025score 10~105 minTraining Methods
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic CapabilitiesLLM Systems2507.06261Jul 7, 2025score 10~104 minLLM Systems
- 2025When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition FrameworkAlignment2506.16411Together AIJun 19, 2025~132 minAlignment
- 2025LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMsInference Optimization2506.14429Jun 17, 2025score 9~112 minInference Optimization
- 2025QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement LearningRL Training2505.17667May 23, 2025score 9~118 minRL Training
- 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language ModelsMultimodal2504.15271NVIDIAApr 21, 2025score 9~103 minMultimodal
- 2025A Comprehensive Survey on Long Context Language ModelingLLM Systems2503.17407Mar 20, 2025~121 minLLM Systems
- 2025MoBA: Mixture of Block Attention for Long-Context LLMsArchitecture2502.13189Moonshot AIFeb 18, 2025score 9~112 minArchitecture
- 2024When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context TrainingArchitecture2411.13476Nov 20, 2024score 8~114 minArchitecture
- 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming HeadsInference Optimization2410.10819Oct 14, 2024score 9~125 minInference Optimization
- 2024Inference Scaling for Long-Context Retrieval Augmented GenerationReasoning2410.04343Oct 6, 2024~124 minReasoning
- 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM InferenceServing2410.21465ByteDance SeedOct 28, 2024score 9~114 minServing
- 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure QueriesEvaluation2409.12640DeepMindSep 19, 2024~114 minEvaluation
- 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector RetrievalInference Optimization2409.10516Sep 16, 2024score 8~105 minInference Optimization
- 2024A Controlled Study on Long Context Extension and Generalization in LLMsTraining Methods2409.12181Sep 18, 2024~146 minTraining Methods
- 2024Dolphin: Long Context as a New Modality for Energy-Efficient On-Device Language ModelsAgents2408.15518Aug 28, 2024score 10~87 minAgents
- 2024FocusLLM: Precise Understanding of Long Context by Dynamic CondensingContext Optimization2408.11745Aug 21, 2024score 9~135 minContext Optimization
- 2024Writing in the Margins: Better Inference Pattern for Long Context RetrievalInference Optimization2408.14906Aug 27, 2024score 8~108 minInference Optimization
- 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse AttentionInference Optimization2407.02490Qwen / Alibaba CloudJul 2, 2024score 9~128 minInference Optimization
- 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM InferenceInference Optimization2407.14057AppleJul 19, 2024score 9~100 minInference Optimization
- 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and OutputMultimodal2407.03320InternLM / Shanghai AI LabJul 3, 2024score 9~105 minMultimodal
- 2024Scaling Granite Code Models to 128K ContextTraining Methods2407.13739IBM ResearchJul 18, 2024score 8~113 minTraining Methods
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?Evaluation2404.06654Qwen / Alibaba CloudApr 9, 2024score 10~109 minEvaluation
- 2024A Human-Inspired Reading Agent with Gist Memory of Very Long ContextsAgents2402.09727DeepMindFeb 15, 2024score 9~103 minAgents
- 2024Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERTRetrieval2402.07440Together AIFeb 12, 2024~97 minRetrieval
- 2024Training-Free Long-Context Scaling of Large Language ModelsTraining Methods2402.17463Qwen / Alibaba CloudFeb 27, 2024score 9~121 minTraining Methods
- 2024Soaring from 4K to 400K: Extending LLM's Context with Activation BeaconLLM Systems2401.03462Jan 7, 2024score 9~109 minLLM Systems
- 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language ModelsTraining Methods2309.12307Sep 21, 2023score 9~93 minTraining Methods
- 2023Effective Long-Context Scaling of Foundation ModelsTraining Methods2309.16039Sep 27, 2023score 9~104 minTraining Methods
- 2023YARN: EFFICIENT CONTEXT WINDOW EXTENSION OF LARGE LANGUAGE MODELSTraining Methods2309.00071Qwen / Alibaba CloudAug 31, 2023~112 minTraining Methods
- 2023Lost in the Middle: How Language Models Use Long ContextsContext Optimization2307.03172Together AIJul 6, 2023score 9~101 minContext Optimization
- 2021Adapting Long Context NLM for ASR Rescoring in Conversational Agentsno summary yetcs cl2104.11070Amazon1 citesApr 21, 2021cs cl